Unless it's someone I've known, or spent any decent amount of time around (like at work, or school) I really have nothing to go off of besides their outer appearance and maybe catching glimpses into their personality. To approach someone who I'd like to get to know/ask on a date, I have to have some kind of physical attraction to spark my curiosity. It may start as something as small as seeing a band name on their shirt that I dig to get me looking, but then catching the sound of their laugh and how cute their smile is can gain my attention.
I definitely think there has to be a physical attraction, later you may find their personality, personal habits or what have you aren't attractive and that fades the curiosity, but everyone has some physical attribute that is attractive to someone else.