Damn, I turn my back for a few hours and people go and have an interesting convo without me!
Attraction can be all about sex (which it is at base, assuming we're outside DnD religion so we're accepting evolution) and still include non-physical stuff, as Gravastar said. On completely biased, anecdotal evidence, I'd guess that humour and confidence can swing things strongly in someones favour, intelligence depends on the kind of intelligence (worth a lot more if you're articulate), and 'kindness' and 'being a nice guy' are overrated for attracting someone in the first place, but will probably matter a lot if you're talking about hanging on to a relationship once you've got that far. And saying that relationships between men and women are all about sex is a bit wide of the mark unless you broaden 'sex' to 'reproduction in general'. Remember that the reproductive impulse involves more than just looking for the right set of genes to pass on to the child - for women in particular, there's also a push towards looking for a partner who is going to be an effective protector/provider for the child/children.
Dragon Girl, thats how most people react to Freud. I can think of no more convincing evidence that he was on to something.