from what i see, that would be the view of someone who had then been betrayed/upset by the person they lost their virginity to.
a few case studies to back this up:
one friend of mine lost their virginity to someone they had liked for a really long time. the circumstances werent great (nasty location, not in the relationship that she so desperately wanted, etc) and afterwards, she felt used. that put her off sleeping with guys later because she began to distrust them.
another friend was in a long term relationship with the guy she lost hers to, although they slept together very early on in the relationship, they stayed together for a couple of years afterward. she has no hard feelings about losing her virginity and only sees it as a change in maturity and knowledge of herself.
and myself, i lost my virginity to someone i was in a relationship with, who broke up with me 2 months later. i was distraught, but have never regretted sleeping with him. the only thing i think has changed in me is the fact that i had sex. before, i didnt know what sex was like. afterward, i did. thats as simple as it is.
dont let your friends scare you off. things like this are different for everyone. losing your virginity is a very personal thing (and not something to be taken lightly in todays social climate).
i personally think its very unhealthy that your friends feel this way. i doubt that the circumstances surrounding their first times were favourable.