Hmm in my experience friends or people who are not as involved in the situation tend to see the guy who's swept you off your feet for what he really is BUT- you describe your friend as being someone who exagerrates. I personally wouldn't believe your 'ex' is going around saying all of that without checking first- maybe you should just ask him straight up. Also, it is a bit different in your case if you know things about him which others don;t which could help you get a fuller picture of him than your friends.
Having said all of this, I would be careful anyway whether your friends are right or wrong- because you care for the guy, even though you don't want to get involved, I think you could still end up getting hurt- he obviously does like you, but if he's just come out of a relationship maybe it's a rebound thing, and if it is you'll be the one hurt. But this doesn't mean you can't continue your friendship, it just means you need to be extra careful and make it clear to him that you're just friends and there must be no more kisses, etc, at least while he's getting over his ex. If you and him are friends though, you should be able to ask him about what he has been saying about you, and why. If you're really great friends you should be able to work through this- maybe your friendship won't be the same as it has been in the past, or as close, but it doen't mean it has to be nonexistent.
Finally, after talking to him (about what he's said, and remaining 'just friends') you should go with your own instincts, they are usually right when not clouded by emotions! Just make sure if you start to feel like he's more than just a friend, to back off just a little. Hope it works out for you x