The problem with these casual relationships is there is always the dnager of one f you developing feelings for the other (i mean sex is a pretty intimite thing so its not surprising). If he has truely found his perfect match, then you probably are looking at rejection. I'm tempted to say that maybe you put yourself ina position where he could just use you for sex, but that wouldn't be fair as i don't know the full story.
It is up to you whether yo want to take the chance and tell him, but if your relationship is only based on sex, i don't think it would go that far, "proper" relationships are about more than just sex.In my opinion I think you'd be better off moving on, forget about this guy, and sooner or later you'll meet someone better who wants you for all of you, not just your body.
take care