If you find them attractive, you've been on a few dates, they have your preferred set of genetalia and everything's going great up until you find out they were born a different sex to the one they now present as, at which point you reject them then yeah that's clearly transphobic. Sorry y'all, but unless you can give me another reason other than you find it creepy/weird/unnatural then it's clearly transphobic.
You can't compare it to trying to make a homosexual have a heterosexual relationship, because there's a fundamental mismatch of preferred genetalia.
Rejecting a pre-op person, is sad for the person who's being rejected, but understandable. Not everyone can get on board with all genetalia, and that's just personal preference. Can't be helped.
If you're rejecting someone who you had no idea was trans, until the moment they told you, then that's transphobia and you should feel bad.
The only exception I can think of, where it wouldn't be reacting based on prejudice and phobias, might be if you desperately wanted your own kids, that was your medium-term goal, and you'd reject an cis-preffered-sex person for being infertile/not wanting kids too too. Then it's just a goal mismatch.