You should definitely talk to him ASAP about the concerns you have. You have to keep in mind that he is a father, and he cannot abandoned his child for another woman. If he did, it will damage the child in the long run mentally. My dad was the one who raised me until i was 11, when he stopped trying to have a decent relationship with me. The reason why was because his girlfriend didn't like that I was his first priority. Ever since I've struggled to form long lasting relationships, and friendships with people in fear I'll end up abandoned. I have no doubts that your boyfriend's child will feel the same if it was to happen to them. So for the child's sake, please remember that. Now I'm not sure why you don't want a man with kids, but I'm sure you have your reasons, so I think You should most definitely have a calm conversation with him about it and just tell him how you feel. I thought giving you what the child's perspective might be would give you some food for thought.