While I'm not suggesting that you should be with the man or even that he will be a good father, I think it would be morally repugnant to make these choices without his input.
By the sounds of your post the question here should not be whether you should have the child but rather who should have custody. If you want a child then you should share custody, if you don't but he wants the child then he should be granted full custody as you get on with your life.