I don't really care for children, so I'm not sure if I'm biased when I say most babies are pretty ugly, especially in the first few days and weeks, before their skin plumps out a bit. I love my niece, but when I first saw a photo of her I was worried I'd end up saying something that would upset her parents. I didn't, I kept my mouth shut and now she's months old she is very cute.
What bothers me the most about what your friend said is that he chose to say 'cute factor' rather than just 'not cute'.