the most important thing is how much you like that guy, physical attraction is only one of the other things that follow. if you don't like him enough, then no matter how gorgeous he looks, you still wouldn't feel as happy with him as with some you do like a lot, even if he's not as physically attractive. and often, you will find that the more you realise the good qualities of a person, the more attractive he becomes because you can see a side of him that you couldn't before, and that would change the way he looks to you.
afterall, beauty is in the eye of the beholder. if you really do like him, go for it, and don't be bounded by the physical factors.