You forget that these other guys may have other desirable traits. For example, intelligence, or a good sense of humor. So, even if they do not have some traits such as being tall, they could have others that are attractive. Of course, you could have a tall person who also has all of these attractive traits, so logically they would be the best option to go for. However, human relationships don't work that way.