Firstly, it's not pathetic to still be affected by something that happened a few years ago.
You say you were teased for being too unattractive for this guy. That is quite a hurtful thing to tease someone about so I understand how it can hurt someone for a long time afterward. It sounds like it's affected your confidence and made you question your self-worth.
You also mention the girl who I assume you were friendly with prior to the bullying. Betrayal can stay with a person for a long time so don't worry about it.
My advice to you would be to work on yourself, try to improve yourself as a person so that you can become more confident and assured in yourself. Don't question if it's pathetic to still be bothered by this bullying, you have feelings, it's normal to still be hurt a long time after having your feelings hurt. Some people can get over things quickly, others can not and are more sensitive.
You are 20, you are very young and have loads of time for relationships, try to focus on yourself for now. Set a fitness goal, travel, meet new people, try to challenge yourself in as many aspects of your life you can. I think once you see some progression your confidence will improve and this will all be a footnote.