At the end of the day, you eat what you want to eat. Reducing your meat intake has never been easier with many restaurants, cafes and fast food places catering for at least vegetarian if not completely vegan.
Personally, I'm about 95% whole-foods plant-based and by that, I mean I allow myself selected types of meat (oily fish, offel) 1-2 times a week for their nutritional benefits (Omega3, B12 etc) rather than supplementing or if I go out and there isn't anything on the menu I want that is plant-based (not very often). The rest of the time I eat plant-based.
As for which animal is hurt the most? Well, they're all killed. All animals feel pain and they all experience fear. Mass bred animals that are farmed cheaply for their meat/milk tend to go through more pain than those that are kept and butchered by smaller, local farms which is why sourcing local meat tends to be recommended (better for the environment too). But, in my opinion, chickens and dairy animals do have it worse. If you want to know more then Land of Hope and Glory, which you can watch on YouTube, is a heartbreaking but informative documentary.