It doesn't sound like a functional relationship to me. I'm a guy and I was in your position years ago. My gf then had many guy friends who acted inappropriately. And she made me feel so terrible for starting arguments with her about it, said that I was so insecure and crazy. She eventually broke up with me and left me feeling like the breakdown of the relationship was entirely my fault. Several months later I found out she had in fact, been frequently cheating on me. All that time she made me feel so awful for my suspicions, i was actually right on the money!
The moral of the story is just go with your gut. Even if your partner is good at turning it around on you and making you feel like it is your fault. There are 3 possible scenarios if you leave him and you win in each of them:
1. He is also behaving inappropriately and still cheating, in which case it is pretty clear cut, leaving him was a good choice!
2. He isn't cheating, but he still hangs around with a bunch of female friends who behave inappropriately, and he knows it bothers you. Yet he still does it. Not very considerate is it? It's making you unhappy and there is no point in staying in a relationship that makes you unhappy. Don't do what I did and try to stick it out for months. You'll never get that time back.
3. Nothing inappropriate is going on and you are just paranoid. This is more difficult to discern but walking away will also benefit you in this scenario. You still aren't happy. You can try again with someone who doesn't have a whole load of female friends after a good long period of being single where you can examine yourself and work on your flaws. But yeah, it's not so easy for someone to figure out if this is actually what is happening. If it becomes a trend over a several relationships, then yeah, maybe you are paranoid/psycho. But you seem pretty reasonable and self-aware. I personally don't think this is your scenario. It's either 1 or 2.
The trick to relationships is knowing when to walk away. Walking away takes self-esteem and self-worth that a lot of people don't have. I certainly didn't and it cost me not just my enjoyment of my first year of uni. But if you manage to do it, it's honestly the most powerful move you can make. Maybe he'll see the error of his ways and come back to you and stop hanging around with them. But if he slides back into old habits, do not hesitate to dump him. If he doesn't see the error of his ways then he's not the right person for you and you still win! If you can do this, you'll be mentally years ahead of most others your age.