Worried about people turning on me

Hi guys, first off to give context I have anxiety. I’ve got rid of someone from my life as he was constantly rude when he would get drunk he would start trouble. Thing is he is still friends with one or two others that I know.
Anyways now a friend from the past who ditched everyone for like 7-8 years when he had a gf has cropped up again & befriended this drunk ex friend. He tried to reach out to me but about meeting up again but I said I’m quite busy.
What I’m worried about is them gossiping about me in a bad way and possibly turning other people against me, spreading false information etc.
Is this just my anxiety making me feel bad about this or overthink this?
How would you deal with it?

How old are you?
At school or uni?

If the friend is rude and gets drunk often, then distance yourself from them. Have you talked to them about their drunken behaviour? Have other people experienced it? It would also depend how rude as I make allowances for some drunk people, but never personalised or aggression.

Have you shared with the other friends? Do they experience it?

I think its anxiety.
The way I would look at is this?

1..Sort out with friend and ask why he needs to be so rude? He amends or ignores and then you distance yourself. In the event he makes trouble and others listen, then they werent very good friends or worth having. You are better off that way and can find better friends.
2. You are just being anxious and nothing happens.

I am 28 and at uni (late starter)
It’s an ex friend I don’t speak to him anymore as he did this repeatedly, even after I spoke to him about it numerous times. They have experienced it yes but kinda just keep forgiving him after months break of falling out. The ex friend who came back around I don’t know if he’s experienced it as I haven’t spoken to him or the old drunk friend since.
I mean should I really sort it out with the old friend if he never really apologised, also a really long time has passed now, like 10-11 months, I think even if I did reach out it might be awkward or he may not want to talk.
My anxiety is super bad, I’m thinking of all these bad possibilities that trouble might be down the line, I feel people might turn against me.

1. Find some new friends.
2. You can still be friends with the others, just have more distance? if it looks like hes might get drunk then dont go?
3. The main part is your anxiety, because I was trying to show you someone without anxiety wouldnt worry, they would deal with the situation rather than being a victim of it. The outcome is much less to be feared than all your worry about it.
4. The importnat thing in life is to like yourself, then you will find other people matter much less.

1. Tbh I have enough friends it’s more the fact I don’t want enemies all ganging up to bad mouth me.
2. The other friends I have know I don’t speak with that person so I don’t see him when I meet them.
3. Yeah my anxiety is very annoying, I like have a fear that they will all sort of smear my name to others or make up lies as they don’t like me for ditching them.
4. That’s good advice, I am really low on confidence and feel full of fear, I really need to work on my self esteem and stop worrying but I find it difficult.