i can't stand my best friend anymore

they're not even a bad person, they're just so annoying, and a bit of a pervert i reckon. at around the time we met, i was extremely lonely and had very low self-esteem. i was surprised and very happy someone was willing to spend so much time with me, and we are both on the spectrum so it was relieving to be around someone like me, and we dated for a few months. i got annoyed by it, they were so clingy and always embarrassing me with excessive PDA and saying **** like "i love you more, i love you most" type of ******** in public. i broke up with them but was still very attached to them since they were still my only close friend and the only person who i believed cared about me. since we met, it's been over a year. i hate talking to them, especially for the past couple of days i can barely bring myself to do it, i only want to play card games with them. every conversation is the same, and i'm not even gonna try to describe it, but they kind of remind me from the worm guy from don't hug me i'm scared, his name's warren the eagle, just distinctly unlikable you know. of course i feel bad about not wanting anything to do with this person anymore, they still like me a lot, and they have mental issues, if i rejected them it would be bad to say the least. my plan is to slowly grow apart. it that an okay way to handle things?