You're still together with him.
Carry on being selfish. It's worked well enough so far.
And in the spirit of Kaizen, continuous improvement, look at how you can make him a close 2nd to you. So that you treat him much better than you would a stranger. Whilst still treating yourself right.
And look for win-wins as much as you reasonably can. So that over time it becomes an automatic habit.
If he wants to do something that you don't want to do, in general, you shouldn't do it. You should allow him to do it on his own, or with his friends. But you should make it so that he never conquers your time. With this including you sticking to your guns as you have been doing up till now.
The big proviso to this is that sometimes in life you should do the right thing. With this including if you said you'd do something you go ahead and do it. And things like visiting his sick mum in hospital.
Also don't let him conquer your reactions. At the moment he's using mild emotional blackmail in order to make you feel guilty. Stop feeling guilty. Instead carry on with a rock solid good mood in the face of him getting upset.
And don't let him conquer your humour. So that when you turn him down, try to inject an element of humour into your refusal.
And when you're doing a win win, something you both want to do, aim to be really good company for him.