wow, he sounds just like my ex.. i couldn't say anything without him being offended and wanting me to apologise.. it really was a joke. i never really solved the issue, and we were only together for 10 weeks.. but meh. he was an ass.
i'm hoping your boyfriend is someone you can talk to easily, unlike my ex - told him anything and it would be around school the next day - uhm, yes, you should talk to him and explain that you are only kidding when you say stuff like that, and he really shouldn't be so defensive. tell him that you are not out to insult him or anything, but you are just having a mess about like he does..
if he continutes to be so defensive after you have spoken to him, then i suggest yuo give him the old heave - ho. don't worry about hurting his feelings - i mean, if what you have said is true, he has hurt you enough to warrent this, and he is the one who drove you to having to end it in the first place.
whatever happens, good luck.