I'd say do whats best for YOU
ive spent far too long of my university life caring and thinking about others, and at times its best just to be a little selfish and do your own thing.
you also have to remember (that people seemingly forget) that university is only for 3years (for most students) and that at the end of that period your going to go separate ways. Sure you may keep in touch, but you won't be living together, and could possibly be living at opposite ends of the country or even different countries. Some students seem to have this crazy idea that its going to be student life forever.
so basically i'd say break to them in the nicest way you can, but ultimately think of yourself and your own happiness, it sounds like you'll get on better living with your long-term friend. Also you can easily remain friends with the others, living with them isn't compulsory for a friendship.
its taken me nearly 18months to realize that your never going to please everyone, so you have to look out for number1, especially when it comes to living somewhere, as if your living in a house you don't like it will annoy you on a daily basis.