I don't really see the problem in the girl sleeping over as one of a group of friends... Do you object to him having female friends at all? What about male friends who are gay? Or female friends who are lesbian? Or any other permutation of "friend" who is not you, and who therefore he has chosen you to be in a relationship with over?
Had he told you which of his guy friends would be staying, and deliberately not metioned a girl? That would be weird. But "hey, I have some friends staying over" in my books says nothing about their sex and/or genders, and everything about their relationships (notably that, unless he is actively lying to you, they are in no way something which could be conceived as your problem).
That said, you feel the way you do, ie. uncomfortable, and that's ok. You can't do anything about how you feel, but you can chose how to act. And in terms of your original question, I would say you have absolutely nothing to worry about. Your boyfriend, along with lots of other perfectly wonderful partners in the universe, can be friends with people, and sleep over with them, without it being romantic at all.
The only possible thing you could be unhappy about in your post is that he asked you to leave while they were having breakfast, and come back later. If they're his friends, it might be nice if you eventually met them, because hopefully you will eventually be doing stuff together. (When you start spending time with his friends as his girlfriend is dependant on lots of things including how old you are, how closely knit friendship groups are etc. etc. of course.) I would be hurt by this action, coming from my boyfriend (who has had mixed-sex groups stay with him on holiday without my being in the least worried). But I don't know what your relationship with his friends is like.
Anyway, that's my thoughts, make what you will of them.