He doesn't need to play video games 4-5 days a week for several hours to keep in touch with friends. He does it because that is what he likes to do, which is absolutely fine, but he also needs to balance that with the relationship that he has with you. If you come over to see him, it is not acceptable for him to play video games for several hours and to leave you with a little bit of time at the end. It equally wouldn't be acceptable for you to demand that he doesn't play video games, but you're plainly not doing that. The answer, to my mind, is to see when he wants to play Xbox, and just don't come over when he'll be doing that. But by contrast, when you do come over you're entitled to expect that he will actually want to spend time with you. Arranging to do things like go out on dates is fine, but ultimately you're going to need to be able to spend time with him in a house where he is willing and able to prioritise you over his video games. That's not asking him to give up a hobby that he enjoys, it's just asking him to also make time for his relationship with you.
I have some doubts as to whether he's willing to do that, because his responses so far seem to be extremely immature. But that what the situation as to be in order for this relationship to carry on. If he's not willing to give you the quite reasonable attention that you're asking for, you should find someone who is.