I have this exact same situation, and I know what you mean about it feeling horrible, but at the same time I think you're definitely worrying about something you don't need to be worrying about at all.
I know this would be hard, but have you considered talking to your friend about it? You don't have to go into huge detail. People are surprisingly understanding and he won't want to be making you feel like this.
I wouldn't say that you feel like you're more invested than he is or anything, because that might come across as ungrateful or make him feel defensive. It depends what you want from him - if spending more time hanging out just the two of you, or feeling like he makes as much effort to see you would help, or whatever, why not just say that? It would take a huge amount of bravery, but if you could say to him that you really value him as a friend he'd probably say the same about you. Maybe just explain that because you're introverted you need to make time to really focus on your friendship - but also recognise that for your friendship to work, you also have to think about what he needs/wants as an extrovert (just as much as you want him to be a certain way, he'll want things to be a certain way.)
It might be that he doesn't feel as intensely about your friendship as you do - sometimes that's just the way things are, but it won't be about you, it'll be about the way he is. It's better to ask him, though.
As someone who is a bit of both (introverted and extroverted) it really is just about the way people do things. It probably wouldn't occur to him on his own, so try to think about the fact that actually telling someone what you want from them makes it easier for them to know what to do, and people really appreciate that.