Hard to say actually. If he’s giving up time from his friends for you, how do you know he also doesn’t consider you a friend?
If the general vibe is just “friendship”, I would take it as that. However, if you see there’s more to it, I would just talk to him more frequently up until you suggest things to do with each other. (or him, as he gets more comfortable around you.)
It’s much more natural this way than just making a move straight away. Actually get to know him first.