Not sure why my friend reacted like this

I have a male friend who I've slept with in the past; I'm 26 and he's 25. We started flirting online and agreed we'd both like something to happen again. He's also a good friend but due to distance I've only seen him once in the past year, but we speak a lot.

Anyway, he now lives about 200 miles away from me or 3h by car/public transport. I suggested meeting up next weekend but he wasn't free sadly.

I'm now going back to my parents' for the holidays, then in November I will be going to visit another friend and he will have exams and stuff, then it'll be Xmas so I was worried we wouldnt get a chance to see each other.

I suggested that I go to see him Halloween weekend as it seemed like one of the few we were both free.

I work till 5pm and then i'd get there around 8:30, and i'd be leaving the Sunday afternoon.

He told me that he had a couple of mates coming the Saturday evening.

i said in that case, if he wanted I could still come on the Friday, have all day Saturday then leave at night if his friends would be there.

I would have stayed until the Sunday but didn't want to impose if his friends would be there.

I also wanted to see the city he lives in and I thought we'd have a few hours to do so on the Saturday.

Maybe it seems far doing 3 hours of each way to stay there 24h, but I don't mind as I said I don't see him a lot, and we'd still have time to go out the Friday then visit on the Saturday.

He replied saying that it was really weird that I wanted to come all that way just for sex and to not even see the town or anything.

I was really shocked as that was not my plan at all. there's no denying I would like to, but he also told me exactly the same thing, and I'd very much like to see him just to chat and spend time together above all, and to see the town.

I don't really understand that reaction as I've done plenty of trips like that as sometimes it's the only option and it's better than nothing.

Does it sound like he reacted badly or is he in the right? I would have happily stayed Sunday too but he made it sound like he didn't really want me to, plus I didn't want to impose.