It's possible that he likes you, of course it is but even if that is the case, it doesn't mean that anything has to happen. You've said that he hasn't flirted with you and he has never treated you inappropriately. I think he was probably just being a nice guy when he first asked you to do something, so that you could see some of the sites. I don't know, nobody knows how he feels so we can't tell you what to do but if you feel uncomfortable with it, politely refuse. I don't see what harm it can do, like someone has already said, bosses are people too and they are entitled to friends and partners. If you want to go, go and see what happens. If at any point you are uncomfortable, tell him.