Friendly coworker or harassment?

A work colleague was giving me a lift to and from work. I saw him for the second time in my life. in conversation he was absolutely neutral and kept his distance in communication. we talked about his girlfriend, family values, etc. at the end of the day he slightly hugged me a on the shoulder (I barely felt it). It seemed strange to me, but on the other hand, maybe this is friendly communication?
after that , already in the car, we talked about sports he was explaining his injury that he got , so he reached over to my foot to show me exactly where he’d been injured. I pushed my foot back a little bit but he ignored it and he reached. After he touched my knee as an excuse again.
on the one hand, if a girl did all this to me, I would be ok, but since this is a guy, can this be considered as flirting? Where is the line between friendly communication and crossing boundaries between male and female colleagues? how do you manage this?