Inappropriate Colleague

I'm a 21 year old female working in a research lab. During my first few weeks, a male colleague in their 30s kept repeatedly winking at me. I mentioned this to a fellow grad, who informed me that it was just part of his character and he was having an affair with another person in our lab (they're also in their 30s).

We never really spoke until he was made my joint supervisor. Initially, he came across as friendly and supportive, but I didn't like how close he got to my face when he was talking (he would almost lean in).

He then began gently guiding me around, by placing his hand between my shoulder blades (just a light pat in the right direction). Then on occasion, he would touch my waist while passing behind me. Sometimes he brings up awkward conversations. For example, we were talking about different sized test tubes and he started asking me whether size matters! Such things only happen when we are alone in the lab.

However, one day we got some good results back from an experiment and he was so happy, that he put both of his hands on my shoulders and gently shook me. I felt very uncomfortable and another colleague saw the awkward interaction but said nothing.

He's openly having an affair with someone in our lab and will full on embrace her and make other colleagues feel uncomfortable. I get on well with her and she is almost like my "work mum", we talk quite openly about things, but I haven't mentioned anything to her about his actions obviously.

I don't understand why he is like that towards me, it's obvious we would never end up being together he's over 10 years older than me, married and has children. I can't go to HR as I don't want it to be blown up into something that could end up making my life more difficult. But how do I subtly get him to stop without making it into a big deal?