Well you said that she has had other girlfriends so I assume it's not that despite everyone making it about that.
How was he to your face ? It seems odd that he would wait until you leave to say it.
I had problems with my current gfs father but tbh I didn't give a ****. I put it in my mind the first day I went there, I'm not too old or too young, I've never hurt there daughter in anyway, we're clearly in love, I come from a good enough family that are well of and I plan on getting a good job (not that any of these matter to me) So I just thought that they can have no logical reason not to like me. If they don't have a good reason to dislike you, something like ''she's a dick'' they will just push there daughter away and they should know it.
I was in a similar situation, the first time I met my gf's dad he was kind of bantering with me but he wasn't. I'm sure you all know what I mean. Like when someones pretending to mess around but you can tell they are serious. I just gave it back to him. Everything he said to me I said it back. My gf's mum and sister already liked me a lot so he was just making a fool of himself. When his wife asked for help in the kitchen I knew he'd shut up.
Anyway I honestly never once tried to impress him. After this exchange of ''banter'' in which I indirectly called him a moron
my gf arranged for us to play golf. She told me to let him win. I still kicked his arse all the way down the back 9.
My advice to you is simple, Don't bend over. Do not take any **** he gives you or in anyway try and make it better. You have nothing to make up for. His problem with you is illogical and eventuall he'll realise you're ok (assuming you are)
My gf's dad got me a very nice xmas present this year, So I'm assuming we're kosher now. Despite the fact all I got him was a golf set with a golf ball that had ''2nd'' printed on it :P