I think it's insecurity, as a black male myself who grew up listening to trap, garage and occasionally grime in South-East London, I always knew that someone who happens to be in a profession where it dominates in terms of ethnicity and gender to always make the first hit on the minority.
It's something I grew up with, even coming from a comprehensive in an impoverished area we still get treated like we are no-marks in society. This is an attitude that wont change, and usually being an optimist I believe that this is a deep-rooted problem in the heart of politics and society in general.
He is clearly racist, and I hope one day a black employee will actually face him and tell him what exactly he thinks of his 'white, middle-aged, upper-middle class boss'.