The reason its so contentious is that whenever discrimination and equality law comes to enforcement, it tends to compel people to do things they don't want to.
The infamous gay bakery cases spring to mind. You have people who are compelled to carry out work for people even if they don't wish to - this goes against every single principle of employment law, which usually underlines that people cannot normally be compelled to work - otherwise it takes on all sorts of forced labour or slavery connotations.
The danger is that if sexual "preferences" are recognised for what they are - discriminations - you would have results which might be acceptable to a small number of exceptionally uber-liberal people, but would be completely unacceptable to pretty much everyone else.
There is also no method of enforcement. The best you could do is end up fining someone, unless what we want to end up with is essentially government sanctioned sexual assault.
And why stop at race? Once you have gone down this road, there is no logical reason why age or gender shouldn't also be similarly protected in these terms.
It's racism. 100% But racism that must be protected for the sake of civilised society.