Neither is "more desirable" than the other - employers generally don't care what subject you studied.
I would not focus on the employer angle because ultimately what makes a difference to employers is what work experience you gain through internships etc, what relevant transferable skills you develop at the uni (e.g. through taking on leadership/committee positions in societies etc), and more than that even really just how well you do in their battery of psychometric tests and assessment centre exercises.
As to whether you want to switch it's up to you, I'd speak to your personal tutor and the other department to understand if you can and how that process works. You may also want to explore either a) if there are joint honours courses between your two subjects or b) if you can take optional modules in the one subject while studying the other (in either "direction"). As those may also be opportunities to enable you to pursue both sides as you seem to enjoy your current subject as well but are particularly interest in history of art

Note also if you want to go into a role within the arts/heritage sector I believe it's not uncommon to do a masters in history of art after a different undergrad degree, so that is also an option in the longer term if you're unsure. Some are quite reputable in the field (e.g. the Courtauld MA and the RCA curating contemporary art MA I hear) and usually don't expect a background in history of art specifically (at masters level).