with me it is difficult to explain. i was taken ill in primary school and so missed out a lot of work..addition, multiplication, subtraction and division - it's all Greek to me. i then missed out of maths in year 7 having to do remedial maths catching up on the work i missed in primary school - and missing out in year 9 because there wasn't enough teachers to teach bottom set maths. but then i moved school and was told i had a "natural gift" for the subject, and i enjoyed it a lot more. with private tuition i gained a B at GCSE and was told that if i'd been given a little bit of encouragement and support from year 7, i could have possibly gotten an A.
so in my experience, i think it is disliked amongst school children because people aren't very good at it, and give up too easily. but then, it is the same with other subjects; modern languages, sciences and technology - you either have a gift for it...or you struggle with it.