I'll get this out anyway, partly for anyone else making this choice and partly to organise my thoughts.
I think it would be fair to say Mandarin and Latin are 'opposite' languages, at least from the perspective of a native English speaker. Compared to Latin, Mandarin has virtually no grammar. Compared to Mandarin, Latin has no speaking difficulties. Their difficulty curves are also opposite IMO - Latin starts off easy and becomes more difficult the deeper you go (though learning the grammar is more time-consuming than difficult). Mandarin starts off incredibly hard and gets easier: in writing for example the first 100 characters you learn will be the most difficult Chinese characters you'll ever learn. Even if you come across a tricky character in your advanced studies it'll still be easier than learning the concepts from scratch, and most of the time the character is constructed of simpler elements you'll have come across a thousand times already. Off the top of my head I think there's only about 250 or so of these elements (radicals), far less daunting than the 5000 characters you'd need to read well. I think knowing those things would have made it much more bearable for me when I first set out on learning characters.
Learning the tones in Mandarin is a total bastard. First they all sound the same, then when you can hear it you still can't quite pronounce them, then when you can pronounce them in isolation they go to buggery when you try to string them into a sentence, then when you finally manage that the tones screw up your sentence intonation so you sound like "I'm JUst gOIng TO-the SHOP". That intonation bit is the problem they never mention in any of the learning material, but it's a big one. The other issue about speech is there's a bunch of sounds in Mandarin that simply don't exist in English, and they're not easy - particularly the pinyin "r" "x" and "v". It's a matter of learning to move your lips and tongue in completely new ways. With Latin you don't even need to worry about speech, but even if you decide to it's nowhere near as difficult.
The upside of Mandarin is the grammar. After learning European languages it's very refreshing to be able to fully use a noun or verb as soon as you learn it. Nouns don't really change at all - there isn't even a difference between plural and singular, let alone accusative, dative, genitive etc. Verbs don't change according to the pronoun, nor do they change according to time. You just use various particles before and after the verb to change the time or make it positive or negative, which largely follow the same rules. No need for pages of tables and declensions and cases.
A big part of how easy a language can be learnt is motivation. There's some pretty cool stuff in Latin to read - and you can get that unique feeling of reading something thousands of years old, but there's not nearly as much variety as modern Chinese culture. There aren't really any Latin songs, movies, forums, comics (I think you can get Asterix and Tintin in Latin though), books and native speakers. The last one is a huge issue, and the difference between learning from dry pages and actual human beings. I know which way I prefer.
Overall I'd say Latin is easier if you're just looking at the languages themselves. If you factor in the learning resources available I think it levels off with Mandarin.
If you're feeling adventurous go for Mandarin - it's such a different language, and I think that makes it a very special learning experience. You'll learn not only a foreign language, but gain a lot of perspective on your own language. You'll stretch yourself and drive in directions you never knew existed, and every turn can reveal something completely alien. Besides all that, you can access a society with a rich history and developed culture - a culture being lived and shared by over 1.3 billion people.
tl;dr Latin is like a sunset drive through spectacular scenery in a finely crafted sports car. Mandarin is like paragliding on Neptune.
Both sound pretty awesome ways to spend an afternoon, but in the end it's all down to personal taste.