Well, what part of maths specifically? As, having done maths (at and beyond 6th form standard) and now focusing on ancient languages, there are certainly some not dissimilar experiences. The abstract reasoning, pattern recognition, and problem solving elements are all very much present (especially if you focus on linguistic/philological angles) for example. So something to bear in mind. I gather it's not actually that uncommon for people interested in maths and similar to be interested in studying ancient languages as there are those parallels in methods of thinking and approaching issues in both.
Obviously though, if you absolutely couldn't bear to stop doing maths then, that is a major consideration. Although equally consider - the maths you do in a maths degree is so different to A-level Maths that you're not really "doing more maths" in that style by doing a maths degree for the most part. If you want to continue with maths in the manner you've studied it in A-level then a physics or engineering degree would be much more aligned with your expectations.