It depends on your budget for maths qualifications and level of accreditation of maths course you want to teach. A joint degree in maths and computer science will qualify you for both maths and computing. If you are on a budget, consider a subject knowledge enhancement course or one of the cheaper online maths courses. If you want better accreditation for mathematics then yes, obviously take more maths modules at university.
Options if you are on a budget:
Cheap maths textbooks.
Private maths lessons with a tutor.
Cheap online maths courses such as khan academy, IXL and reed.
YouTube videos on maths.
Subject knowledge enhancement courses.
BBC bite size online.
Options on a moderate budget:
GCSE and A level retake exams. (You’d be surprised some questions on the syllabus today are still tricky.)
Level 3 to 5 teaching credentials. (These are intended at people who want a more specialised maths route for their needs.)
Options on a high budget.
Degrees in mathematics from the open university or your local university.
Obviously, the higher the budget, the more qualified you get and the better accreditation you get.