As you're predicted an A* in maths, I'll assume you're intelligent. And as you're talking about "the better unis", I'll assume you're very intelligent.
I did humanities based subjects at GCSE, AS and A2 (history, art, psychology, religious studies). Predicted mainly A*s, 1 A. I applied for psychology at UCL (ranked no. 1) and was rejected because I didn't do biology. I also applied to Oxford, this time applying for psychology & philosophy, got to the interview, had no idea about A2 maths terminology, and was subsequently rejected.
If you're seriously thinking about top unis for psychology - Oxford, Bristol, UCL, Warwick etc - in an ideal world, you should study at least one science and maths. Psychology isn't even a requirement, but obviously it's valuable. Think about it this way - you might not enjoy maths, but you'll only have to study it for 2 years, and it will just improve your chances of pursuing the career you have such a passion for. 2 years vs. your whole life...
Not having maths and a science has forced me to take a gap year to fast track both at A2. Don't make the same mistake I did!