M1 physics is a real kick to the system and is such a massive step up from GCSE. For the first time, you can't easily visualise how the theory connects to the numbers, which severly dents your confidience. Also, about 30% of it is standard form, graphs and unit conversion, which I found very mentally draining.
Maths-wise, C1 is essentially high level GCSE algerbra and C2 is pretty straightforward, with the possible exception of logarithms. I would go with maths, but be very careful with the side modules; they can drag your grades down considerably.