While I don't want to give you false optimism, in my own experience I got a C in both my Maths AS core units. At A2 level, I resat both and ended up with an A on the resits and overall. In fact, I cumulatively resat all the core modules, but only in my final exam period did I really decide to 'buckle down' and do my best. So it is certainly possible to drastically improve your grades through resitting, but only if there's a good explanation for your initial poorer grade.
But I think the answer has to be that only you know what you're capable of. Saying "I just didn't work hard enough" was true for me, but since you did very well in your other courses you perhaps were giving it closer to 100% to begin with. And of course, you're trying to do this with further maths, which may well be harder. As has already been said, four grade jumps is a big ask.
Finally, if your grades are top and include maths and other sciences, I think your chances of getting into a red brick university to study physics are still very high. They don't care so much about further maths because that's basically what they teach you again in first year.