The way you're saying it, I think changing to further maths might be better, but not necessarily appreciably so.
Further maths would be slightly better aligned with what you aim to do, though physics isn't a poor choice. I should think a better grade in physics would be better than a worse grade in FM (I think Oxford CS is very maths heavy so they might require FM, but most I believe don't).
If you have aptitude for and interest in mathematics already then it may be easier for you than physics. I found it easier than physics, but I enjoy maths a lot more and it's probably the other way around for most people. Physics has more of a memorisation element - not a lot - but a good memory for facts is useful there in a way that it isn't for FM.
TL;DR, they'll look pretty similar, do whichever one you think you'll prefer/feel more confident in.