Some things to consider:
Cambridge Mathematics with Physics is a first year only course that is part of the mathematics degree there. After the first year may either continue in the maths degree there, or change into the natural sciences degree to specialise in physics. Consider whether this is what you are expecting/wanting from a course. At Oxford you can study either maths or physics separately, however there is a fourth year course jointly between the two (the MMathPhys) which I understand can be entered from either side. Oxford also allows you to combine philosophy with physics.
Something to note is for Cambridge maths with physics you would normally be set a STEP requirement for your offer which is usually 1,1 in STEP II and III. This is quite a hard requirement to meet and on average about half of offer holders for Cambridge maths (including maths with physics) fail this requirement. It is possible that if you are sitting a non-UK curriculum you will be asked for S,1 or S,S in STEP which is even harder.
I think for Cambridge the hard part is not being rejected pre-interview (which I think, unless you apply to Trinity College for some reason, is probably unlikely if you otherwise meet their entry criteria), but meeting the STEP requirement after getting an offer. With Oxford it's more a case of trying to get the interview in the first place, for which the PAT will make a big difference for physics there (or the MAT for maths).
The big difference otherwise is, based on what you've said, you are considering physics for Oxford or what is functionally maths for Cambridge. There is a lot of difference between these two degrees so you should carefully investigate that and decide which is best suited to your interests - if you don't want to be writing proofs about abstract algebra and real analysis then maths may not be your cup of tea.