No, it will not make any difference. Firstly, I'm guessing since you posted this in the GCSEs forum you aren't yet doing your A-levels - in which case it's far too early and anything you do now won't be "current" by the time you apply.
Secondly, there is no such thing as "maths or physics work experience". Maths and physics are academic fields. Working "in" maths or physics means being a researcher at a university (and likely teaching degree level maths or physics). This isn't something you can gain work experience in - best case scenario you would be shadowing a university lecturer/researcher, a good chunk of which will be watching them doing largely admin work and if they're an experimental physicist, you probably wouldn't even be able to shadow them for any of their experimental work due to safety matters.
As a result, neither Oxford nor Cambridge (nor any other university) look for work experience for academic subjects. Because it doesn't exist. The only courses where work experience is required are healthcare professions courses and related professional fields like vet med, social work, teaching and so on.
The only other courses where you could get work experience which might be a) possible to find and b) marginally useful might be in fields like engineering or law, just to get a sense of what the actual professional field does (compared to the academic side of studying those areas). It's still not necessary and won't be a factor in admissions though - it would just be for your own personal benefit in deciding if it's for you or not.
The work experience you are doing now is not designed nor intended to be relevant or useful for applying to uni. It's to provide you some basic experience of working in a professional environment so when you get your first job as a school leaver (be it a full time job if you don't go to uni or a part-time job/casual work while at uni/before starting uni) you have some kind of experience to put down.
You will probably actually get far more out of your work experience by getting some kind of generic and probably not very interesting or fun job at tesco or something where you can then actually demonstrate specific examples of working and e.g. managing competing priorities, working to deadlines, working independently in a team etc. Rather than office based work experience (which would necessarily include any possible work experience that might exist for any academic degree subjects) where you just sit there watching someone else do stuff all day.