Should I apply to Oxford?

I have wanted to do physics at Oxford for a long time now and my school have just asked us to put our names forward if we want to apply.
however, I just dont think my GCSE results are good enough and I wasn't allowed to take further maths a level which I also think could be an issue. is it worth applying?
here are my grades
English lit-9
English lang-9
art-8
bio-8
phys-8
maths-8
spanish-8
geog-8
hist-8
chem-7

GCSE grades are fine, and physics at Oxford don't put a huge amount of weight on them I gather relative to other factors (at least historically, e.g. the PAT).
Not having further maths is probably a bigger challenge especially if your school offers it. Your referee can explain in their reference why you were not allowed to take it but ideally I'd recommend seeing if at least you can do the AS Further Maths or something.
Not really clear why you weren't permitted to take FM either since you have an A* equivalent grade in GCSE Maths.
At the end of the day it's just one out of 5 choices so, no great harm in rolling the dice and seeing if you get an interview and going from there.

my school said my gcse results were not good enough to take Fm at a level. They literally told me the second I opened my results on results day that I wasn't good enough and I ended up sobbing on results day!! lol

If you're motivated, then negotiate with them. Tell them you will study over the summer and ask them to conditionally say yes—for example, if you achieve an A* in the first two modules of your AS Maths under exam conditions.
I did something similar: I had a grade 3 (equivalent to a D previously) in English, and scores of 8/9 in double science and 9 in maths. I wanted to study Maths, Further Maths, Physics, and Chemistry. I received an offer from the sixth form I applied to, agreed to resit my English, study the science modules I missed over the summer, and be tested upon returning, with plans to complete my A-level Maths a year early.
I was ultimately accepted to Oxford to read Mathematics and Computer Science after achieving A star A star A A, and I earned a C in my English GCSE resit. You just need to approach the head of the 6th form as an adult and see what you have for him to conceed.

Really you should have 9s either maths and physics (preferably both). But if you think you can do it, then you can show in your AS-levels that you can get A*s. So it's not lost yet.