You have to declare all qualifications and attempts at exams on UCAS, so they will see you're resitting it and I don't think it will make a difference. The idea is they want to see that you've done the material of 3 A-levels over the course of two years in parallel. So I wouldn't suggest doing that. I still would recommend dropping physics by the sound of it, as it doesn't sound like you'll do well, or that you enjoy it. Also if you take it and get an E then it's still not going to get you into Oxford.
Your best bet is, unless you can pick up a new subject and do it intensively in one year, to just focus on other unis for Japanese for your undergrad and if you still really want to go to Oxford, look at doing a masters there. Worth noting also that for Japanese, debatably SOAS is better for Japanese in terms of language development in general (particularly reading/writing; Edinburgh apparently might be a bit better for developing speaking/listening skills, but apparently SOAS students always get the best outcomes on the JLPT which doesn't test speaking and listening).