Not in STEM, but I successfully changed courses in my first year at Oxford, so I can offer a bit of advice on it, if you haven't already moved forward! Firstly, I'm sorry you're not having a good time, I've been there and it can feel really horrible and demoralising to get to what is essentially a dream and realise it's not what you thought.
The stuff you do in your first weeks/term can end up being vastly different from what you do overall in your degree (I've heard this across all subjects). It may be worth talking to your personal tutor now and asking especially about the options available for the rest of first year, and at FHS (you can also browse the online handbooks, they will have a lot of information); it may be possible to take papers which are more aligned with your interests than what you're doing now (and almost every course has the less interesting things at the beginning).
In terms of your college not offering Maths+Phil, I do think it is VERY unlikely you will be allowed to switch. Oxford will only let people switch colleges if it is a matter of personal safety (ie you're being harassed or abused by someone at your college). It may be worth considering if there's a course your college does offer that would be better. However, that being said, it is always worth asking, the worst they can do is say no.
Now if all of that still isn't good enough and you're really unhappy, if you do ask to switch, they will want to make sure it is for very good reasons, so you'd need to show you gave it every effort possible. The person in charge of this would the Senior Tutor at your college, and if you meet with them, come really well prepared. I was questioned a lot in this (I switched because of subpar teaching in my department which I and every other student in my year, at different colleges, had complained of); especially they wanted to make sure I wasn't switching because I found it 'too hard', and even if the person is nice it can be a bit unpleasant. They are trying to do something quite logistically tricky, because they have to retroactively figure out if they would've admitted you against last year's cohort. If it's a matter of marks/interest in maths they probably won't let you switch, since you'd still be doing that in your other degree, so I agree with the person on this thread who suggested emphasising your desire to do Philosophy alongside maths, and possibly have a good explanation why you didn't originally apply for it.
You will also have to have another interview, and possibly submit materials such as essays. The standards for the interview may well be higher than if you'd done this normally, as they are judging you against last year's cohort, and also the incoming students (there are often quota numbers for each subject, so giving you a spot this year could be tangled up in lots of logistical admissions issues that are entirely irrespective of your smarts or suitability). However, if you know you want to switch, and you have good experience in your new subject (this is really key, you'd have to show considerable background in Philosophy), this can go well. It can be an opportunity to show you can thrive in this new subject, and to prove that you would be happier and more fulfilled, and most importantly successful, if you did switch. I was horribly nervous for mine, but doing it ended up re-emphasizing that I really wanted to switch, because I had way more fun doing it than any of my other tutorials.
If they did let you switch (bearing in mind the unlikelihood of switching college, so probably only if you found another course at your college you'd prefer doing), you would have to finish out the year in Maths. You are only allowed to switch courses if you have achieved sufficient marks at Prelims (this is in the university handbook); this is usually a First Class, but there is room here depending on the department and personal circumstances (I got a 2.1 but was still allowed to switch). From personal experience, I can say that this is incredibly stressful, not knowing if you are going to be allowed to switch, and having that extra pressure on your Prelims. Depending on how horrible your current course is, and your own circumstances, this may or may not be worth considering. I am very glad I switched, but the stress and pressure, on top of the difficulties of my course not being good, made my first year a bit of a living hell, and I had all sorts of health issues from it.
All this to say, switching courses is really hard...but not impossible (within the limitations, which may or may not be workable in your case). It can also be incredibly worthwhile, and I encourage you to advocate for yourself if you feel you aren't satisfied with your course/teaching/experience. Oxford is amazing, but you are paying for your degree, and you deserve to have the experience you want (within reason of course, and sometimes that may mean you need to unenroll and reapply for the course you want in the next admissions cycle, if that is feasible and you truly think you'd be better off). But also, there may or may not be ways to make your current course better, and time may be one of them. Oxford degrees get more specialised, and therefor hopefully more interesting, as time goes on, though sometimes there are *****y papers or tutors you just need to grit your teeth and get through. I encourage you to speak with your personal tutor and college welfare officer in the first instance, and your college Senior Tutor after if this doesn't resolve.
I don't know if this is of any help, since you may have already sorted it out (and I hope you have!), but hopefully if anyone else is considering changing course at Oxford this might be useful.