So a few a years ago I had the same issue but for a course in Architecture, my primary reason to switch was the fact the Uni I went to wasn’t accredited for the course and wanted to switch to a university that was accredited for the course. Therefore, the prestigious factor did play a part.
I switched from Lincoln to Leeds, which I think is a Russell Group university. This helped me slightly and improved my job prospects in graduate roles, but I can assure you that no one cares after a year of working, so if your plan is to stay at Oxford Brookes, it's also not the worst thing (I'm just playing devil's advocate).
For my A-levels I primarily had 2 aswell both were less than your grades at (B and a C).
Lincoln gave me an unconditional at the time so I got lazy in 2019 and by 2020 I used Covid as my excuse for the switch.
It is worth mentioning when I applied for the switching Uni (through UCAS) for Year 2 they did ask for my grades (A-levels and Year 1) but played more heavily on a reference from my university faculty like your module coordinator/course leader/head of department from the university course.
If they provide a sufficient reference and you ensure you complete all the modules and gain credits required for year 1 which I believe for me was 120 credits for the full year. There should be no reason for the university to reject you. They are more likely to assess your current grades and ensure your personal statement reflects your learning for Year 1 and I do recommend finding a course that is similar to your current course. If your plan is to continue Psychology in another university, this will be fairly easy process. However, when explaining it to Oxford Brookes why you want to change I would state the reason to be more about personal/family/location/health/home-sick rather than mentioning its because of the university standing, as this will help gain more brownie points.