I would note that Warwick maths (even diluted in a joint honours degree) is also going to be fairly intensive afaik, particularly first year. But I'm not sure it's necessarily less pressured, just perhaps in a somewhat different way (I got the sense mathmos at Warwick have to fend for themselves a bit).
@_gcx might be able to offer some insight into maths at Warwick in that frame? Bear in mind Warwick maths is generally considered comparable if not equivalent to Oxford (or Cambridge) maths (albeit you are considering between Oxford physics and Warwick maths & physics, but relative "quality" of Oxford physics is likely to be similar to Oxford maths). So even if you do go to Warwick, you aren't necessarily "going down" in quality so to speak.
In terms of the courses, the question is, do you want to study the kind of maths that makes up a maths degree? Bear in mind you
will study maths in a physics degree, and a lot of it! You'll be using mathematical methods every day, and I expect on a course like at Oxford you might be more likely to spend a bit more time on more abstract formulations of those methods and covering some more applied forms of what is fundamentally pure maths (e.g. group theory).
Also bear in mind the different style of teaching for each, as at Oxford you will have the supervision format, having a supervision with an academic once a week with 2-4 other students. This does then offer the opportunity for some very close engagement with academics, although does also require you be able to communicate your thoughts on and reasoning for a problem in that environment (which I definitely can see would be a stressful environment for some - some people do better where they can fade into a large lecture hall!).
The difference then is on a course like at Warwick you will probably spend a reasonable amount of your time doing maths also doing more proof-based pure maths, alongside the necessary methods-y topics needed to support your physics material. So if the choice of maths and physics at Warwick is more out of a belief that you would do less maths (or less frequently) in a physics degree, then that is not really accurate and either course would suffice. If it is specifically because you do want to write analysis and algebra proofs alongside doing physics work, then of course the Warwick course may be more appealing.
You really do need to disregard your boyfriend as a factor here, imo. Not to be overly negative but, the odds of a school couple still being together by the end of a 3-4 year degree are honestly quite low. If anything going to the same uni might just make things more awkward if (or perhaps, when) you break up. It would be especially bad for you if you did break up, and on reflection you decided actually you would have rather gone to Oxford but picked Warwick to be with him and then regret that decision and "what could have been".
There are reasons to choose Warwick over Oxford, but you need to be doing so for the right reasons; fully understanding the differences in the nature of teaching, the content and style of the courses (particularly the extent of maths done in a physics degree, and the type of math's done on the maths side of a joint honours degree), and not on the basis of interpersonal relationships which may or may not last for your degree (much less the rest of your life). Taking into account your mental health and how that may (or may not) suffer at Oxford (and whether this may happen at Warwick as well even) is important too.