If you intercalate, even if it's a fixed part of the course, you get NHS bursary and paid tuition fees for the last two years of your degree instead of just the last one, as is the case with a five-year degree with no intercalation. So (and someone please correct me if this is wrong), you won't have an extra £9k in tuition fee loan to pay back if you go to Oxford.
You're right that the university you go to is largely irrelevant when it comes to getting your first job, and Medicine at Oxford and Cambridge is a different animal to Medicine anywhere else: the first three years are strictly pre-clinical, and I don't think there's another medical course in the country quite like that. What matters most is whether you think that different structure and different style of learning would suit you. The extra year will seem completely irrelevant ten years down the line - so many people choose to intercalate anyway (since you can intercalate anywhere, and it's optional in most places), and many others have taken a gap year before they even started. Some, like me, start med school well into their twenties or even their thirties, so one little year really isn't going to make a difference.