If you look over on forum.thegradcafe.com , you'll see a lot of advice about that-- finding fit with supervisors, etc. That is less true in the humanities, where you don't need to set up a whole lab to handle your research question, and competence/status is a little harder to measure without the brand name of the university, but it's worth a thought.
In any case, for an MA/MSt, the prestige differences won't be too much to worry about-- assuming you end up performing well where you wind up. If you're eventually doing a doctorate, nobody will care where you got your master's unless that is the only blue-chip name on your CV. And to get admission to a doctorate, the main thing is getting a good proposal together and finding someone who wants to take you on, while putting enough smiley faces on your transcripts to keep the admissions committee happy. Assuming you want to get back into Oxford (or Cambridge), as long as you can put out feelers and develop a credible proposal while you're getting your dissertation together at Durham, all will be well. Just keep your contacts at Oxford warm and earn recommendations on next year's research.