I would suggest speaking to your GP if your anxiety is affecting your day to day functioning, but bear in mind rankings and league tables of colleges are meaningless. They are all academic - and assessment is departmentally oriented, as is most teaching. You will probably have tutorials at other colleges at some point, and the marking of tutorial work is primarily if not wholly formative, rather than summative, to my knowledge. The end of year exams are not associated with any particular college, so the train of thought that one is more "academic" than another is clearly pointless...
Just because historically this wasn't the case, and thus a college achieved a reputation for being more academic, and parlayed this into receiving more applications and thus the prerogative to be even more selective among an already extremely capable pool doesn't reflect in any way on the actual current status of the teaching and marking of the college. The fundamental way in which Oxbridge degrees are assessed has changed a great deal over the years (they are you know, written exams rather than all oral viva voce style sessions now...).
Besides the point, the interviews are meant to be challenging. They don't care about the "right" answer - they want to know how you reason your way towards it, whether you reach it or not. They may well ask a question with no right answer, or that has not been answered even by the highest echelons of researchers (if it's sufficiently simple but deep to be accessible to those without specialist knowledge but not necessarily answerable by those with it - several maths problems for example are simply stated as such but considerably more complex to solve).
So see your GP if you like - but realise you're being irrational about this and putting far too much stock in meaningless idle chatter from TSR and similar places that talk about prestige without any real experience or understanding of the complexities of the matter.