MIT has an excellent reputation, However if you want to do physics, I say go for Imperial, as there are very few universities in the world which can even equal imperial in this subject. If you're think about switching to engineering or something got to MIT.
Also from a purely league table perspective Imperial comes in at 5th and MIT comes in 11th overall (the new ones)...but this shouldn't affect you decision.
If youve been to both universities, I say just go for whichever you liked more because i'd say theyre about equal, Imperial would be the MIT of the UK and vice versa