Yeah it matters a whole lot. You see when you graduate and if your college was ranked in the bottom 6 or 7, you have this big red stamp put on on your transcripts and degree certificate and thus employers and other institutions, the rest of society etc. will be warned that the college you went to ranked......
NO! It really doesn't matter where your college ranks. There are also rational reasons as to why some colleges like say Wolfson will rnak lower: their undergrads are mature students and generally for a whole host of reasons won't easily obtain firsts. Some colleges like Trinity have a larger student population so the statistics are skewed.
Which college you end up at won't affect the overall quality of the education you receive and will have zero bearing on future prospects. Sure some colleges might be more "attractive" to attend than others but you will find whichever college you end up at, you'll still have a great time and at the end of the day..... YOU'RE AT CAMBRDIGE!
I'm a graduate of St Edmund's myself aka Eddies. In fairness I received an offer very late in the cycle and thus many of the more popular colleges were already full. But for me to even say I was a Cambridge student and in the end a Cambridge graduate, and to graduate with my degree and a first/equivalent of a first, that's all that mattered. I'd rather have gone to the lowest ranked college at Cambridge, than to have not gone to Cambridge at all.
And here's one more thing to think about for you or anyone else who might be feeling irrationally upset that they got admitted to a perceived "less prestigious college" at Cambridge........... there are PLENTY of students out there who got rejected from Cambridge who would have taken that offer in a heartbeat.
My college might have ranked at the bottom of the Tompkins Table but I don't care, I get to forever have Cantab beside my name, and a Cantab email address.