Sidgwick site is just the name for one of the bits of the university where there are lecture theatres, libraries etc. The other main one for pre-clinical medicine is the Downing site. You can see where they both are
here - nearest colleges for Sidgwick site are Selwyn, Newnham, Robinson, Queens' (plus Clare's first year accommodation in Memorial Court, which is next to the UL and not marked on the map) and for the Downing site are probably Downing, Emma, Pembroke.
BUT:
1) all of the central colleges are less than 10 mins by bike from either of them, and even Girton/Homerton are only about 20-30mins away depending how fast you ride. You may also wish to consider other location factors like proximity to town centre/shops/sports fields/boathouses as well as just lectures.
2) for many colleges you will not be on the main college site for the whole 3 years, so may end up considerably further away - for example Clare's Mem Court is very near to the Sidgwick, but most 2nd years live in the Colony up on Chesterton Road which is a fair bit further.
3) you will be spending the last 3 years down in Addenbrooke's (and in various DGHs around the region) so it's worth finding out what your college's accommodation for clinical medics is like too.
4) there are many other factors which are important in choosing a college, so while location can provide a small advantage, it shouldn't be the deciding factor. Good luck!