Hey! I'm a third year at Newnham doing MML, French and Spanish (the latter ab initio). It's super close to Selwyn (and department, really don't knock that!) and I can honestly say the distance out of town isn't an issue, it's nice and makes it quieter, particularly because you avoid the tourists.
A couple of things to bear in mind - not all colleges will have on-site accommodation for the entirety of the course so you may find yourself moving around town depending on your year, if distances to things are an issue (e.g. Caius has accommodation next to Sidgwick site as well as in the town centre).
Another thing to perhaps consider is what each college provides in terms of travel grants - I didn't even think about this when I applied and got lucky that my college is really generous, but others don't give so much for it.
Best bet? Check out applytocambridge.com - it's the alternative prospectus written by the students and you should hopefully find lots of info there to help make a decision! All the best, and if you have any other questions let me know.