Right so let's start by listing the non-gowned colleges as a starting point. Collingwood, St Aidans, Cuths, Hild Bede, Stephenson and Van Mildert. Stephenson is fully self catered and Cuths is about 75% self catered, so let's remove them. Hild Bede is the only college out on it's own and not near other colleges so let's get rid of that too.
So now we're left with Collingwood, St Aidans and Van Mildert. Those are all fairly large colleges, so let's ignore that point for now. It's worth noting all of these are hill colleges, so unfortunately none are particularly close to town, however hill colleges do tend to be more spacious and have less shared rooms, so I'd say a hill college is still a better choice for you.
Collingwood is very near the new computer science building (which should be complete by the time you start) if that's a factor for you. While Collingwood has a reputation for being sporty, this is really only because they're a large college so offer a lot of sports teams, where maybe other colleges might only have a couple per sport, so it's not a major thing, also they have some good places for practice (eg. a football pitch which a lot of colleges don't have) so people who like sport are perhaps more encouraged to apply there. If you don't want to be involved in sport then there'll be plenty of people there that aren't too!
St Aidans is probably the one of the three least near other colleges, and it also has the most shared rooms and is the smallest, so I'd possibly say out of the three it isn't the best choice for you.
Out of collingwood and Van Mildert it's really up to you
There's not a great deal between them and I'd say they meet all your criteria apart from the close to town bit