I only know one graduate student at Oxford, and there at St John's, but I think as a graduate your social life is even less dependent on your college than as an undergraduate (this is what they told me, anyway, I'll just repeat it for you).
For graduates it's unlikely you'll be taught by people in your college, and if you are, you'll meet people at lots of different colleges coming to be taught by the same person, and that can be a route into a bigger, more "vibrant" college. Also, graduates normally, having already done the whole uni thing, know what they're into, i.e. this person was very into politics, and hence spent a lot of time getting to know people, both grad and undergrad, through the various political extra-curricular opportunities Oxford offers (eg the union, the labour/tory/libdem... club, CND, Greenpeace, wahtever). Also, St Cross is quite/very central, so it'll be really easy to get to know people outside college.
And if all else fails, I gather that if you row then you spend your whole time rowing or working, and that forms your social life.
This is just what I heard second hand, so dont take it as gospel. But I think Oxford's too small a town to be too daunting socially (as opposed to London; I know people who've went there and hated it, because its so hard to break into the social world... apparently!)