I'm actually at cambridge so don't know anything about specific colleges in oxford HOWEVER I would say that whichever college you go to will be essentially the same experience as far as your course goes. Most people choose based on accomodation, area, architecture, sporting/musical prowess, facilities, number of people doing the course there, size of college etc.
My friend is at Corpus Christi Oxford doing PPE and he loves it there, but the college really effects social life a lot more than work/degree I would say. Also he didn't even do maths at A level, just did physics, so corpus doesn't care too much about the maths side - no idea about other colleges.
If you are particularly academic you may want to google supervisors at individual colleges to see what they're into - though the accomoplishments of a supervisor (or i think they're tutors in oxford) don't mean too much in how good they'll be for you!