The colleges aren't going to necessarily make or break your experience there, and none will be "better" or "worse" academically than any other; in order to be a college (or PPH) at Oxford they all need to meet the same academic requirements. Also some of the teaching may be arranged by the department anyway. The only course-related aspect which may be worth considering is some colleges insist on having all their undergraduates taught in-college for the first year, and so that may constrain your range of options from the first year course in some cases. Usually for the final honour school you can take any papers I gather, and if necessary you will have tutorials arranged at other colleges as I understand.
Otherwise you are probably best focusing on practical aspects of life and living in college. So for example, facilities available, rent/food costs, location, architectural style (if that's important to you), what societies/clubs the college has, college bar (if that's important to you), storage space availability, ensuite availability, living in college out of term, library hours, etc, etc. Bear in mind also normally applicants get interviewed at two colleges I believe, and you may be made an offer by either (or by a third college you weren't interviewed at) so you may well end up at another college than the one you applied to anyway (not sure of the numbers at Oxford for this, at Cambridge I think its about a quarter to a third of applicants get pooled but the system is slightly different there I think).
@The_Lonely_Goatherd might know of some other things to consider when applying to a college?