Okay well I'm an American and attended university in the US for two years. Regardless of the drinking age, people drink. A lot. There are plenty of outrageous house parties that get wildly out of control, more mellow house parties, people go out drinking/clubbing/bar hopping with fake IDs all the time, etc. Honestly, most of my high school friends who are now in university have fake IDs to get into bars, clubs, etc. So don't go to the United States assuming that the drinking culture will be vastly different. It won't be. In your first post you said "...with those who are merely tipsy. " Drinking in the US =/= merely tipsy.
In both countries you can find the parties if you want them and avoid them if you don't want them. You don't have to drink and regardless of which country you're in, you'll be able to find a group of people that aren't huge pub crawlers.
I know you said it won't be the sole factor, but honestly it shouldn't factor in at all. I wouldn't say that drinking is a part of British culture more than American culture. I'd say that drinking is a part of university/late teen early 20s culture on a worldwide scale. No matter which country you're in, you'll be surrounded by loads of people who want to party and go clubbing. I know it sucks to be in the minority and not want to drink, but it's up to you to work extra hard to find those other people who are like you. They're out there, just a little harder to find, regardless of which country you're in.