I think a lot of my fellow Oxonians past and present have given you great information. I just wanted to add my own response (as someone who was also neither posh nor rich nor private-schooled), to reassure you that whilst it's natural to have doubts, I really do believe you would be very pleasantly surprised if you decide to accept your offer and make your grades, and end up at Oxford in October! As someone else said, mixing with people different from you is one of the best things about uni (imho) and
A few things I'd like to emphasise/add, to what others have already said:
1. Once you're there, it's actually really hard to tell what someone's background is (schooling, class, etc.) unless you actually ask them! Sometimes you can guess (e.g. if someone has an extremely posh, southern accent, then maybe they went to a big public school like Eton) but the overwhelming majority of the time, it's near impossible to tell!
Case in point: there was a very well-spoken, confident-sounding/seeming guy in my year in my college, reading German. As he came across so confident and had what I considered a posh accent, I thought he must have gone to a private school. It was only during conversation, when I mentioned in passing that I was from a comprehensive school and he went OMG ME TOO!!!!!!!!!!!!! that I realised how wrong I had been!
2. Even if someone IS from a much more privileged background to you, that doesn't mean that they are going to be a complete tosser, or that they'll be rude to you. One of the nicest, sweetest, most mild-mannered and unassuming people I ever came across at Oxford (the college husband of a friend from a different college) was the son of a Lord!
Obviously his family were loaded and he'd gone to very good schools, but he was a great person and loved sharing a grotty house off the Cowley Road with my friend as much as anyone else
3. Oxford takes instances of student bullying quite seriously, in my experience at least. So in the unlikely event that someone WAS rude to you about your background, you can assure your reporting it would be taken seriously
4. There is a heavier workload and a heavier focus on academia at Oxford - that cannot be denied. But that's not to say that most of us are nerds or spend all our time working! There's loads of fun to be had in Oxford, of any kind, including drinking and clubbing and those kinda things. Granted, I've heard the clubs in Oxford are not as good as the ones in London, but then that's probably true of many cities, given that London is the capital of the UK
5. Someone else above mentioned societies you can get involved in to meet like-minded people. I'd like to add that if you're looking for a particular society and it doesn't exist - you can set it up yourself
To summarise: you may well enjoy yourself more somewhere else, but I think you'll enjoy Oxford a lot more than you're imagining at present
The overwhelming majority of Oxonians have a great time there, and there's little reason (from what you've said so far) to think you wouldn't have a great time too
That said, I do appreciate the worry because I suspect you come from a background not so dissimilar from mine. If it would help to talk to someone who went to a comprehensive school and who applied under the Oxford Access Scheme (nowadays called Target Schools IIRC), do feel free to PM me