Would I be crazy to turn down a place at St Andrews uni for Warwick?

Okay so i'm having a really difficult time trying to decide between St Andrews and Warwick.
I've had unconditional offers from both but cant really seem to settle on one.
I have visited both unis and done lots of research. I have actually been to uni before (Edinburgh) but I am starting again after a disastrous first year.
I know what uni life is like in big city, and i'm from a small village so I know what a quieter life is like too, so those differences don't worry me.
The only thing that worries me is that St Andrews is even further away from my parents house, and I am worried that it could be an issue. Apart from that I see no other problems with St Andrews other than possibly elitism from some students ? (which worries me slightly as I went to state school and come from a modest economic background).
Warwick also doesnt have many negatives for me, other than its maybe not as prestigious as St Andrews but it is a target uni for big employers which is an advantage.

Does anyone have any insight into this matter as I'm going mad trying to make a decision and I don't seem to be getting anywhere. If I go for the safe option of staying closer to home (ie Warwick) I would be turning down St Andrews- would I be crazy for doing this as it is a highly sought after uni??