If money is a concern you are going to come out of Warwick with much less of a debt but, apart from that, it is probably purely lifestyle.
If you are into status I suppose internationally Imperial has a rep but I think anyone who is hiring Maths graduates is going to know that the Warwick degree and the standard of the students is as strong.
At Warwick you are going to get the full student experience of living on campus, etc Imperial you are going to get... London, with all it's up and down-sides. Their halls are spread out, including some new ones out on the A40 past Acton hemmed in by several multi-lane roads and there is a lot of travelling involved to do sports activities. However, when you do hang around where you are taught at Imperial you are surrounded by the Victoriana and post-Great Exhibition history of the area.
Warwick has a great but different feel more like MIT or another non-Ivy League US Uni or, less kindly, a business park. Facilities are good. It hasn't got the architectural vision of Bath or Sussex, but a lot of it is sleek and new (though most of the halls are slightly less up to date) and you are surrounded by green. Because people are living the campus life in the first year there does seem to be a friendliness and sociability at Warwick that might be harder to reproduce at Imperial perhaps. Both have large overseas student populations and at either place doing Mathematics you are going to get shed loads of work. They both work you hard.
I think it comes down to what you think of London I suppose. It is one of the world's greatest cities and it is fun to get to know and doing it at University at least gives you a support structure. But you can do that later in life too and maybe have a more focused student experience at Warwick.