I'd note there are plenty of good universities outside of the RG, depending on your particular subject of interest.
Also in a lot of cases this can be subject specific. Some courses tend to be more or less diverse. It would probably be helpful to know what your subject choice is.
In any case a lot of London unis tend to have more diversity in general (albeit sometimes that diversity skews to particular demographics anyway). UCL and SOAS have been/are fairly diverse in my experience, for example.