This sounds like a terrible idea. It sounds like colleges are just going to randomly pick applicants to accept, regardless of the contents of application.
The whole point of the departments choosing applicants is they, by being involved in teaching the courses, are best placed to choose suitable applicants. I'm sure there is already a range of measures to make this process fair, and take individual circumstance into account. Also, in terms of any subject specific references made within the Personal Statement, academics within the department of that subject are most likely to pick up on them, or someone who would, will be closer at hand.
Selection may have its bias, but it is there for valid reasons.
University should be focused on teaching the academic subject. You study for X years to learn about a specific area and not to go to a specific college. Switching to a college-based system seems very counter productive. Would the colleges each have staff from every department? If not, would you choice of subject affect your chance of admission through that college etc?
A change to a more 'random' application process wouldn't put me off. It would be more stressful, as its far less clear how an application would be assessed.