Well essentially debt is debt. Regardless of whether it will effect your credit rating or not, knowing that you owe someone money is a burden, knowing that you have £20,000+ to pay back is a burden.
Imagine your student loan debts coupled with mortgage payments, car payments, household bills etc etc, its just another expendature on top of all the other ones. Student debt on its own is all well and good, and on its own probably not something to stress about, but you seem to be over looking the reality of modern day life which for most people involves keeping a roof over your head, and plently of other bills and things to pay for.
You're right that no one forces anyone to go to university, but so many people have degrees these days that you really dont stand a good chance of getting anywhere career wise unless you have one. Also alot of careers do still require a degree. Its not as clean cut as you've made it sound.
Although I completely understand what you mean about loans giving you and many others the oppertunity to go to uni- including myself. I think however you have chosen a very idealistic approach to it all- which is probably a good idea, no point being stressed for the next few years.