You have to realise from your colleges prospective you haven't been able to demonstrate your ability in maths at a suitably high level, all of which is compounded by the fact its two years since you've taken GCSE maths. Most places will want an A for maths at AS and in general people who get bellow this tend not to do well and drop the subject after poor AS results. I think the fact that you haven't been taught A* maths is worrying as well because those skills are essential and taken as a given when starting c1/2, your college probably would be more lenient if your poor grade was down to illness or other circumstances rather than just plainly not being taught the subject.
Essentially when your taught AS maths everyone in that class will be of a certain standard and the tutors,as a result of this, won't account for people who are bellow this when teaching. You have to prove your ability rather than your college second guessing your potential, which will be hampered by your limit of mathematical knowledge.
It's still possible to enter as an independent candidate for GCSE maths (If you asked your college they may let you take it there this summer), which would make your college much more confident in allowing you to take maths next and achieving an A* isn't particularly difficult with proper revision.