There aren't that many GDL scholarships to go around, so not getting one is certainly not an indictment of your CV or your pupillage chances - whether not getting an interview suggests a more genuine problem depends on what proportion of people they interview. If they interview 90% and deliberately excluded you that would be significantly more worrying that if they're super selective even at the interview stage - which Inn was it?
As for why you didn't, from what you say it sounds like the reference was the obvious problem. Did it mention your disciplinary problems or just suggest your grades weren't going to be as good as you suggested? Neither's great but the combination would probably ring alarm bells. Let's face it, they wouldn't ask to see the reference in advance if it wasn't going to be a factor in deciding who to interview. Is there anyone from Oxford who could give you a decent (or at least better) reference? Maybe a tutor you had tutes with for one term and gelled with? There's no rule that it has to be your main college tutor. If that is the biggest problem, then it's easily resolved to some extent once you can get a reference from one of the GDL tutors (assuming the disciplinary problems aren't going to continue). Be warned however that in most cases, a tutor will see you for about two hours a week as part of a class of 15, so references will often be pretty generic. I think most chambers would prefer the academic reference to come from an academic tutor - I've seen one or two specify that.
The other problem that jumps out at me is a lack of any legal experience/exposure at all. No one is going to expect a pre-GDL student to have tons of experience, but I suspect something - one mini-pupillage or a week in a solicitors firm - could have made all the difference. Certainly I was told that was the main reason I didn't get a GDL scholarship (I got a fairly big BVC one once that problem was resolved, though there are far more of those to go around).
In conclusion, get a decent reference and some legal experience and keep building up the EC's and you can probably be classed as a credible, maybe even good candidate - but bear in mind that the odds aren't great even for very good candidates at the moment, so if you're worried about the cost and taking the risk, and it isn't a burning ambition, then its probably best to look at a different career. To be honest, even if you had got a scholarship I'd be giving broadly similar advice - the risk:reward ratio isn't particularly great even for strong candidates for whom it's a lifelong ambition. I don't buy the argument that you can only succeed if you're super passionate and committed, but if you just want a respectable grad job and don't really care what it is, surely it's better to go for one that doesn't require two years of study and nearly thirty grand of costs.