Could an algorithm ever actually award grades?

Our team will review your submission for job-related skills and abilities with the help of computer-assisted evaluation. This experience is designed to help the team focus on what matters, reduce bias, and make better, more objective decisions.

The assumption that teachers' predicted grades are more accurate than an algorithm's prediction seems highly speculative to me. It is well known that predicted grades are highly inaccurate.

I think it would be fairly trivial to formulate an algorithm that best retrodicts achieved grades based on predicted grades and all the other factors you can throw in. But people probably wouldn't like it.

The common impression seems to be that the problem was the use of the algorithm last year. If you ask me, the use of CAGs was just as problematic.

In an emergency situation like the pandemic, in which public examinations are cancelled, could an algorithm ever be trusted to award grades again? Is there any way it could work?

With their rejection for this year's exam series, I'm curious to see if people could ever have faith in them again, or whether they believe they're too unreliable. They were still used to award grades in 2020 if its awarded grade was higher than the CAG.

I think that they could work with a better appeals system but only God knows what that is.

Interesting use of trust. Last year's algorithm was 100% trustworthy. The problem is that it just didn't work. It wasn't a rogue algorithm. It did exactly as it was told to. Just the concept was all wrong.

Personally, I think a solution might be to get teacher assessed grades and then run an algorithm on them to flatten out the peaks and troughs. Some teachers overestimated their kids by a country mile whilst others perhaps didn't do their kids a fair service. If the starting point is a teacher assessed grade, you can tweak grades to fit expected outcomes across schools and the country much more accurately and fairly.

I honestly think that the problem with the algorithm was a lack of an appeals route. If they had put more thought into how pupils could challenge these grades, maybe things would have been different. I can't necessarily think of how Ofqual would go about doing that though.

Problem was/is, they will get literally hundreds of thousands if not millions of appeals. What workforce is going to work through all that and deal with each one adequately and fairly?

What we actually needed, was either a) exams to go ahead in traditional format but with a short delay. Covid cases were actually really low by late June time, and we did Eat Out to Help Out not too long after. People will say that I'm only saying this as I have hindsight, but honestly it was pretty obvious at the time, or b) online exams taken from home. Again people will probably say that's not realistic, but honestly... it was. A driven person if not swaddled in red tape could have done it. But that's a huge 'if' - this country, and especially education, just loves pointless red tape so that the bureaucrats can stay feeling important.

I did some analysis at my school sometime ago where I tried predicting students' GCSE grades for some subjects based on the data I had on them (CAT score, mock grades etc). On average I was better at doing it than the teacher in all but one of the subjects I tried it for - so overall my algorithm was closer to the final exam results than the teachers predictions were. The sample wasn't huge but I think it highlighted an issue that teachers are too influenced by students' behaviour, reliability in doing homework, general niceness and (in some cases) how much hassle the parents give.

Algorithms are responsible for student grades in normal years, so the question feels somewhat pointless.

Algorithms are responsible for student grades in normal years, so the question feels somewhat pointless.

Sorry, I should be more clear (although I think I mentioned what I meant in the post). I was referring to the kind of algorithm used in the absence of exams, as was the case last year.

I just wanted a snappy title, and I guess I didn't put too much thought into it.

Well if your question is that specific, then you've already answered it in this post - yes they could, last year.

But would anyone ever want to see one used again if a similar situation were to arise? Gavin Williamson already said that the DfE would put its trust in teachers ahead of algorithms for 2021 because of the circumstances (i.e. exams cancelled due to the disruption caused by the pandemic), but could there ever be a time when more trust could be put in the algorithm in those circumstances, or are they just too unreliable to ever be used in that way?

The algorithm awarded a very small number of grades. It was only officially used to award those higher than the CAGs. CAGs were officially used, so the algorithm wasn't really trusted that much.