To ask what is this ‘mutant algorithm’ that Boris Johnson is talking about?

[Totickleamockingbird]

I have no idea how an algorithm can be mutant. And if it was the algorithm that went rogue (if it did!), how come the top civil servant was sacked for it? I simply can’t visualise a mutant algorithm?
Can anyone please explain?

www.google.co.uk/amp/s/amp.theguardian.com/politics/2020/aug/26/boris-johnson-blames-mutant-algorithm-for-exams-fiasco

Has he been reading the plot to Tenet?! (about as comprehensible as he is) That had a bad algorithm in it.

One of Cummings ‘freaks and weirdos’ data scientists fucked this one up. It was also probably the result of one of their dodgy contracts which evaded procurement rules.

They are so fucking arrogant. Probably only one or two data scientists involved. No peer reviews. No risk reviews. Using shit, unclean datasets. No subject matter experts or diversity in the team undermining critical analysis. Just a couple of Cummings freaks and weirdos with a massive budget who are only answerable to him.

When they fuck it up, they get rid of the top civil servant who probably doesn’t even understand 99% of what the data scientists were doing. They probably just gave a weekly one page update to him with some big confusing words about Python and R and assured him they had it all under control. And here we are, another expensive embarrassment.

No idea. Maybe it involves sacking another high level civil servant.

Tellme
The scenario you’re describing is sickening. 2/3 people using flawed thinking and analysis responsible for the futures of an entire cohort of children.

Boris has a very cavalier attitude towards his own children, it stands to reason that he would care even less for the children of the nation:(

I thought his election message was the buck stops with me' .... 🤔
Perhaps he should just say sorry we fuxked up ... but actually perhaps just review the situation before it's too late (like this fiasco!)

Also - weren't they 11 year olds? They don't care about GCSE & A Level results.

Bluntness do you think the children would have understood if he'd tried to explain what the difference is between a mutant/shit/bad algorithm and a robust one?

That is the one that's puzzling me. One day he says it's robust, the next that it's mutant- that certainly does sound as if it's changed.

What he is signalling very clearly here is that he doesn't think it matters what he says, he doesn't owe the electorate (or their children) any explanations or even any thought. And that, in a Prime Minister, seems pretty unprecedented. Trying to lie- well, yes, that has happened in the past. Trying to avoid uncomfortable questions- certainly. But actively signalling that "I can stand here and talk complete drivel and it doesn't matter because I don't owe you lot anything"- that is bad.

This is the link I was trying to find.
Can anyone please tell me if it’s possible for members of public to sue this company? Some lessons need to be learned.

Sue them for what?
Running the PR campaign?

"A spokesperson for Ofqual said it initially contracted Public First without a tender to work on “insight on public opinion for this year’s exam arrangements”. The spokesperson added: “Public First is currently assisting Ofqual’s small communications team with an unprecedented amount of media interest in a complex policy area.”

"It has not been suggested that Public First was involved in developing the algorithm itself."

@Tellmetruth4

One of Cummings ‘freaks and weirdos’ data scientists fucked this one up. It was also probably the result of one of their dodgy contracts which evaded procurement rules.

They are so fucking arrogant. Probably only one or two data scientists involved. No peer reviews. No risk reviews. Using shit, unclean datasets. No subject matter experts or diversity in the team undermining critical analysis. Just a couple of Cummings freaks and weirdos with a massive budget who are only answerable to him.

When they fuck it up, they get rid of the top civil servant who probably doesn’t even understand 99% of what the data scientists were doing. They probably just gave a weekly one page update to him with some big confusing words about Python and R and assured him they had it all under control. And here we are, another expensive embarrassment.

This is what I suspect too. An unclean dataset, incorrect algorithm(s), and lack of proper control samples and even post-analysis review via some independent method. This is probably why no Royal Society scientist wanted to come aboard.

1. Ask them to provide the algorithm - they have, and it's been roundly condemed.
2. Ask them to include the full process of post-algorithm confirmation via control - this is what we need to see.
3. Ask them to provide which independent analysts were used to verify the final results obtained via this whole process - same as above.

I have also read somewhere that they tried to hire two Royal Society statisticians for this job who refused to come onboard. I will try to find that link - They refused to sign NDA's, which are required for any government contract.

It’s another classic Boris and Tory tactic of blame everything and everyone apart from themselves.

Reminds me of my children

”It has not been suggested that Public First was involved in developing the algorithm itself."
I want to know more about who was actually involved in this work and what was the role of this company. Could it be that they knew it will be a fuck up and thought they could manage it via a PR firm? Or was there anything else too? They should open up about what happened so lessons can be learned.

@Baaaahhhhh Where have they shared the algorithm? Can you please post a link? Thanks

I think it was an algorithm involving artificial intelligence/machine learning. You "train" the algorithm by marking its outputs as good or bad and over time it can adapt itself to provide the desired results.
That's the theory. However in practice they often just magnify human biases.

For example:
www.google.com/amp/s/mobile.reuters.com/article/amp/idUSKCN1MK08G

www.bbc.co.uk/news/technology-33347866

So it could mean the algorithm evolved in an undesirable direction = mutant

Unfortunately there is a trend for people to see tech as a magic, infallible fix for everything (I work in tech). But actually its usually expensive and doesn't work as well as envisaged.

This is exacerbated by having Dominic Cummings in a very influential position because he loves machine learning. See also: covid app fiasco

I have also read somewhere that they tried to hire two Royal Society statisticians for this job who refused to come onboard. I will try to find that link -They refused to sign NDA's, which are required for any government contract.

That's the same linked provided by Totickleamockingbird;

news.sky.com/story/a-levels-exam-regulator-ignored-expert-help-after-statisticians-wouldnt-sign-non-disclosure-agreements-12049289

The RSS are quite clearly blaming Ofqual, not the government.

Totickleamockingbird It was all over the press a couple of weeks ago.
Here's one.....www.bbc.co.uk/news/technology-53836453

This is a good analysis: unherd.com/2020/08/how-ofqual-failed-the-algorithm-test/

What does concern me are quotes such as: "Ofqual also ignored the House of Commons Education Select Committee’s call to publish details of their methods before releasing the results" which does give some insight perhaps into the Ofqual (rather than government) resignations.

The ‘mutant’ algorithm must have been such a shock for the politicians! After all they had requested it (in place of other forms of moderation). They were warned it was going to disadvantage students but Gavin Williamson then dug his heels in the sand and stood by the results it produced www.politicshome.com/news/article/alevel-results-day-arrives-amid-row-over-new-grade-system

Before they U-turned the policy after public outrage at its inherent unfairness.

So now Boris suggests the Government and his Minister’s management of the situation has nothing to do with this fiasco it’s purely down to the algorithm. Absolutely stinks!

I feel very sorry for the children going into yr 11 and 13 who may have missed chunks of curriculum and potentially face more disruption this academic year. Would you trust any new guarantees from Gavin Williamson that exams will be fair for them this year!?

And what a shock that the mutant algorithm produced this type of result in the first place

Either he is an idiot, he thinks we are idiots but most likely both.

The government asked for the principle of "no grade inflation" to be applied.

This means that 2020 results overall had to mirror those for 2019.

Teacher assessment on its own DID result in grade inflation, ie on this basis overall students had higher grades than in the previous year.

Therefore to ensure no overall grade inflation, a method had to be found to adjust some students grades downwards from the teacher assessment.

The method used was to reference what grades each school usually attained in exams over previous years and to standardise against that.

In basic terms, if a school usually got
20% A*
20% A
10% B
10% C
10% D
10% E
20% fail

Then that's what initially happened this year under the algorithm, even if the teacher assessment was
25% A*
25% A
15% B
15% C
10% D
9% E
1% fail

So the government had unequivocally instructed ofqual that there was to be no grade inflation and what followed was inevitable and clearly foreseeable weeks if not months ahead even by a very average Joe such as myself.

The fact is that there is no obvious or fair way to award grades for exams people didn't sit. They went for no grade inflation so as not to devalue grades achieved in 2020 and to keep university admissions from being disrupted by people overall having inflated grades thus courses being unexpectedly over filled.

The government then back tracked to be more popular. Giving out higher grades makes you more popular with everyone taking the exams and their families.

There was never any complex algorithm. OFQUAL adjusted the results as the government had instructed them to, and then the government changed its mind at the last minute and tried to shift blame for this.

Historically, teacher assessment was accurate in predicting exam results for just 16% of pupils. Any estimate of grades without taking the exam would always result in inaccuracies and in winners and losers.

And what a shock that the mutant algorithm produced this type of result in the first place

From the Sky link;

"Stian Westlake, chief executive of the RSS, said that if Ofqual had been more open, the issues with the algorithm "would have been identified sooner"."

"We could have helped Ofqual reassess the criterial," he said, citing the decision not to adjust the results for small numbers of students taking a particular subject at a school, a move is believed to have favoured private and selective schools."

"Even if Ofqual decided they didn't want to change anything, greater transparency would have helped the Department for Education prepare for the response to their decisions."

Mutant Algorithm is probably the name of his next love child.

@Clavinova

And what a shock that the mutant algorithm produced this type of result in the first place

From the Sky link;

"Stian Westlake, chief executive of the RSS, said that if Ofqual had been more open, the issues with the algorithm "would have been identified sooner"."

"We could have helped Ofqual reassess the criterial," he said, citing the decision not to adjust the results for small numbers of students taking a particular subject at a school, a move is believed to have favoured private and selective schools."

"Even if Ofqual decided they didn't want to change anything, greater transparency would have helped the Department for Education prepare for the response to their decisions."

Really the Government didn’t ask for a sneaky peak from Ofqual?

The Government must be very disappointed in the performance of these agencies they created, first PHE then Ofqual! I wonder which of their inner circle they will shoehorn in to head up Ofqual?

Boris has no idea what an algorithm is, but he wants to make it sound like it's the algorithm that's to blame for the screw-up. "Mutant algorithm" takes people out of the equation like it's all the algorithm's fault.

He means the algorithm that they programmed to reflect the fact that
private schools and grammar/outstanding schools with mainly MC, white students do far better than those with a large proportion of BAME, poor and SEN kids.

Not mutant. It did exactly what it was programmed to do. They just got caught out on it and are now claiming that deliberately taking out the possibility of an improvement in standards or that some kids actually do well despite the RI/Inadequate school environment was a random glitch that appeared from nowhere.