Results: Anyone admitting to crazy algorithm uplift?

[vellvellvell]

Just reading about a student being bizarrely upgraded from a 1 to an 8 and wondering if anyone here had seen any strange uplifts in grades? ... www.tes.com/news/watch-gcse-2020-bizarre-rises-moderated-grades

That's really strange. So the teacher predicted a certain grade (CAG) but then the algorithm has predicted a much higher grade and they can use that one instead. I hadn't known that and just thought everyone was getting the CAG now

That sounds like a computer error.

Ah I've just read the article in the Guardian on this topic. One of the schools quoted is in my county (Dorset). It also mentions a school which, in recent years, has had amazing French GCSE results due to a number of bilingual / native speakers taking the exam. This year there were no such candidates, but guess what the algorithm did ....

@Julmust yes, students are getting either the CAG or the GAV (algorithm grade), whichever is higher

Here's the Guardian version. The sub-heading is "Exams fiasco continues as heads fear results raised by algorithm set pupils up to fail"

www.theguardian.com/education/2020/aug/20/schools-consider-appeals-against-gcse-grades-that-are-too-high

Well I suppose you could have a student with very high prior attainment, at a school with historically excellent results but never attends school so the teacher enters a 1 as the CAG grade but the algorithm doesn't compute personal circumstance and therefore gives an 8.
The algorithm is much more likely to be wrong the other way, i.e. student with rubbish poor KS2 results, or no KS2 data as new to the country, at a poor performing school but works their arse off, teacher knows they would get a decent grade in exam but algorithm doesn't compute personal circumstance!!

My dd got pretty much her predicted grades as did her friends . Don't think a 1 student would ever get a 8! Our girls have been bright and academic since primary !

There are quite a few cases posted by MN teachers, and more local and national news are highlighting cases where this has occured. Some whole schools who are in special measures have had rafts of top grades. One that stood out for me was a student who hadn't even been in school in the last two terms, given a U by CAGS, got an 8 by the algorithm. The school, to it's credit, are appealing. It has definitely worked both ways, and now of course, having sorted out one problem, the media have switched to the next.

@gubbbbbddaaaa Well, if you read the article, it actually did happen.

On Twitter, someone has been collating the results of small private schools whose results have gone up by 148% and in some cases even more. There have got to be some quite surprising uplifts in there. One tiny school now has results on a par with the highest-achieving girls's schools in the country.

twitter.com/queenofswords6/status/1294471488351985667

[quote artisanparsnips]On Twitter, someone has been collating the results of small private schools whose results have gone up by 148% and in some cases even more. There have got to be some quite surprising uplifts in there. One tiny school now has results on a par with the highest-achieving girls's schools in the country.

twitter.com/queenofswords6/status/1294471488351985667[/quote]
If it's a tiny school the chances are the uplift was due to CAGS not the algorithm.

Because for classes smaller than a certain number they just used the CAGS.

Yes, classes under 15 weren't subject to the algorithm

Algorithms can do crazy things.

Many moons ago, I sat sixth form exams (different country) that were then put through a bell-curve style algorithm.

I got given an A in one when my exam mark was 66%, because the school had a really strong cohort the year before in that subject and so was awarded lots of As to give out. I was on the top 5% of my school for the year I sat it but absolutely nowhere near the top 5% of the country which my mark then suggested.

I also got a C in a subject where my exam mark was 70% where the process worked the other way.

DD got a 9 which I suspect was the result of the algorithm - she would certainly have been predicted a 7 and possibly an 8 but can't imagine that she would have been predicted 9.

After the initial thrill she isn't too happy about it - says she doesn't feel she deserves it. But her grades as a whole do reflect what she should have got "on a good day" - they just eliminate all the ups and downs of unlucky questions, coughs and colds, period pains and brain farts.

they just eliminate all the ups and downs of unlucky questions, coughs and colds, period pains and brain farts.

And more besides. My DC got a 9 in a subject he was perfectly capable of getting a 9 in but was at high risk of not getting because of teacher-churn ... he had 3 different teachers in 2 years with two half terms of cover in between them. The latest new teacher was very good but up against it to get the curriculum covered before the exams. Thankfully the grade he got assumes she would have got it covered.

I expect the same sort of scenario accounted for many other optimistic CAGs too. I'm so glad most students got what they deserved in the end, but the algorithm-induced uplift is nuts and just creates further grade inflation with less justification.

but the algorithm-induced uplift is nuts and just creates further grade inflation with less justification.

She would have been happier with an 8, I think as she could certainly have got that "on a good day".

In the end, though, she's doing the subject at A Level so maybe it'll give her a motivation to work hard and prove to herself that she has a "right" to it.

Yes, classes under 15 weren't subject to the algorithm

Nothing to do with class-size, but cohort size.

If Eton enters 100 maths students each in classes of 4 people, they are still subject to standardization.

Also the threshold was not even 15, but around 4.

There was a secondary threshold of 15 to be partly used.

And it was based on the harmonic mean of the current year and past three years.

So for example, if you had:

2017 - 10 students in school for A Level maths
2018 - 10 students in school for A Level maths
2019 - 10 students in school for A Level maths
2020 - 10 students in school for A Level maths

then it's the harmonic mean of 10 and 30, which is 15, and in that case the weighting of the algorithm is (15-5)/(15-5) = 100%.

As an example, if you have 4 students each year forever, then that's the harmonic mean of 4 and 12 (they use 3 years), which is 6. In this case the weighting is (6-5)/(15-5) = 10%

Thus:

3 per year = CAG used in full
4 per year = 90% weight to CAG, 10% to algorithm
5 per year = 75% weight to CAG
6 per year = 60%
7 per year = 45%
8 per year = 30%
9 per year = 15%

Obviously with a brand-new subject you got full weight.

At Rye St Antony they appear to have had 21 students total in 2018. I guess a number of them will have had low cohort sizes. Anyway, all schools now get the CAGs for everything, so there will be bigger uprates for more students elsewhere, as Ofqual originally said 80% increase in A*s if they accepted the CAGs.

My understanding when they introduced 1-9 grades is that 8 = a solid A grade to a A* and that a 9 is meant to be slightly higher than that., with 7 A/B.

For GSCEs the first year to do most subjects on the grade 1-9 scale, marking scheme etc was only in 2018, so actually it would be reasonable to expect that 2019 students would have done a bit better in the exams, and 2020 even better had the exams taken place, because there are past papers and marking schemes and evidence of how exams were marked and graded to go on which didn't exist for the the guinea pig years (2017 was apparently Maths and English). This presumably would have been helpful in curriculum planning and exam preparation, and work for centre assessed grades too.

2018 was 3.4% 9 English Lit. 2019 3.6%. 2020 with inflated grades, 4.7%.

9-4 went from 72.9% to 73.4% to 79.7%

French more ludicrous - 5.0%. 5.1% now 8.3% 9, pass rate 69.7% 69.7% 82.1%

Maths from 2.9%, 2.9% now 4.2% grade 9, and 9-4 59.7%, 59.6%, 66.6%

PE 9 grade went from 3.6%, 3.8% to 8.4% Music from 8% to 14%. Pass grade up from 75.6% to 89.2%

Engineering from 50.6% passes to 76.8%. 9 grade from 1.6% to 5.4%

Apparently pupils have got better at Maths. So some of the increase would have happened anyway.

schoolsweek.co.uk/grades-boost-for-maths-after-nrt-showed-improved-performance/amp/?__twitter_impression=true

It's amazing the amount of anguish over Grade 9's going up from 4.7 to 6.6. Four years ago there weren't any 9s anyway.

Article about crazy algorithm grade hikes
schoolsweek.co.uk/from-1-to-8-the-exams-algorithms-bizarre-grade-hikes/

How do people know what was algorithm and what was CAG?

@iolaus - school will issue the CAG results. They were not moderated by Ofquel. For A levels we had the Ofquel allocated grades and the CAGs. Students can choose either.
However CAGs may not be the actual teacher assessed grades as the school senior management team may have modulated them before submitting to Ofquel.

The increase in French pass rate and higher grades was to be expected as it had already been agreed to bump up those subjects as they are seen to be a lot harder than Spanish GCSE - which is probably fair TBH. Given that decision the level 9 increase for French doesn't seem that extreme when compared to other subjects.

@elkiedee

My understanding when they introduced 1-9 grades is that 8 = a solid A grade to a A* and that a 9 is meant to be slightly higher than that., with 7 A/B.

For GSCEs the first year to do most subjects on the grade 1-9 scale, marking scheme etc was only in 2018, so actually it would be reasonable to expect that 2019 students would have done a bit better in the exams, and 2020 even better had the exams taken place, because there are past papers and marking schemes and evidence of how exams were marked and graded to go on which didn't exist for the the guinea pig years (2017 was apparently Maths and English). This presumably would have been helpful in curriculum planning and exam preparation, and work for centre assessed grades too.

@elkidee. That isn't right. The bottom of 7 was explicitly pegged to the bottom of the old A grade. There is absolutely no "b-ness" to a 7. Grade 8 does intrude a little into old A grade - the calculation of its position in each set of results for each exam is a very complex calculation shaped by where the 9 threshold is, but it is mostly Astar. Grade 9 is therefore "super A star".

This was explicitly built into the system when it switched.

4 5 and 6 are spread over B and C, with 5 being much more half and half than 8 is, if that makes sense.

Back on main topic, does anyone know why they kept the moderation at all for gcse (I believe it ended up as best of both). Surely by definition the only time a result was moderated up (so the only moderation still retained) would be if there had been strong cohorts at that school in the past but the teachers didn't think this candidate would perform like their historic peers ? Why should that grade be moved up?