More that 1.5 million wrong grades will be awarded at A level, AS and GCSE this year

[dennishsherwood]

Across all subjects, and across A level, AS and GCSE, about 1 grade in every 4, as awarded this August, is WRONG. That amounts to over 1.5 million wrong grades this year. But no-one knows which specific grades, or to which specific candidates. So maybe your DD or DS is an unwitting victim.

In response to a hum-dinger of an article on the front page of the Sunday Times on 11 August (www.thetimes.co.uk/edition/news/revealed-a-level-results-are-48-wrong-xsj33jvnh), Ofqual immediately posted a statement on their website (www.gov.uk/government/news/response-to-sunday-times-story-about-a-level-grades) which includes these words

"more than one grade could well be a legitimate reflection of a student’s performance ".

You might like to read that again, and think about it.

I think that what they are saying is "“grade [X], in subject [Y], is just one possible legitimate reflection of your performance, but there are others too - though we’re not telling you what those other grades might be, or whether they are higher or lower”.

To me, this is outrageous. And if you think so too, please complete the very brief survey on www.surveymonkey.co.uk/r/gradereliability.

If you would like more information, and to check the evidence, click on www.hepi.ac.uk/2019/08/15/dear-ofqual-%EF%BB%BF/ and follow the embedded links...

Unfortunate typo!

I don't have issues with Q by Q marking but with different examiners marking ach question (which isn't done in lit)

That's not how Comp Marking is done, though..

In terms of practising teachers marking, in my own subject the principal on one paper was still teaching. I had teams of 7 for both papers and the vast majority this time were teachers, with a couple of retirees and a couple on mat leave..

It's not a perfect system ,but the exam boards are definitely looking at it.

Omg horrendous typo, many many apologies

I was mentioned in this context during the interview with Nick Gibb, the Schools Minister, in the 07:10 slot on this morning's Today programme (Thursday); I am also participating in tomorrow's (Friday) More or Less at 16:30 on radio 4.

have reported your post Rosie to try and get the typo corrected, you may need to report it yourself though.

Thank you my visions, will do so

Thanks for highlighting this issue - it’s really worrying that there’s such a gap between the exam boards’ stance and the impact on our children. DC has just missed an Oxbridge place because of a humanities exam marked at 0.6 UMS (0.3%) below the A needed, and the College doesn’t seem to be aware that the grades themselves are far from reliable, although I do understand they have to differentiate somehow. The remarking options seem designed to confirm the original marks, so we are creating a group of students who feel very let down by the system.

Thank you. You're right about the college not knowing. But they are not alone - most people/teachers/schools/colleges/universities... don't either. So the more people that know about it, the more people that have read the HEPI blogs (www.hepi.ac.uk/?s=dennis+sherwood), the better. Please spread the word!!!

Here is a link to my interview on Radio 4's More or Less :
www.bbc.co.uk/programmes/m0007rtv

Thanks for all your work on this!

That's lovely, thank you!!!

There's also a bit in the interview with Nick Gibb, the Schools Minister, on Thursday's (GCSE results day) Today programme www.bbc.co.uk/sounds/play/m0007rkr - pick it up at around 1:12:20.

Thanks - this is interesting, albeit worrying. I agree a numbers based system, like in continental Europe, would be far preferable to the Letter based cliff edges we have now.

Have you looked into the impact of the change from full remakes to reviews of marking in 2017 on grade changes? Having now had two children who have had unsuccessful reviews of marking for English Lit A level papers where the marks were unexpectedly low, and where teachers felt the quality was at least equal to other papers which had received much higher grades, I wonder whether the new system builds in confirmation bias, so that the reviewer tends to maintain the previous grade, whereas a full remark might have changed it? Especially when the exam boards are incentivised not to make changes by the charging system. It does seem contradictory that the Ofqual work you highlighted showed such unreliable marking for English but the reviews of marking only change 18% of A level English papers submitted for review, according to the government figures.

Hi Gettingthroughtheweek - thank you, and yes I have seen the documents you refer to.

I don't think the problem is confirmation bias. I think it is the need to comply with Ofqual's rules, which state that a remark can be allowed ONLY if there has been a "marking error" - any "difference in academic opinion" is disallowed, as, for example, stated here

"It is not fair to allow some students to have a second bite of the cherry by giving them a higher mark on review, when the first mark was perfectly appropriate." (See www.gov.uk/government/news/fairness-at-the-heart-of-proposed-changes-to-marking-reviews-and-appeals-system.)

I think this is really bad - see www.silverbulletmachine.com/single-post/2018/10/28/Biting-the-poisoned-cherry---why-the-process-for-school-exams-is-so-unfair.

Let me also point out that in 2017 AQA were severely reprimanded for failing to comply with the letter of Ofqual's rules, so that served as a powerful deterrent - see www.tes.com/news/aqa-admits-breaking-rules-exam-re-marks .

Why, in 2016, did Ofqual change the rules for appeals, so as to make it HARDER to appeal? What other regulator deliberately denies access to justice?

I am glad I found this thread, there is so much I agree with. I had already forwarded the programme link to several friends. I love More or Less and @dennishsherwood's piece was excellent.

The inaccuracy and cliff edges really bother me. For example, DS, like so many others, would like to do medicine at Cambridge. Standard offer AAA, he got AAA, but both within a few marks of A. If you still had UMS, both would be 89%, but due to cliff edges, his 2 x 89% UMS are worth less than a 90% and an 80%.

We have requested reviews of marking, but as people have said, access to that is a lottery due to cost and schools.

At our school the exams office was open for a few days. We ordered priority scripts which arrived 23rd Aug PM, then the exams office was closed until 3rd Sep, giving us very limited options for getting the most promising papers reviewed first and then more if necessary - I don't really want to spend £300 for something which has a 20% chance of success.

Will follow the other links when I have time.

Good luck with your campaign!

This is a travesty, for kids, parents and teachers.

Particularly, as usual, working class children.

It didn't used to be like this: in the 1980s, predicted grades were often spot on.

Marking was seldom questioned as it was generally done correctly and run efficiently and accurately by non profit making exam boards.

I wonder if re marking is now another nice little earner for the exam boards?

This is a shameful state of affairs, whatever the reasons, in a country which used to have a world reknowned examinations system.

Thank you, Monkey2001 and Woman19. Your point about this being a travesty is true. And it will remain a travesty until one of two things happens.

(1) Someone takes Ofqual to court on the grounds that the statement they made on 11 August that "more than one grade could well be a legitimate reflection of a student’s performance" (see www.gov.uk/government/news/response-to-sunday-times-story-about-a-level-grades) is an admission of their failure to fulfil their statutory duty "to secure reliable assessments", as defined by Section 22 of the Education Act 2011 (see www.legislation.gov.uk/ukpga/2011/21/section/22).

(2) Parent, schools, universities and employers kick up such a fuss that the authorities have to listen.

If you know a good lawyer, try (1).

And under all circumstances, how can as many people as possible be made aware of this great injustice? See, for example, www.thetimes.co.uk/article/revealed-a-level-results-are-48-wrong-xsj33jvnh, www.telegraph.co.uk/women/life/can-children-trust-a-level-results-used-think-son-opened/] and www.hepi.ac.uk/2019/07/16/students-will-be-given-more-than-1-5-million-wrong-gcse-as-and-a-level-grades-this-summer-here-are-some-potential-solutions-which-do-you-prefer/

@dennishsherwood do you ever go on The Student Room. There are some strange stories of schools where lots of students are getting 2 grades below their usual grades in AQA Sociology and English. The reviews of marking are coming back unchanged - see www.thestudentroom.co.uk/showthread.php?t=6085336&p=85187840&page=48#post85187840

It is not in the interests of the exam boards to change marks, I wonder whether there are different approaches between various examiners at various boards. If too many change it is significantly embarrassing - bad for reputation and finances. Reviews of Marking should be carried out by somebody independent.

We're in Scotland but have the same issues.

DS was awarded a D in one of his subjects after consistently performing at either high B or low A level.

On further investigation by the school and exam board, it transpired that the board had marked all of the first paper, but only the first question of the second paper. 70 marks worth of answers had gone astray and couldn't be accounted for. Luckily school submitted his prelim (mock) papers and other supporting evidence and the board awarded him his predicted grade of B. No explanation of what could have happened to his papers.

So it's not just dodgy marking, there are lots of other dodgy things going+ on. Luckily for DS it was his Nat 5 exams rather than the Highers, which are used to apply for Uni. He's continuing in the subject so in the long run, the grade won't matter as much. But it's still fairly crap. He went from being confused, to shocked, to angry, to upset, to confused again in about 5 seconds. Started to doubt himself and wonder if he really had done poorly enough in the exam to get 26%.

He's a fairly resilient lad who has bags of confidence so didn't let it affect him too much. Had it been DD though we'd have had tears for weeks and it would have ruined what little self-confidence she has.

Hi Monkey 21 - thanks for the links to the Student Room. Under the 2016 rules, exam boards are allowed to re-mark ONLY if there is evidence of a 'marking error', such as the 'lost' marks spoken of in the last post. 'Legitimate differences of academic opinion' are NOT RECOGNISED, and so the candidate is stuck with the lottery of who marked the script first. My simulations of subjects such as English Language show that the (legitimate) fuzziness of marking can straddle 3 or even 4 grade boundaries, so a 2 grade difference is common. But disallowed.

And your point about embarrassment is true. Here is a quote from page 70 of a report published by AQA in 2005 (yes, 2005):

"However, to not routinely report the levels of unreliability associated with examinations leaves awarding bodies open to suspicion and criticism. For example, Satterly (1994) suggests that the dependability of scores and grades in many external forms of assessment will continue to be unknown to users and candidates because reporting low reliabilities and large margins of error attached to marks or grades would be a source of embarrassment to awarding bodies. Indeed it is unlikely that an awarding body would unilaterally begin reporting reliability estimates or that any individual awarding body would be willing to accept the burden of educating test users in the meanings of those reliability estimates."

(cerp.aqa.org.uk/sites/default/files/pdf_upload/CERP_RP_MM_01052005.pdf)