1 in 4 GCSE and A level exam results could be wrong.

[ehughes7777]

1 in 4 GCSE and A level exam results could b e wrong. Can I please refer you to the Facebook page of CAGE - Campaign Against Grade Errors, and the petition at Change.org. You could help do something about it and end this unfairness:

www.facebook.com/CAGE-Campaign-Against-Grade-Errors-GCSE-254326905277715/?modal=admin_todo_tour


www.change.org/u/me?source_location=my_petitions_dropdown


Please support our efforts to get the Government to review this.


Thank you.

[noblegiraffe]

Ok, so 1 in 4 grades could have been awarded a different grade by a different examiner BUT half of those would have been awarded a lower grade. I suspect that you don’t want the kids on a lower grade boundary to be moved down a grade.

If you want everyone who could have been awarded a higher grade to be given the higher grade, then actually the grade boundaries would shift to be higher because grade boundaries are set to ensure that a certain percentage of candidates get certain grades. Then you’d be back where you started - kids who had been moved to a higher grade would be moved back down again to maintain the grade distribution.

[dennishsherwood]

Hi noble giraffe

I think we might have talked about this last year... you are right about grades being wrong both ways, so if the entire cohort across all subjects were to be fairly remarked, about 1 in 4 grades would be changed, with about 1 in 8 being up-graded and 1 in 8 down. That demonstrates that grades are (very) unreliable.

And, as you also correctly point out, simply shifting the grade boundaries doesn't make things any better.

But there are several other possibilities - ways that really do work, and that solve the unreliability problem totally, and fairly. One possibility, for example, is not to award grades, but to declare on each candidate's certificate their subject mark AND (and this is the important bit) a measure of the 'fuzziness' associated with marking (defining the range of what Ofqual call "different but appropriate marks for the same script") for that particular exam. So, for example, for maths, this might be, say, 69 ± 2; for history, 66 ± 8.

That's just an outline... there's a lot more (and some new stuff since last year) on my website //www.silverbulletmachine.com.

[ehughes7777]

But noblegiraffe, why is it right, or fair, that those who are in the higher grade boundary, but wouldn't be if the papers were remarked can stay there, but those that are a few points under, but really should be in the higher grade, be denied that?

[Acopyofacopy]

I think the problem here is that nobody wants to grade those papers! The pay is rubbish and the training insufficient. It’s the last thing on Earth I would want to do.

[TeenTimesTwo]

I sort of think that except on the 3/4 borderline it doesn't really matter much in the long run at least at GCSE level.

Say someone does 8 GCSE they end up with 66666666. What the OP is saying is really they should have got 56666667. For most people that will make not one jot of a difference.

For some it may impact whether they can do a particular chosen A level. But they would be the 'borderline' to do the A level anyway, so it might actually be doing them a favour to not be allowed to start it (or to move schools to one with less exacting standards).

If something really matters, they can get it remarked (and you get the fee back if the grade changes I think?)

I would hate a system with scores and confidence intervals, sounds way too complicated for general usefulness.

[Bestseller]

A friend's DS gained 17 marks in an English remark a couple of years ago and went from a D to a B. That's life changing and the children it's going to affect worst are those with parents without the means/knowledge to get it remarked

[TeenTimesTwo]

Agree that is lifechanging (and I did caveat the 3/4 borderline).

[titchy]

What can the government do about it though? Exam marking is poorly payed, examiners under huge amounts of pressure to turn round scripts in a ridiculously short amount of time. If exam boards put the pay up that cost would have to be borne by schools in the form of exam entry fees.

[goodbyestranger]

Er.... allocate appropriate funds to schools?

[titchy]

What and force them to spend extra funding on exam fees - to exam boards which are private companies? Rather than teachers? Or teaching assistants? Or building repairs? Or equipment?

[goodbyestranger]

There's no point in any of that if the DC can't rely on properly marked exams at the end of school, in an exam led world. So your question was: what can the government do? My reply: pay enough to schools. Or nationalize the exam boards perhaps. Also, why 'rather than' as opposed to 'as well as'.

One of my DS's also went up 17 marks on a re-mark to a top A* at A level - it's just absurd. Another DS had his shockingly bad RS GCSE forwarded to Michael Gove then Education Secretary as an example of just how incredibly slack some examiners can be. I mean - just do the job or don't take the money. These DC deserve better than that.

[Witchend]

Each candidate's certificate their subject mark AND (and this is the important bit) a measure of the 'fuzziness' associated with marking (defining the range of what Ofqual call "different but appropriate marks for the same script") for that particular exam. So, for example, for maths, this might be, say, 69 ± 2; for history, 66 ± 8.

Subject mark, fine, but "fuzziness level"

How on earth do you work out 1 in 4 anyway?
Looking at figures for example in 2015, around 15% of the appeals resulted in a change.
We can assume people only appealed if they had good reason to think they'd go up or were near a boundary. Even if we assume that there is an equivalent 15% that would change to go down (but for obvious reasons won't be appealed) the majority of exams will be far enough between grades that even if the score is not right, that it would stay the same.

So the actual percentage of exams that are at the wrong level is going to be far less than 15% which is considerably less than the 1 in 4 chance you are giving.

[noblegiraffe]

This isn’t about incorrectly marked exams, it’s about how exams can be marked totally correctly by two different markers and come out with two different grades due to the subjective nature of the marking (not such a problem in maths tbh).

The idea of giving scores and confidence intervals is just way too complicated, and I agree it only really matters near cliff-edge grade boundaries like 3-4.

[oneteen]

My issue is more to do with how grade boundaries are set...surely when an exam paper is designed it should be set so that a given mark is a pass, good pass, very good pass and excellent pass (whether grades are used or not) based on the level of competency. It should not be based on well such and such a % of the cohort can only achieve a grade 9,8,7 - 4 etc.....Why can only a certain % of DC's just achieve a grade 4 - we set some of our children up to fail before they have even arrived at secondary school never mind sat an exam.

[errorofjudgement]

I haven’t clicked on the link, but poor and incorrect marking does happen. Even in maths and science.
My DS had 1 A2 physics paper remarked and his score went up by 12 marks that gave him an extra 4 UMS and got him his A grade and place at first choice university.
These exams represent the culmination of 14 years schooling, and all the work that goes into that. Not marking the paper correctly is an insult to the student (and the teaching staff).

[CherryPavlova]

The problem is this is completely inaccurate rubbish.

[CherryPavlova]

OfQual provide very rigorous oversight and heavy sanctions to exam boards.
Some independent schools argue that with grade boundaries ensuring only about 30% get an A grade it’s unfair because they have the brightest ALL their pupils should get As.
It isn’t 1 in four at all. A very, very small percentage receive grade changes on reviews of marking.

[Rosieposy4]

The number of kids changing grades has dropped massively due to new OFQUAL guidelines. All the exam boards put rigorous measures in place to ensure the best quality of marking possible, but you have to use humans. MQ cannot test all the skills OFQUAL require, and with the best will in the world the odd marker will be a bit slack or slightly lcking in knowledge.
Having said that over the past couple o years far fewer students at my school have gone up compared with staying the same or even occasionally dropping a grade.

[dennishsherwood]

Hi Witchend

The evidence that 1 grade in 4 is wrong is this chart, presented by Ofqual director Dr Michelle Meadows at a conference in 2017, www.gov.uk/government/news/presentations-from-ofquals-summer-series-symposium-2017:



This shows the results of a detailed study (assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/681625/Marking_consistency_metrics_-_November_2016.pdf) in which huge numbers of scripts (entire subject cohorts) were double-marked, as if they had each been appealed, regardless of the original mark given. This is important, for it implies that the study is not biased to appeals of scripts marked close to grade boundaries. The key question then asked was "Is the original grade confirmed by the re-mark, or is it changed?".

The vertical axis shows the probability that the grade is confirmed, for each subject. So, about 85 scripts physics scripts in every 100 have their grades confirmed, and 15 changed; for English language, about 70 confirmed, 30 changed; for history, about 60 confirmed, 40 changed. From the candidate's point-of-view, if the grade is changed after a re-mark, the original grade was wrong.

There is no published result averaged across all subjects, but this average cannot be 'better' than physics (85 right, 15 wrong), nor (we pray) 'worse' than history (60 right, 40 wrong). So around 75 right, 25 wrong is likely to be a reasonable estimate. 1 grade in every 4 is wrong.

The grades used in this study were A*, A, B...; reliability is (even) worse using 9, 8, 7... because the grade widths are narrower.

None of this is due to errors in marking (although these can and do happen) - it is all attributable to the fact that, as Ofqual admit (ofqual.blog.gov.uk/2016/06/03/gcse-as-and-a-level-marking-reviews-and-appeals-10-things-you-need-to-know/), different markers can give "different but appropriate" marks to the same script. That's what I referred to as 'fuzziness'.

So the grade your DS or DD is awarded is determined by the lottery of which marker marked the script...

[dennishsherwood]

Hi CherryPavlova

You're right that only a small percentage of grades are actually changed (around 1 ish percent).

That's because there are very few appeals.

If you have a moment, take a look at //www.silverbulletmachine.com/single-post/2018/10/28/No-Pearson-992-of-your-grades-were-not-accurate to see why the claim by Pearson/edexcel that "99.2% of our results were accurate on results day" is very wrong...

[ehughes7777]

So, if Ofqual admit 1 in 4 grades are wrong, why should someone who is in a higher grade, but on another remark would be in the lower grade, be allowed to stay in the higher grade, when someone in the lower grade, but on another remark would go up to the higher grade, not be able to do anything about it? This issue has life changing consequences.

[Rosieposy4]

They are not ehughes. If you have a paper remarked the new mark stands, whether up or down.

[CherryPavlova]

Actually the number of appeals is probably higher than one might imagine. Most independent schools routinely resubmit papers for marking reviews.
Exam boards are also required to have robust internal moderation and review processes that check the marking of all papers.
If one appeal is based on, say, an ambiguous format for a question then the papers of all potentially affected papers are reviewed.
The regulations require boards to act in best interests of students and grades are upgraded and students given benefit of doubt as far as possible.
The impact is considered in any identifiable marking or grading errors and action is taken where the board has made a mistake.

[dennishsherwood]

Appeals are the last-chance saloon. Far better that the assessments of our young people are right-first-time, trustworthy, reliable. But GCSE and A level grades are none of these things.

UK assessments are essay-based, with human markers - in contrast to an unambiguous multiple-choice system marked by computers. Different (human) markers can give the same script (slightly) different marks, for two reasons:

(1) One marker made a mistake. This can, and does, happen, but as CherryPavlova correctly points out, this is relatively rare, and the exam boards have very thorough quality control procedures.

(2) The two different markers have (slightly) different judgements, and so give (slightly) different marks, perhaps just one mark apart. Neither mark is 'right' or 'wrong'; there are no marking errors...

...but that one mark difference in judgement can, and does, have life-changing consequences. If the [x,y] grade boundary is 45, then being marked 44 has huge impact. In my previous note, I referred to this as 'fuzziness', and if a 'fuzzy' mark straddles a grade boundary, the grade awarded is the result of the lottery of which marker marks the script. A lottery with a one-time bet, for Ofqual's new rules on appeals do not allow the original 44 to be changed to 45, even if 50 other markers were to re-mark the script.

An assessment process that awards grades is trustworthy and reliable only if two conditions are always true:

(1) The (large) majority of 'fuzzy' marks fall safely within grade widths, and do not straddle grade boundaries. And
(2) For those few scripts whose 'fuzzy' marks do straddle a boundary, each individual case is considered wisely and fairly, before the results are published.

For GCSE and A level, both of these conditions fail, big time.

Firstly, grade boundary straddling is very common: for intermediate grades in subjects such as history, English Language and English literature, some 50% or more of all scripts do this - only very high 9s/A*s, and very low 1s/Us are 'safe'. I'm not making that up. Look at Figures 12 and 13 in assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/681625/Marking_consistency_metrics_-_November_2016.pdf.

And secondly, you don't need me to tell you that "the individual and wise consideration" of borderline cases just does not happen.

Grades aren't fit-for-purpose. They are untrustworthy, unreliable, misleading. And - most importantly - unfair. There are much better, more reliable, fairer ways of awarding assessments, even in an exam-based system - which itself is a big issue, but if exams are there, then at least the outcomes should be fair!

But Ofqual aren't listening. Their main policy is to make it harder to appeal (which is why the number of appeals has gone down) - so deliberately covering all this up.

So your opinion, your voice, is important. If this makes any sense, tell your friends, speak to your children's school. And here are some links too

//www.change.org/p/department-for-education-one-exam-board-for-gcse-and-a-level-exams-to-end-unfairness-in-grade-lottery

//www.facebook.com/CAGE-Campaign-Against-Grade-Errors-GCSE-254326905277715/?modal=admin_todo_tour

//www.silverbulletmachine.com

Thank you!

[mocha70]

The current system is completely unfair as some exam boards let you see the paper before requesting a re-mark (eg. Edexcel) whereas AQA don't. If your child has a 4 in an AQA subject and you suspect they would have been capable of a 6 you cannot risk a re-mark is if for some reason the mark goes down it would end up as a 3 = fail.

The exam boards also have to give you your £46 remark fee back if the grade goes up so they have no incentive to revise a mark upwards.

Furthermore the re-marks are not re-marks but reviews of marking so you are not being given a proper remark by a different person.

I also think that grade boundaries in modern languages need to be adjusted depending on the number of native speakers entered as otherwise it is harder for those who are not bilingual to get a top grade in modern languages.

In response to the person near the top who said does it matter if the grades are slightly wrong for GCSEs well yes it does. Some sixth forms require certain grades to do certain A levels and some Employers will want a higher grade than 4 in English language for eg.