Are this year's GCSE and A level grades going to be fair?

Hi. No conflict. As it happens, I am a consultant, not an "assessment" consultant. I don't know why it appeared like this in the Guardian. But it's the only item in that article that, as far as I know, isn't absolutely right.

And yes I did carry out an assignment at Ofqual in 2013, but that (certainly) was not to give them any "advice" on assessment. They know a lot already about that already. And in so far as they seek external resources on this, they go, for example, to the Oxford Department of Education. But my assignment did kindle an interest.

If you look at my website, you will see the nature of my consultancy activities, which have nothing to do with education, assessment, teaching or whatever, although I have worked with a few schools and many universities. But nothing concerning teaching or assessment.

Since 2013, I have had no contact with Ofqual, Ofsted, DfE... The only information I have had access to is that which has been published. And let me assure you that everything that I have written elsewhere about grades and this year's process, and been interviewed on, and has appeared in the press, is the result of my own work, at my own cost, in my own time. I have received no income from any of this, nor do I intend to receive any. And if you are reluctant to believe that, I can give you the names of those I have worked with, and you can contact them to determine whether or not they paid any money at all, including expenses for travelling to meetings.

I really have no vested interest, save a desire to throw a light on some issues that I think are important, like the unreliability of grades, and the lost opportunity that has just been experienced attributable, in my opinion, to the failure to explain, exactly, the 'rules', and the absence of any opportunity for a school to explain, and justify as required, their submissions.

Some may disagree, and believe that it's fine for the rules not to be clear, and for schools not to be able to offer explanations. That's fine. People may believe whatever they wish.

I am simply a concerned member of the public, no more, no less. I don't teach, I don't mark exams. But I have done some homework, and I have done some thinking. And, with everything I write, I do not seek that anyone 'believes', for I am not an 'expert' who should be 'believed' - nor should I portray myself as such, for that would indeed be manipulative. What I do seek is that people should think, to look at the sources as appropriate, and to come to their own conclusions.

But I don't know everything, nor do I claim to. And mumsnet, amongst other places, can be a very valuable way of exchanging knowledge, and learning new things. You never know who's there. For example, you might be Amanda Spielman or Sally Collier, and you might be willing to point me towards something I would find helpful. It's always dangerous to jump to conclusions.

As I have said, the purpose of my original post was to point out four new websites or postings that take this matter forward.

So I'm not sure that you are in a strong position to determine the degree of 'altruism', or otherwise, of my posts, here, and - much more thoroughly - elsewhere.

Your suspicion that there is a conflict of interest is understandable, but unfounded. So perhaps you can help me unravel my moral dilemma by informing of what those "interests" might be, and the nature of the "conflict"?

I am not an educational consultant. And I only mentioned that I don't have O level maths (which I don't) in response to your original posting.

And if you know more about the process than I do, that's great. There are many things I really don't know, and you might - so if there is a document I haven't read, or have understood incorrectly, I want to fix that. And, if you look, you will see that most of my (earlier) posts were asking questions, enquiring. Not declaiming.

Hi Buttery Puffin: My agenda is that I believe that all assessments, of which exam grades form a substantial component, should be reliable. The A/A* is only one example: the principle applies throughout, especially at the 3:4 boundary in GCSE English and Maths.

And catndogslife: yes, I quoted that phrase, but am happy to quote the whole thing - which is why I included the link, rather than left it as the few words. What important material have I missed, please? And yes, I agree about not jumping to conclusions - as in the last part of my response to titchy.

Dear titchy

Thank you. You ask why I posted the initial item in this thread. The reasons are these:

1. I wished to draw attention to the piece on The Guardian, published in Saturday 20 June - it contains some personal stories that I thought might be of interest to anyone in the mumsnet community who had not already seen it.

1. That article mentions a recently-started crowdjustice campaign that some might not know about.

1. As a result of the Guardian article, a new facebook group was created, with the intention of linking together those who might be affected by the bright-child-in-weak-school issue.

1. I also provided a link to a HEPI blog posted 18 June, which is a 'retrospective' of the last several weeks, and discusses the bright child issue, as well as some others - I also flagged it might only be of interest to teachers.

All of those items are new, having taken place over the last few days. And although the centre assessment grade issue has indeed been "debated, discussed, petitioned to within an inch of its life", it hasn't gone away, and there might be more to be both said and done.

May I ask you to clarify, please, what "ignorance" you believe I claimed? There are many things I don't know - like how statistical standardisation will take place, exactly. Where 'exactly' means what it says - so that any school would have been in a position to replicate the process before they submitted their grades so that they would be able to assess the likelihood that their grades will be confirmed, rather than over-ruled. So that any school will know how their outliers will be treated, and that the fears articulated by those in the Guardian article - and very many others, for the problems extend far beyond the 'bright child' - may be assuaged with confidence.

Perhaps many people think all that doesn't matter. That's fine. But it does have consequences - as discussed by Will Hazell, whose recent article suggests that "millions of proposed grades are set to be cut" (inews.co.uk/news/education/gcse-a-level-exams-2020-millions-proposed-grades-cut-generous-predictions-england-450236).

We won't know until the results are announced in August - but if Will Hazell is right, and millions of grades are cut, why did teachers have to go to all the bother, and anguish, of deciding them and then submitting them? That's what the 18 June HEPI blog is all about.

I don't use a pseudonym because I don't need to hide. And given that the HEPI blog I linked to in the original post also has my real name on it, it's not hard to make the connection, as catndogslife clearly did. Also, if you read the Guardian article, you'll find my name in that too.

And please tell me what "skin I have in the game". Yes, I do have some knowledge, and I do have some opinions. That knowledge might be helpful to others, and the opinions might stimulate lively and constructive discussion. But what "skin"? I'm not betting on the results (although that might be an idea a gaming website will wish to pick up); I don't have any relatives sitting any exams; I'm not on the board of Ofqual; I don't work for an exam board; I am not paid to write blogs. I am just a member of the public with an interest.

And I do hope you will read all of my material with a very healthy pinch of salt - and I hope that of everyone else too. Please don't believe a word I say. But - as quid pro quo - please think about it. Check the sources. Look through the analysis. And please come to your own conclusions. I might be wrong. My only request is that you do all that without the "sneer".

catndogslife - let me reassure you all my opinions and analyses are not based on mumsnet postings, but firmly rooted in Ofqual's own publications, dating back to 2016 and earlier, as well as the results of my own work at Ofqual in 2013. My 13 HEPI blogs www.hepi.ac.uk/?s=dennis+sherwood provide links to all the sources.

My "research" is fully robust, albeit controversial, for Ofqual have been in no rush to publicise the fact that GCSE and A level grades have been so unreliable, until forced to do so by an article in the Sunday Times on 11 August 2019 - see www.gov.uk/government/news/response-to-sunday-times-story-about-a-level-grades, where you will read Ofqual's admission that "...more that one grade could well be a legitimate reflection of a candidate's performance...".

That statement is not qualified at all, so, presumably, applies to all grades in all subjects, at GCSE, AS and A level.

You might like to think about that, and what that means for all those students with an A ... might another "legitimate reflection" have been A? And if the A is as "legitimate" as the A, why was the A, and not the A*, on the certificate? And in that uni's admissions officer's mind?

There is much more on my website www.silverbulletmachine.com/copy-of-news, and you might like to listen to the first 10 minutes of this episode of More or Less, broadcast on 23 August 2019, the day following the announcement of the GCSE results - www.bbc.co.uk/programmes/m0007rtv.

Ah... I've just seen this:

Michael Bell: And to add one more thing, Ofqual have confirmed in an email to me that the workings of the statistical standardisation model will NOT be released until the results are out, so there is no way of looking at what they are proposing in detail. (see www.facebook.com/groups/Alevel2020exams/)

Michael Bell is the chap, with his daughter Lexie, featured in The Guardian this weekend, www.theguardian.com/education/2020/jun/20/against-natural-justice-father-to-sue-exams-regulator-over-a-level-grades-system, and has been running this crowdjustice campaign www.crowdjustice.com/case/challenge-ofqual/.

Hi once more titchy - it seems as if we're been reading different documents. I can't find the details in anything on the Ofqual website: please correct me to where that oh-so-vague word "consider" (as in "For AS/A levels, the standardisation will consider historical data from 2017, 2018 and 2019. For GCSEs, it will consider data from 2018 and 2019, except where there is only a single year of data from the reformed specifications." - page 9 of the 'Guidance') is defined. If I am writing a computer program to do this "consider" just isn't good enough. You might also like to read of the experience of Huy Duong (see www.hepi.ac.uk/2020/05/18/two-and-a-half-cheers-for-ofquals-standardisation-model-just-so-long-as-schools-comply/), who wrote to Ofqual to ask specific questions of this type, but received only woolly answers, such as "We are still refining the detail of the statistical model and will publish more detail in due course” (that's dated 27th May).

You're right that 'grade inflation' used to be welcomed as a sign of better education, better teachers, brighter students (www.independent.co.uk/news/education/education-news/pupils-notch-up-record-gcse-results-5334652.html is just one example). But all that changed on 2010/2011 when Michael Gove was Minister and Dominic Cummings has SPAD. That regime is still in place today. And if you look at the data on Stubbs or JCQ and plot the graph, you'll see that 'no grade inflation' has underpinned policy ever since Ofqual replaced the QCA.

And as regards what I think: I really do believe this year's results will be fairer - see, for example, www.hepi.ac.uk/2020/04/18/weekend-reading-this-years-school-grades-ofquals-consultation/.

And ButteryPuffin: two things - publish the full details of "statistical standardisation" and allow for schools to explain their 'outliers'. Those would have made a huge difference.

Hi titchy - four thoughts if I may.

1. I don't share your optimism about flexibility. To take your example. The average grade for history for a school might indeed be 5.7. But the boards know more that that: they also know the grade-by-grade distribution. So they can, in principle, check that each grade, individually, is 'right' - and they can then identify that 10 x 8s and 12 x 1s is taking them for a ride, even though the average-over-all-grades is the required number. So, are the boards going to standardise on the all-grade average, or grade-by-grade? I don't know. The 'Guidance' document does not specify, exactly, one way or the other. For small cohorts, that raises problems. But what is 'small'? Once again, I don't know. And I don't think Ofqual have told me.

1. Why not ask schools to fill out a grid each year? In principle, they could, but with exams run by boards, they haven't needed to. The overarching principle underlying the whole system is to control grade inflation, not to give students the grades they deserve. Sometimes those might be the same thing, sometimes not. But it's 'no grade inflation' that has been the key driver since about 2010, and there's no reason to believe otherwise this year (although that is not explicitly declared). So Ofqual sets a policy - say, "only the top 5% of candidates can be awarded grade 9". To make that work, you need a rank order of all students, so you can slice it at 5%. In the past, for each subject, the rank order has been determined within each board according to the (single) mark given to each script. The scripts are marked, and then lined up, from top to bottom, with [so many] candidates given 65, [so many] 64, [so many] 63. At each board, the cohort is large, mingled across the country, and easily 'sliced'. The variability in the % of each grade, as seen year-on-year, is primarily attributable to the large number of students marked '63' (or whatever): drawing the grade boundary at 64/63 as compared to 63/62 makes a measurable difference.

This year, the ranking can't be done at the boards using exam marks. It has to be done at the schools using judgement - giving many more much smaller cohorts, each of which needs to be 'sliced' separately and independently. And to control grade inflation, each school must 'comply' of its own accord. That's what (I think) "statistical standardisation" is doing, and that's why all the rank orders must have only one student on each 'rung' - if they didn't, the boards would not know which of the 2 students on any one rung would fall which side of the grade boundary they need to draw to separate them. Which itself is a clue that the process will operate on each grade separately.

1. And I agree (very much) that this year's process will be fairer than the past. Because the teacher rank order must (my opinion!!!) be much more reliable than the rank order determined by necessarily 'fuzzy' (but still professionally sound) marking. That's good.

1. My disappointment, though, is that their year's process could have been really good, if only two things might have been done differently. Firstly, Oqual could have published the 'rules' of statistical standardisation so that the process was more transparent, rather than opaque. Secondly, there could have been some way in which schools could justify 'outliers' rather than have the boards over-rule the centre assessment grades unilaterally and without consultation. Yes, I know that's fraught with difficulties, and gives fraudsters a gift. Of course. But there are ways to counter that - I can think of many. But that's not going to happen, with three results. Some students will be treated unfairly (but fewer than in previous years, so that is 'better'). And - I fear - anger that teachers have been ignored. Worst, the possibility that teachers will be discredited: "You had the chance to show you could be trusted, and to submit the 'right' grades. We've had to over-rule [x%]. So you failed." To me that's bad news - and my benchmark of danger is more than 25% over-ruled, for that is the average unreliability of exam grades - about 1 in every 4 is 'wrong')

Indeed - they would have to adjust everyone's results. So I come back to my fundamental question: given that possibility, why ask for grades in the first place? Would it not have been far simpler to go the other way around, and for the boards to say to each school "our model predicts you will have [this many] grades [x] - please fill in the box with that number of names". And once again, I seek the explicit evidence that there will be 'flexibility' within any single grade. To control grade inflation, they need to ensure not just that the % 9 - 4 is OK, but that each individual grade is OK too. If they didn't do that, for my cohort of 100, I'd submit 65 9s and 35 3s, which keeps the % 9 - 4 at the 'right number', 65% (or whatever). Yes, I'm sure you're right that the odd one or two here and there doesn't matter. But I fear that the actuality will not be just one or two here or there - rather, it will be a systematic drift up, as everyone 'asks for more', as FFT indicate. But in doing this, no one has been 'greedy' (or only a very few). The problem is that each school can be sensibly within its local variation, but if each school is just a little 'optimistic', submitting say 1 more that the average, the overall result gives a variation far greater than that of the aggregate, whole-cohort, data held by the boards: the variation of this aggregate is far, far smaller than the sum of the individual variants of each compliant school. What Ofqual failed to do is to make it quite clear that the benchmark is the actual average of specific years, and to state, quite explicitly, that any submission greater than [this much] from that average is likely to be over-ruled. That's what I meant by 'exactly'. And there are lots of other anomalies too. ho hum. Anyway, thank you once again. By the way, there's an example of some of this on www.hepi.ac.uk/2020/05/18/two-and-a-half-cheers-for-ofquals-standardisation-model-just-so-long-as-schools-comply/.