GCSE summer 2020 thread 6 - Carry on Corona Cohort, Further adventures aboard the Corona Charabanc.

That's a really good blog from Ofqual.

I think there is a fundamental popular misconception that CAGs are students actual grades and students are potentially being denied justice if they're adjusted after submission. They're clearly not and that's why exam centres were under strict instructions to not reveal them to candidates or parents.

The reason that one of the assumptions is that this year can’t be a bumper year is because exam results in any given year are norm referenced to ensure that no year is a bumper year. If kids had sat exams this year the number of each grade available would have been limited by a statistical model that meant that, no matter how well kids did on the exams, there would have been approximately the same number of each grade as last year (with small adjustments for prior attainment of the national cohort). Before any kids has sat an exam, Ofqual (or SQA) already know roughly how many of each grade there will be. When the exam marks come in they set the boundaries to give results as close to the expected number of each grade as possible. It is 100% certain that the grades given out on results day will be, on average, better than they would have been if the kids actually sat the exams as they’ve allowed a larger increase than they otherwise would have done.

neutralintelligence mass downgrading was mainly as a result of mass overinflating

I’ve just seen that Ofqual has published a new blog post. It’s aimed at A-Level students but also relevant for GCSEs.

ofqual.blog.gov.uk/2020/08/10/looking-ahead-to-a-level-results-2/

I think the Scottish government have done the right thing. Appeals should be possible and mass downgrading based on a statistical model that has as one of its assumptions that this year cannot be a bumper year should definitely be challenged.

Unfortunately, I think the sets were also based on y6 SATs to some extent. When looking at swapping a set 1 and set 2 pupils, I expect the science teacher who is coincidentally the person who does all the predicted grade modelling pointed out that the set 1 pupils had better SATs and better computer modelled predicted grades so should be given priority, There was also the male/female balance issue also mentioned. It was really annoying at the time as set 2 had lots of people who only got 5/6 in the year 11 mocks, so set 2 were not being taught the very hardest grade 9 material. My DS had to teach himself out of the CGP grade 8/9 workbook and then one science teacher commented on how he knew topics that hadn't been taught yet (not positive comments - she was implying he had a tutor, which he did not, he taught himself so that he could get the grade 9s he wanted despite being stuck in set 2).

I heard the news about Scotland and just thought- that sounds like a political decision taken in the face of public anger that is now going to be a complete nightmare for someone else to sort out. It was always going to be a blunt instrument to adjust grades on a statistical model but it seemed like the best of a limited number of options. I just don’t know on a practical basis how they can possibly collate the personalised evidence to sort this out.

Yes overlap at ours too. Remember sets are just a blunt tool for timetabling. Ability and performance will feed rankings

Our school rankings did not follow the sets. It’s pretty usual for the top of set 2 to outperform the bottom of set one on results day and teacher predictions and rankings reflected that this year in our school. I can’t vouch for every other school but would imagine some overlap would be the norm.

@Janie74 I’m the same, I’m torturing myself.
DS is quite chilled! 🤣

@Northumberlandlass - I was the same - DD has worked her socks off since she’s been at high school and I trusted the results to reflect that, more or less. Then the hullabaloo over the IB followed by the Scottish results really worried me, and the media frenzy this past week or so has exacerbated my concerns. And I really need to stop looking at Twitter

Did the school rankings follow sets? I previously thought everyone accepted that there were overlaps between the tops and bottoms of sets and my Ds school certainly moved very few people since the GCSE's sets were placed at the start of year 10 (single figures).
I worry about the rankings - my DS got 3 grade 9s in science mocks but is in set 2, but most of set 1 didn't get 3 grade 9s. If they stick to sets, then that is 30 pupils ahead of my DS to get a grade 9, which obviously he won't get. He might not even get a grade 8 since most of set 1 got 8s. So that is 30 pupils ahead of him to get a grade 8. It seems unlikely that the school will place a set 2 pupils number 10 in the rankings, ahead of 20 set 1 pupils who they seemingly considered better despite not achieving as highly.

Thank you @MadameMinimes, @Piggywaspushed and any other teachers on here - I hope that all schools were as diligent and objective as you were. It must have been especially difficult with a large cohort with multiple classes and different teachers - as is the case in DD’s school and countless others.

From DD’s point of view, the only subject in which I think she would have been borderline for grades will have been maths. On a good day she’d get a 6, on a bad one a 5 - just depending on the questions and her anxiety levels. The mock result was a high 5 (2 marks off a 6), but her teacher seemed confident she would pull herself into definite 6 territory with more exam practice. That was said at parents’ evening the week before schools closures were announced, so who knows?

Thank you @Piggywaspushed & @MadameMinimes for the info on rankings.

As the time gets closer, it feels quite frenzied.
Early on I was quite chilled - DS has worked hard for 2 years & I just want it to be a true reflection.

The Scottish Gov review - I really feel for those who have lost out on Uni places. What a mess. Do we know if the Scottish Gov & our Gov have used same product for calculations?

Thursday & A-level results will be interesting.

SeasonFinale- I do think that it looks like the decision to review the grading in Scotland is being driven by the public outrage and negative headlines. As you say, this year’s results actually narrowed the disadvantage gap in Scotland. I don’t think it’s actually that surprising that schools with disadvantaged students had more adjustments. I can see why teachers in schools with very disadvantaged cohorts would have wanted to give kids the benefit of the doubt, especially around the pass/fail boundary. They are also likely to have a larger proportion of students on those critical borderlines, where there have been more adjustments nationally.

I thought it was clear from the start that rank order was most important - I remember quite early dd and I looking at last year’s GCSE results and trying to work out her rank for each subject!

I’m pretty sure they’ll put more weight on recent work than year 10 exams @neutralintelligence? It wouldn’t make any sense to do otherwise. And it will be all work presumably, not just mocks.

I echo everything madame says. We knew rank was most important. What we didn't know was how much tolerance there might be of generosity or meanness. We have a red hot data guy who basically used the model Ofqual seemed to have done. It's annoying that he doesn't always see the humans in the system of course!

The rank order was awful for us as we knew it was important but it involved 400 students and 13 teachers, not all of them confident or experienced and our HOD is quite new. At times it became a little 'he who shouts loudest' but I think we got it as right as an exam would and, as I have as I before, I am pleased that some of the very anxious ones will get a grade they deserve rather than one that reflects anxiety and exam nerves.

I’m on quite a few teacher networking groups on FB and it’s no more than what the vast majority of people were doing to make sure they got this right. I’m not expecting students or parents who are disappointed by their grades to be very happy with me but I can stand by the process and a negative reaction is totally understandable in the circumstances. It’s going to be a very emotional day I think.

Excuse my phone and its inappropriate use of apostrophes!!

I am now worrying about something else. Did the teachers consider year 10 exams or just year 11 mocks? My DS had an undiagnosed mental health disorder that meant he couldn't handle the year 10 exams and didn't do well at all - maybe 3-4 grades below his year 11 mocks, by which time he was a little bit more aware of why he was struggling and could help himself a little. By the time real GCSEs were going to take place, he would have received a lot of treatment and be in a better place. But these things are confidential and I don't think most of his teachers were aware so his year 10 performance would have looked v bad compared to year 11, while others without this mental health issue would have been more consistent. Can appeals based on mental health issues be submitted?

The reason that the data won't be released until schools have results is because itbis a completely different set of data for each school ie. THEIR historic data and THEIR prior cohort data. They did not want people to incorrectly try to apply the data to their submitted grades. The school's will get this on Wednesday and will be able to check it is correct. Obviously if not this is one of the grounds to appeal!

Although there will be national statistics there is no national standardisation due to it being related to that school only.

MadameMinimes - re Scotland the most deprived although downgraded the most as a % still had a higher uplift than the least deprived already .

Yes it was perfectly clear in the guidance to centre heads that the ranking was of greatest importance to the modelling.

@MadameMinimes - thanks for your hard work. I hope your students and parents appreciate you.

It was really hard. The trickiest part was trying to decide between closely-matched kids who had different teachers.
Luckily we only had 2 people teaching my subject in year 11 this year as a result of a timetabling issue in September, which simplified things a lot. We each did a gut feeling ranking from looking at our mark books and data just to get our initial thinking clear. Then we used a scoring system that combined various kinds of data (some based on teacher judgement, some based on mock results, some based on other assessments) to create a ranking for the whole cohort.
We then went through that rank order one student at a time and looked at any student whose positions didn’t match up with what we thought to for things like personal circumstances, patterns of improvement, performance on individual papers etc that might justify moving them up or down from the rank order indicated by the raw data score. Looking at the data scoring also showed us examples where actually we didn’t have a good reason for our gut-feeling ranking and helped us to weed out unconscious bias. We had a rule that you couldn’t put a kid up or down the rank order without a logical argument from the evidence that we had available.
In a few really tricky cases we actually ended up scanning in their mock papers to compare over teams to separate out silly mistakes from lack of understanding. The top and bottom were quite straightforward but we really did have to look at some of the 5/6 kids in minute detail.

@MadameMinimes Thank you - that is good to know. It must have been very difficult indeed to rank students of very similar ability - I don’t envy you that.