Exam data nerdery

[noblegiraffe]

I'm fairly sure that there are other nerds on this site so I thought I'd post some stuff I'd collected in case anyone else is interested. Also, if you see something in the data, maybe we could chat about it.

First offering is this widget from Ofqual. You can look at the grade distribution for various subjects, or you can see how grades in one subject affects grades in another subject (the link between maths grade and computer science grade is quite pronounced, for example).

https://analytics.ofqual.gov.uk/apps/GCSE/9to1/

What I noticed is that the grade distribution for English language is weird. And there are far fewer top grades awarded than for Maths. Why? (This is with the resit data stripped out). It's the same cohort that sits both so it's not about prior attainment.

Second offering - A-level top grade data.

Maths is the most popular A-level. 41% of those who sit it get an A*/A which is probably because it has much higher entry requirements than, say, sociology.

However, the reason for the high entry requirements is because attainment at maths A-level is massively tied to attainment at GCSE, not because maths teachers are only interested in kids who can get top grades.

Third: a link to all the GCSE data, by exam board.

https://x.com/mathsnot/status/1695116584258322499?s=61&t=U9XrcF693-JpMxeIueYG7g

AQA is the most popular exam board for English language by far. However, only 2.5% of AQA candidates get a 9. OCR entry is tiny, but 8.7% of their candidates get a 9. Why?

That 3/4 data for English is indeed strange.

As for the top grades, it's really very difficult to access the top bands of the English mark schemes, much less the top marks ... but that should be resolved by the placing of the grade boundaries.

Yes, I love that Ofqual site. My family think I'm weird 😀
Quite struck by, but weirdly enjoying, the 'listen to the chart' feature.

I think the links A level grades to GCSE grades should be compulsory reading for anyone trying to push their grade 4/5 kids into doing A levels.

To be clear that table of GCSE to A-level grade mapping is for maths A-level only!

But the table of A*/A grades does suggest that pupils with lower GCSE grades may be accepted onto other courses but won't necessarily be looking at top grades.

I agree that the distribution for English Language grades is strange.
The number of grade 3s is much higher than I would have expected.
Perhaps it's related to the fact that Maths has Higher and Foundation tier papers, so there is a way to reach a grade 4/5 on both tiers.
English Language on the other hand no longer has tiers, so I wonder if this is affected students who are average / below average more.

Do you get on well with your Data Manager, @noblegiraffe ? 😁

OCR tends to be used by a lot of independent schools. I think there's a bit of history there from it being the Cambridge board, so was seen as the exam to guide knowledge and ability towards successful Cambridge entries.

I also think there's an element of Psychology in the English Grades - examiners know that a five is what kids really need rather than a four, so they look really hard for anything they can give an 'ah, benefit of the doubt on that' mark to, which means they're more likely to creep over the boundary if they're borderline, compared to a low one, where it's clear it's a three - so it's a behavioural/social artefact on a subject that does not have cut and dry Correct/Incorrect answers.

I wonder if it is because Maths marks are cut and dried. An answer is right or wrong with very little examiner's discretion or wriggle room. English is more subjective. What one examiner might percieve to be a high grade answer, another examiner may not see that in the answer at all.

Controversial, but I think this makes it easier for a Maths teacher to teach to the paper. They can ensure that all of the required Maths skills are taught. In English, and in several other Humanities subject this is more difficult. Yes, there are mark schemes, but there is still the human examiner and their opinion and perception which cannot be predicted.

The use of 'seeds' should pick up on this, but they still only sample a % of the papers marked. I have seen experienced Humanities teachers give a huge range of marks to the same piece of work many times.

I also think there's an element of Psychology in the English Grades - examiners know that a five is what kids really need rather than a four, so they look really hard for anything they can give an 'ah, benefit of the doubt on that' mark to, which means they're more likely to creep over the boundary if they're borderline, compared to a low one, where it's clear it's a three - so it's a behavioural/social artefact on a subject that does not have cut and dry Correct/Incorrect answers.

Except we don't mark by grades...

It's just a figure of speech. I'm not assuming everybody here is fully conversant with the marking schemes and modalities of different subjects.

https://filestore.aqa.org.uk/sample-papers-and-mark-schemes/2018/june/AQA-87001-W-MS-JUN18.PDF

My first thought re OCR was - is that the board preferred by selective independents? Neverdrop's comment seems to reinforce that. That would explain the higher % in top grades.

The English distribution looked like that in 2018 and 2019, so it's not a cohort issue...

As each paper is marked by several different people that 'ah, benefit of the doubt' would have to keep happening on each section, and the marker would not have developed a 'relationship' with the candidate surely?

No, we don't mark by grades. but most people who mark have a pretty good idea of the mark that will push a grade into the 5 territory. In fact I have been told by team leaders in the past to look again at any boderline students to make sure there are not one or two marks more that could be awarded.

I also think there may be a reluctance from English/ Humanities examiners to award top level marks for answers. In Maths once you have the answer then that's that - marks awarded. In English/ Humanities there is usually something else that could have been said by the candidate. No answer is fully perfect (or rararely!) I know this is wrong and that candidadates should eb rewarded for what they HACE said not penalised for what the HAVEN'T but it happens.

I'm pretty familiar with how English marking works :)

The English distribution looked like that in 2018 and 2019, so it's not a cohort issue...

Yes, in 2018 when they first introduced the 9-1 grades I think the percentage of 4+ was set by the percentage of C+ grades in 2017, and the percentage of 7+ grades was set by the percentage of A+ grades.

I can't remember how the grade boundaries were set between the 4 and the 7 - was it equal steps in marks? It's just really odd that you have more kids coming out with a 5 than a 4, and now that's going to keep happening because of comparable outcomes!

But not everybody on the thread would be 🙂- Noble's not said 'Teachers only' - or 'Data Managers Keep Out'!

I thought it would read out the results from the chart, but no, it plays them as a tune!

Agree that OCR is used a lot by independent schools which Jay explain the difference. It frustrates me that fewer trade 9s are awarded for English. I often think the top bands of the MS are wooly and phrases like 'perceptive' are subjective and expecting a lot from 16 year olds in a limited time with unfamiliar texts or writing tasks.....

Argh apologies for typos, definitely not 9 worthy

Love the listen feature!

Grade boundaries GCSE reformed June 2018 (aqa.org.uk)

Almost - steps seem to have been 10, 11, 11 marks (grades 4-5, 5-6, 6-7). So pretty even.

This is all fantastically interesting and that's my Sunday afternoon of a bank holiday gone.
I have tried searching, but have you any idea if there is a resource that shows Welsh exam board(s) taught in England? DC school did two Eduqas exams, both went worse than expected for him and some of his friends, I'd be curious of the percentages for those too.

When I read the level descriptors for the highest level marks on the longer essay questions I often wonder how any 16 year old is ever going to meet the requirement. Take a look at the philosophy highest level descriptors - I think some published philosophers would struggle!

True... except "benefit of the doubt" really isn't a thing, and I can't remember a time in the last decade of marking exams when I could look at "borderline" students because that's not how it works when you're marking online.