Why weren't teachers/schools more realistic with predicted grades?

[nervousnelly8]

Genuinely looking to understand how the A Level results seem to have gone so wrong. I don't work in education and DS is too young to be impacted, so hoping that those more informed might have some input!

Everywhere I see, people are calling for teacher assessed grades to be used. But if this happened, wouldn't the results be way out of line with history, rendering them useless as a form of comparison with other year groups? I understand that the model that has been used appears flawed when considering individuals, but does aggregate performance not also matter? Why wouldn't schools and teachers have been sensible in their predictions relative to previous cohorts so that their assessment could be used reliably?

Not really an AIBU I suppose, I'm sure IABU for seemingly missing the point completely!

a year in which everyone got slightly higher than usual grades and so all got their uni places.

Not “everyone”. A lot depends on how generous their schools were in submitting CAGs. Students from more cautious schools would appear to have underperformed and so may lose out on places.

There will be kids who have been predicted As and have been getting As all year but then are massively nervous during the exam, feck up and get a C. Teachers know that some kids will do this, but don’t know which ones and so predict them a A. The algorithm doesn’t care about individual personalities and so arbitrarily assigns the C.

If the CAGs had been awarded it would have sent university admissions into meltdown. They all over offer to varying degrees but have to take everyone who gets the results. If the CAGs has stood the top universities would have almost certainly had more students than they could have coped with and it would have caused problems especially with medicine, education etc. with required placements as they wouldn’t have had enough. Smaller universities would have struggled for students and the government acknowledged that this was a big problem so put in a student cap.

The algorithm isn’t fair either especially those with big cohorts and penalised those schools who historically hadn’t performed well or sixth form colleges with their large cohorts, on top of this a small minority of schools weren’t fair and over-predicted meaning that they did have to rely on the algorithm. (That could have easily been remedied by Ofqual sending the whole lot back for those schools and asking them to be re-done or asking schools to provide further evidence - the exam boards had moderators and examiners on their books already - they could have gone through evidence and moderated the results - mock papers, class work, incompleted coursework etc.)

There is now no good solution. If they reinstate CAGs and make universities honour original offers it will send the whole system into meltdown. Cynically I think they will reinstate CAGs but not until most have committed to a university and therefore are unlikely to change to their first choice (which on previous years would be a deferral if grades are later upgraded and there is no space available.)

@manymanymany and others - I really didn't intend to bash teachers as I know it must have been an impossible task. It simply can't be the case though that all schools submitted predicted grades which were roughly in line with what their previous cohorts had achieved. Some must have submitted significantly higher grades, otherwise the algorithm wouldn't have adjusted so many people down to make the "average" comparable?

What I don't really understand having read all of these really helpful replies, is why the CAGs for schools which had submitted "sensible" predictions (e.g. within 5% overall of recent cohorts) didn't have those grades accepted, whilst those that submitted clearly high predictions didn't just have them sent back or adjusted down? It doesn't seem logical to adjust everyone down regardless of how optimistic the predictions were

A terminal exam in itself is hardly very exacting in terms of assessing ability, and in such unprecedented times with so many extenuating circumstances even more so. At university level, extenuating circumstances are very often used to bring up students' grades (I've been extern examiner on. many BA courses), and as such just going with CAGs makes much more sense than any method.

Schools did submit reasonable grades. Again, some children are always going to do worse than everyone expects.

@Omelette9 Teacher predictions are always over-optimistic, although with the best intentions. I don't agree with that. One of mine was a marmite child, teachers either loved him or hated him. One teacher predicted he would get a u in his A level in his subject. University interview went well and at the end son was asked why he had been predicted a U when he had actually taken the exam the year before and already had a B. He got his place anyway but I think I can safely say teachers don't always over predict.

[quote nervousnelly8]@manymanymany and others - I really didn't intend to bash teachers as I know it must have been an impossible task. It simply can't be the case though that all schools submitted predicted grades which were roughly in line with what their previous cohorts had achieved. Some must have submitted significantly higher grades, otherwise the algorithm wouldn't have adjusted so many people down to make the "average" comparable?

What I don't really understand having read all of these really helpful replies, is why the CAGs for schools which had submitted "sensible" predictions (e.g. within 5% overall of recent cohorts) didn't have those grades accepted, whilst those that submitted clearly high predictions didn't just have them sent back or adjusted down? It doesn't seem logical to adjust everyone down regardless of how optimistic the predictions were[/quote]
Fair enough - your question did seem to suggest negativity towards teachers - and then some of the responses were pretty nasty about teachers. I don't think the grades were highly inflated - just, as others have said, it wasn't possible to guess what contingencies might happen during the 'live' event of an exam and so predicted grades will average out at being a bit higher than exam results. Maybe it's fairest though to take on board that as circumstances were unprecedented this year, a professional assessment of student ability should be accepted, especially as teachers had to submit evidence of how those predicted grades were arrived at. In previous years with modular A-levels and coursework factored in, students still went to university based on teacher-given grades. In addition, the predicted grades are used by universities to make offers - so it really seems fairest to let them stand.

@nervousnelly8 @manymanymany and others - I really didn't intend to bash teachers as I know it must have been an impossible task. It simply can't be the case though that all schools submitted predicted grades which were roughly in line with what their previous cohorts had achieved. Some must have submitted significantly higher grades, otherwise the algorithm wouldn't have adjusted so many people down to make the "average" comparable?

Teachers I know say they were really careful and kept their overall predictions in line with previous years as they thought they would just get pulled down if the over inflated predictions. If some schools didn't do this don't you think the fair thing would be to accept predictions for schools that were in line with previous years and to actually have a discussion with the other schools to see if they could justify their predictions. An example I heard was a private school that celebrated their centenary 7 years ago. As part of celebrating that they gave a high number of free places to very able students that year and as a result expected record results this year due to a very able cohort. They had a lot of results moved down. They could have explained the background, schools who just inflated results could have had their results adjusted.

It’s a minefield and I feel for any teacher or student going through this.

My child last year for example for GCSE
English Predicted 5 - her nerves etc the teacher said she’d be lucky to get a 4...on the day she got a 6.
Geography predicted a 9 - was so nervous on the day she got a 6.

If the teacher went with her predicted grades this year she wouldn’t have got into the sixth form due to her English mark :(
I just don’t know how we can reach a fair balance for all :( & it’s so upsetting.

I'm not generally sympathetic to teachers but I think they have been put in an impossible position with exam predictions.

It's a long time since I took my A levels, but I do remember that our year group significantly outperformed previous years - in one of my A level classes, 3 of us got As (the other 2 got C and E), in a subject which in previous years no one had done better than a C. In our mocks no one did better than a B...under the current system, could we have been awarded an A grade - I suspect not, even if our school was prepared to predict it, I feel it would have been reassessed downwards. So Bs if we were lucky, maybe only a C if they looked at the school's past performance. That absolutely would not have been fair and would have resulted in us all probably losing our uni places.

I have every sympathy with the current A level cohort.

Im so sorry blue 💐💐💐💐

@TheMarzipanDildo

There will be kids who have been predicted As and have been getting As all year but then are massively nervous during the exam, feck up and get a C. Teachers know that some kids will do this, but don’t know which ones and so predict them a A. The algorithm doesn’t care about individual personalities and so arbitrarily assigns the C.

No, the algorithm doesn’t arbitrarily assign them a C. The teacher’s own ranking decides which one will be “downgraded”.

Teachers are constantly assessing their pupils' ability, application etc and have a record of the childrens' results and records so are well placed to predict outcomes (certainly better than anyone else under the current circumstances).
They not only had to assess individuals but also give them a position out of the total number of pupils in the class.
They could not place 2 as equal, so in a class where they could predict two pupils would get A* they still had to position them as 1 & 2.
Class figures are then combined within the school, so they would then be positioned out of, for example, 90 rather than 30.
If, in the previous year only one A had been awarded then only one would be awarded this year, meaning the second child assessed as A would be downgraded.
Deeply flawed, and horribly unfair.

I think what happened, having done a little reading, is that all the decent statisticians who could have perhaps avoided this situation were not willing to sign non-disclosure agreements and correctly decided not to touch it with a bargepole for fear of damage to their reputations. Therefore somehow the stats people they used ignored the law of small numbers (most A-level classes are too small for variations to be statistically significant - plus there's a choice of boards).

My CAGs came in to the same mean as the previous 3 years of results, plus about 1% (I designed them that way). I used mocks, classwork and GCSE scores. The top 3 and bottom 3 CAGs were kept while the middle 3 were adjusted down one grade. No-one lost a uni place (except the student who messed up in a different subject, and they'll get in somewhere in clearing I expect). Our overall A-A* went up because we offer a wide range of subjects and will run them even if only 1-2 students want to take them (independent).

I can't speak for all independents but mine was very careful and moderate and there were a couple of subjects with unexpected results. But of course in such a situation if it's 50-50 B or C, you're going to go for B. Multiply that by 100,000s of students and you've got an overall uplift of a grade nationally (which is what 12% is basically).

It is a rubbish situation for sure, although I wonder if it will improve understanding of forecasting and statistics. Cold comfort for some though.

For those who'd like to understand algorithms and their pros and cons a little more, Hannah Fry's book Hello World is excellent.

The teacher’s own ranking decides which one will be “downgraded”.

This is massively disingenuous when its a forced distribution. Laddering people is infamously bad at identifying even relative performance, I can only assume whoever in gov. and Ofqual went for this model is 20 yrs out of date in assessing human performance.

In a normal year a school doing predictions well should get a small overshoot as they predict based on what the student should achieve. Its a very small percentage of centres consistently over predicting grades.

Algorithms reflect the subjectivity of their designers. The designers could have made different choices in favour of students, they chose to bias against them, particularly outliers.

However even at the basest level the algorithm hasn't been working when such a high number of centres have had grades lower than the previous three years instead of aligned to the previous three years.

I can only assume that the over allocation of high grades to small cohorts has been clawed back from larger centres where 97% of grades were automated.

The teacher’s own ranking decides which one will be “downgraded”.

This is massively disingenuous

I’m not saying it was right. I was responding to a poster who said the algorithm would choose one student arbitrarily.

In a normal year a school doing predictions well should get a small overshoot as they predict based on what the student should achieve

We were told explicitly not to predict what they “should achieve”. It’s easy to see why there was such variation between schools.

It would have also made sense to have stress tested the decisions made on big downgrades and re-assessed with contextual information. Hopefully someone is FOI ing the universities to understand how many young people have missed their offers and the demographic profiles of those students compared to those who achieved their grades.

DS year 10 will have his maths GCSE based on mocks, he’s given 25% extra time in exams and not mocks so his grades will be lower - he would’ve had more marks if he’d been given the extra time.

Slightly off topic, but why on earth wasn’t your DS given the appropriate extra time in his mocks?! Even without the COVID-related complications (which nobody saw coming), part of his preparation for the real exams should have been practicing using his extra time effectively.

In answer to the op, the vast majority of teachers gave realistic CAGs. There are bad apples in all professions, so it would be naive to think every single teacher / school acted fairly and therefore some form of standardisation is necessary. But that should (imo) have been done by asking schools to hand over the evidence for the CAGs where they were significantly out of line with recent school performance.

My subject’s CAGs at A level were exactly in line with recent school performance (after moderating down my initially over-optimistic estimations), but a couple of students were still downgraded. Frankly, I’m eager to hand over the evidence I have collected for those students!

We were told explicitly not to predict what they “should achieve”

I know. That is why I said "in a normal year" and that this year the predictions were "ratcheted back" for CAGs.

You seem very fixated on assigning blame to teachers. Focus your frustration on a group of people who built and theoretically tested an algorithm which by design favours private schools with small cohorts and either didn't care or didn't notice because it seemed normal to them.

I've spoken to at least 4 admissions tutors over the past few days describing the best heeled intake ever as the candidates getting the offer grades are overwhelmingly from private schools with classes of 15 or less, whilst their state school candidates are missing out. At least one is going to honour offers by using 2021 entry but for most kids, even if this cock up is resolved, it is simply too late.

OP are you saying that teachers don’t know their own students and that the exam boards know them better? You do realise that each student is an individual and how some random student performed X years ago has no bearing on how this student will perform. Can you also explain why Public schools saw an increase this year and only State schools saw a decrease in predicted grades?

I assume that teachers, when looking at borderline cases, gave them the higher grade. I don't see what else they could have done to be honest. Multiply that out across every borderline case in the whole system, and yes, grades will be higher (because in an exam maybe half of those borderline cases would have had a bad day and got the next grade down.)

Can you also explain why Public schools saw an increase this year and only State schools saw a decrease in predicted grades?

Where are you getting this data from? I work in a private school and we have been given our lowest A level grades in over 5 years, including a fair number of downgrades from CAGs.

I can't find the stats on how many gov funded uni places there are compared to the around 750k pupils that have A-level results today.

It would be interesting to know, because:

• if there are enough places overall, but per subject there are over subscriptions, etc... than there's a way to fix that
• if there are loads more kids than places than uni's could actually ask for additional info this year: maybe end of year results? Or some other things

It would also be meaningful to know how these grades actually affect this cohort. Not anecdotally. I feel for every one of these kids, teachers and parents, it's a horrible situation.
But I would like to see that x% didn't get in because of being downgraded,....etc...
It might as well turn out that - as heartbreaking as it is- it might be a 0.xx% of students who are really being effected. Again, if that's the case then it would be much better to deal with those cases on an individual level.

In general I think CAGs would be the best for this cohort, but I'm not sitting in a velvet chair....