School reports which give your kids GCSE grades or 'flightpaths' are bollocks

[noblegiraffe]

I've posted quite a lot about how reports which say 'your child is currently working at a 4+ in maths' or 'your child in Y7 is making good progress towards their GCSE target grade' are nonsense.

No one knows how your kid would perform in the new GCSEs because no one has had any results for them yet. This is especially nonsense for the subjects other than maths and English where students haven't even sat their mocks. Students from Y7 up have had GCSE target grades generated for them through a computer program that doesn't have any information about how anyone has actually performed on those GCSEs. These targets will be bobbins, and will be revised the instant any results roll in and so will change year on year as each new Y11 cohort sits the exams. If you've been told these targets by the school, the school is stupid to do so.

Teachers are forced to make up 'working at' numbers, and give a 3+ or a 4- to give an impression of accuracy.

If your kid is in top set Y7 then that's a good sign. If they're in bottom set that's a bad sign for a GCSE pass. But there's a long way to go and a lot can happen. That's as accurate as you're going to get.

There's an interesting twitter thread here where experienced headteachers are discussing exactly how nonsensical these reported grades are.

twitter.com/teacherhead/status/885923507858792457

Comments like 'GCSE grades extrapolated back to KS3 despite the fact we don't even know what they mean for year 11. Utter madness- completely meaningless'

And 'It's nonsense, but the shambles we find ourselves in nationally doesn't help.'

• just confirm that many teachers feel the same way about this illusion of science.

Rafa I'd like to know the answer to your question as of course it is Ofsted fear driving SLT to come up wit these crazy flight path systems.
It's such a shame it is almost impossible to do something about this on a national level.

Personally I don't give a shiny shit if the marks are nonsensical. The report has finally given ds a good kick up the bum and proven that he actually needs to work. his 75-85% on his end of years were not enough for him to achieve his target in some subjects. Smug git has not only coasted his way through school so far but been less than sympathetic to those who have actually put in some work. Most of those who have had to work to achieve in the past have all achieved their target for the year, so it was a real eye opener for ds that getting above average and 'good' marks was not enough when you are capable of so much more.

Progress isn't linear, it's messy, and it happens in the brain of a child so can't always be observed.

This all over.

We have a list of statements that explain what we expect for y7,8,9 for top/mid/lower entry. Then when we report to parents it's a holistic judgement of whether we think they are at, below or above expectations. It's not a perfect science (far from it), but most teachers can tell you whether each child is their class is where they should be or not.

Equally, I'm honest with students and parents about lack of clarity and work on the grounds of 'I will challenge them as much as I can and if I have any concerns I'll call you'. Seems to work so far.

I'm with EinsteinsMum report came home yesterday, bollocking to DTB for being lazy. I'm still at the point of hoping that they will pull their socks up in next academic year How old is your son EinsteinsMum? My twin boys are 13 and will be going into Y9 in September. Hoping a bit of maturity may kick in

Now you mention Tom Sherrington, I've remembered John Tomsett and Alex Quigley are using a similar system to the one Maisy describes at KS 3.

There may be hope vloopy. If they're taking the same line they haven't been at all impressed by primaries who have lots of data but it's meaningless or has just reinvented levels.

I'm currently on mat leave and from looking at emails it seems there are seeds of doubt being sown about the flight paths. Fingers crossed

It's not a perfect science (far from it), but most teachers can tell you whether each child is their class is where they should be or not

Yes, of course they can, and in a "my teacher is always right" culture that is fine. But, we all know that:

• there are good teachers and bad teachers;
• even the good teachers sometimes see children differently;
• parents often see their children differently to teachers.

So, any "judgement" ideally needs to be backed up by some sort of evidence framework, just as it is in any other walk of life (don't get me started on the competency frameworks used in most workplaces to measure performance - they're controversial too, but I understand why they're needed).

The problem we have is that the Government has thrown away it's national framework and created a sort of free market in alternative frameworks.

There is going to be a long period of uncertainty when schools/trusts come up with their own methods, but eventually some will be proven to be better than others and will be widely adopted.

In the meantime I think it's wrong of Noble to dismiss evolving flight paths out of hand. Yes, some might be bollocks, and the ones used in Noble's school might be particularly bollocks, but I can see the logic that is going into our school's flightpaths, and know that the teachers creating them are experienced, competent and justifiably cautious. The paths will be refined whenever more data becomes available, and in the meantime appropriate communication of the caveats to parents is taking place.

So, let's all be grown up about this - if we think a particular system of flightpaths is blocks then let's see the disputed methodology and evidence, but don't dismiss other flightpaths until you've looked at their methodology and evidence too.

But the methodology is irrelevant at the moment as there is NO data for the new GCSEs we are totally in the dark in English and Maths.

From a logical perspective doesn't it make more sense to assume a flight path is likely to be inaccurate until there is evidence to suggest otherwise? It's an experiment with mainly variable factors, and as of yet nobody even really knows the full extent of some of those variables.

Even something as simple as the L6 leading to an a/a*, was just a best guess, so surely with a new curriculum and exam system any flight path is even more unpredictable? Dd's Y8 has loads of dc who started either with l6 or were of that standard, and nobody is willing to plot the entire cohort to 7/8/9. So how on earth do you do it for the full ability range, in a state school that will generally have even more variables funding, dc's background, Sen, Eal etc?

there is NO data for the new GCSEs we are totally in the dark in English and Maths

There is a lot of data for the old GCSEs. There are also a small number of reference points for mapping between the old and the new - not nearly enough to be definitive, but it's a starting point. A lot of work has been done by our MAT to fill in the huge gaps, but it's fully acknowledged to be a work-in-progress.

There are also a small number of sample papers, which can be given to students whose position on the old framework was known - these give additional reference points for the mapping. In the case of our MAT they are using consultancy from experienced assessors to verify teachers' assessments of sample papers.

When the summer GCSE results come in there will be more data, and the flightpaths will be refined in the light of that. It is a continuous improvement process.

Yes the teachers are rueful/cynical about it to various degrees, but they are all pulling together to make the best of a bad situation. They are openly including parents in the conversation, so I'd be surprised if they felt the need to do a Noble and label all their hard work as "bollocks" on Mumsnet.

but I can see the logic that is going into our school's flightpaths

How will teachers be providing accurate data about whether a child in Y9 is a particular grade and thus on their flight path or not?

They can't. Internal school assessments can't provide that sort of data. The whole thing is a bust because it starts with the totally flawed assumption that this is possible.

Look at GCSEs. These are written by experienced exam-setters and yet grade boundaries are set after the national results are in, by looking at KS2 results and using comparable outcomes. Then Ofqual might disagree about grade boundaries because it doesn't fit with the other exam boards.

Compare that to a school knocking up an end of year test, a teacher running their eyes down the scores and writing 4-, 6+, 3 onto reports. How much science do you think goes into that? How accurate do you think those assessments will be as a national comparison? Even when we were using a system that was well known (levels), these assessments were inaccurate. With new GCSE grades we know nothing about, it's rolling a dice.

And so the concept is bollocks (re linear progress) and the data is nonsense.

I pretty much deal with the difference between 7, 8 and 9. With the odd 6. It's not a question of 'filling in the gaps' the top end has pretty much been all gap. There has been such little support on what a 9 grade candidate will look like that our HoD has refused to predict any. If you're trying to distinguish between a 3 and 7 then obviously it's not easier.

There is a lot of data for the old GCSEs. There are also a small number of reference points for mapping between the old and the new - not nearly enough to be definitive, but it's a starting point

I do hope you're not suggesting that they tried to construct grade boundaries for the new GCSEs. That way madness lies.

There are also a small number of sample papers, which can be given to students whose position on the old framework was known

If that means that they sat the old GCSE (and that's the only way to really know), then giving them new GCSE papers for a course which they haven't been taught and expecting the results to be meaningful seems odd.

As for experienced assessors verifying teachers assessments - no one knows what the new GCSE grades will look like till the exams have been sat nationally, so how is that helpful?

How much money is being sunk into this?

Yes the teachers are rueful/cynical about it to various degrees,

I bet they are. They're going to be the ones making up the data and trying to justify it to parents once the expensive experts are long gone with the cash.

At the moment it's like knitting fog.

We have collaborated with other schools to try and come up with our best estimate of wherevthings will be....butuntil this first cohort goes through the new spec GCSE exams next year, it's a bit like sticking a damp finger in the wind and accurately predicting the weather.

Mr Gove has a lot of stress and unnecessary work to answer for.

generallypreoccupied
I hear you, I really do.
I get there's good and bad teachers.

I'd like to think most teachers (even in the sheer lack of information erasmus we have now) can say whether a child is working well or not.

It's easy for us to complain that levels have gone, but they weren't really any better in terms of clinical objective judgement.

Assessment is complex. Parents need to be aware of this and so do teachers. Meanwhile, we make the best of the situation we're in and the information we're given.

Noble your biggest issue appears to be with predicting precise grades and how kids are progressing against them, or even that some schools are saying currently a 4- or a 6+, is that correct?

If so, do you also think that my view that my y7 DD is likely to get grades 3-6 is also not actually predictable, or if these 'flight paths' in general were wider to encompass 4 grades, as in my prediction, would that be more reasonable to you?

If the flight paths were wider, then parents would probably complain that they were no use at all. A 3 is a fail and a 6 is a potential A-level candidate so that's a pretty wide band!

But it's the sort of thing I do at parents' evening. I'd say to parents of Y7s 'if your child continues to work at this level then they'd be headed for the top grades (e.g. 7-9) at GCSE. Or 'they're currently struggling and if they continue at this rate they're unlikely to pass their GCSE'.

Back when we had KS3 SATs I'd talk about the stats. 'A quarter of students who got a level 5 went on to get a C at GCSE, so it's possible but she is going to need to work hard and get as much support as she can'. Or 'more than half of pupils who got a 6 went on to get a C so it's looking positive but she obviously needs to continue to work hard and ask for help'.

It's not a nice line and an arrow, it's a band of probabilities. The 'expected' value when we get the data in will only be the most likely out of all outcomes, but the probability of that expected value could still be pretty low.

Oh and when I'm talking about expected values, I'm only referring to nationally benchmarked data, which in the current situation is KS2 SATs and GCSEs only. Anything in between is bobbins.

Yes, basically the issue is applying stats that work nicely at a population level to an individual level. It just isn't accurate or particularly helpful.

I don't know if you've recently had a Year 7 parents evening noble, or just done school reports, but I don't half feel for you.

Had my sons first parents evening at high school last week and the amount of parents walking round taking notes and pushing the teachers for predicted GCSE grades in every subject was ridiculous.

I felt a little bit of a crap Mum to be honest, just asking if he was happy and whether there were any issues I could help with. Then you get all the Facebook posts from these parents gloating about how their darling children are going to get straight A equivalent at GCSE.... think they're in for a rude awakening in 4 years time.

The only teacher,funnily enough, who even mentioned GCSE to me was Maths, but that's because they made them do old GCSE papers for tests ...

Parent's are bonkers, absolutely bonkers

they made them do old GCSE papers for tests

ARGH. It's bonkers. In this drive to be able to show progress to Ofsted inspectors, schools are taking absolutely bonkers decisions.

Amanda Spielman, the new chief of Ofsted is on the case about this sort of thing.

"Ms Spielman told The Sunday Times pressure to succeed under the new regime meant schools were finding it "hard to make sure they put children's interests first and think children, children, children".

"The real substance of education is getting lost in our schools," she added.

An inclination to drill students for exam success could follow, she told the paper, compromising their chance of getting a "broad and balanced education".

She claimed to have seen a class of 11-year-olds being led through the GCSE mark schemes in place of their normal lesson at one school.

Schools were also said to be broadening some courses from two to three years so pupils could be adequately prepared.

This method might ensure they get "cracking grades", she said, but it could leave them short of the skills required to thrive in later life. “That is not something any of us should be happy with," she added. "

www.tes.com/news/school-news/breaking-news/gcse-reforms-risk-losing-real-substance-education-ofsted-boss-says

Look at GCSEs. These are written by experienced exam-setters and yet grade boundaries are set after the national results are in, by looking at KS2 results and using comparable outcomes.

Yes, and the people who do all that assessment and analysis at a national level aren't DfE employees sworn to silence - they're senior teachers and educational consultants, who are already talking amongst themselves about how things may or may not pan out. It is that level of experience that is being used on a consultancy basis by some MATs and schools to start mapping out flightpaths. They are cautiously taking a lead, where others will eventually follow.

Compare that to a school knocking up an end of year test, a teacher running their eyes down the scores and writing 4-, 6+, 3 onto reports

Our school uses extracts from past (old style) GCSE papers and sample (new style) GCSE papers in assessments from KS3 onwards. In the latter case they obviously only use extracts that are relevant to what students have been taught.

The transition period is the only opportunity to do that mapping - no point in waiting till all the kids on the "old" system have left.

It's not in every subject though - just the core subjects at the moment.

pressure to succeed under the new regime meant schools were finding it "hard to make sure they put children's interests first and think children, children, children

They're under pressure to help the children to succeed under the new regime. If the children succeed then the school does too, but I'm not in any doubt that the children's interest come first at our school.