Headline GCSE stats....

[BertrandRussell]

...any other education nerds looking at them? How did your school do? I don't think i realised that, despite everyone agreeing that a 4 was the new C, it would be a 5 for performance measures. Which is a bit shit for schools like ours.

I didn't realize one child could affect the results- I obviously don't understand it as well as I thought I did......

My daughter cost her school 0.01 of their score last year by misreading a single instruction in the literature paper. That only has to happen 100 times for a school to crash from well above average, to well below
This isn't right. To move from well above average to well below average every child in the cohort needs to drop a full grade in every subject (or half the children drop two grades in every subject or equiv), so unless only ten or less children in the year, your dd's slip cost less than 0.01 of total school score. Factor in that most slips don't cost a grade and they have to to be in subjects that count towards prog8 and you will find you'd need an awful lot of slips to make much impact. (Plus all schools will have some children making such errors and they are often offset by lucky guesses by other DC or in other subjects etc).

“Under Progress 8, the half a grade positive progress made by 27 students in a form class can get wiped out by three others who, for many reasons, fail to sit any GCSE exams.”

educationdatalab.org.uk/2017/01/outliers-in-progress-8/

It’s not ‘slips’ that have an impact, it’s kids who completely crash and burn, and these kids are not unusual. The DfE have said that from 2018, outliers having a disproportionate impact will be dealt with (but they are still investigating how). For this year’s set of data the problem remains.

That would fit in with our disastrous results this year. Maths results were pretty universally catastrophic, so progress 8 and everything else dragged down with it.

The average mark for the foundation paper kids (the majority at the school) was 40%. I think the head of maths might be on the way out..........

Ouch! You can tell that the results have been really volatile. I teach in a top-100 P8 school and even though our A*-C went down by about 10% (largely due to Maths) P8 went up. The P8 scores for the top performing schools look much higher this year and the scores for the lower performing schools look much lower. The new system seems to have created greater inequality.
I can see the same happening in my subject (history) with the new specs this year. Some schools seem to have really struggled with delivering the new content in time for the exams. There was a poster on here the other day whose daughter's school were shockingly behind. I expect this summer's results in history will be very volatile with some schools doing much much better than usual and some seeing huge drops in results.

Any minute now, our local paper will publish its annual "Every school in your area, listed from best to worst!" piece of clickbait. That's the title. It makes me so angry.
I hate that too. It feels like some schools that are deemed worst work really hard but have got an uphill struggle because of less parental support and more social issues which will affect progress/results, whereas the ones that are deemed best have had people jumping through hoops to get into them and lots of parental support and tutoring to help with progress/GCSE results.

I looked at the Colchester grammar schools and while the girls school progress 8 was exceptionally high, the boys was rather low which was curious

The EBacc result for the boys' grammar school is showing as 0% this year - last year it was 82%. From the school website it appears that the school are still entering their pupils for IGCSE sciences which won't have been included in the league table stats. Therefore the Progress 8 score for this particular school is quite meaningless.

That would definitely explain it Clavinova, thanks for that. So it may well still be true that a good p8 is easier for a highly selective school, but the old 5+A*-C was even easier for them to score well at, and with p8 lower ability intake schools at least have a better chance of doing well. Outliers are a definite problem which can hopefully be resolved. Poor maths or English results will have a big effect too. Still not sure abolishing tables would be a good thing though - maybe just make them less public? I do think schools need to be accountable somehow.

I think the 'banding' of Progress 8 (so average, above average, well above average etc - as shown by the colours on the tables) would be a sufficient level of accuracy for almost all purposes.

I may be renewing this incorrectly. But I think there are more points towards progress 8 for the highest grades if they are above expected progress. So i think the school gets higher for a student who scored an 8 who was expected to get a 7. Compared to a student who scored a 4 instead of a 3.

This would account for why progess 8 is higher for selective schools.

I'm sure it was the deputy head that said this to me. I have no link nor checked this out.

The problem you are thinking of, user is how the A-G grades were converted to numbers this year compared to last year. Lower grades counted for fewer points than previously, and it was worth 3 times as much to get a grade A student to an A than it was to get a grade F student to an E.
Next year when all GCSEs are numerical, this won't be an issue.

So this year's Progress8 is not comparable to last year's, and won't be comparable with next year's, but after that [well, to be absolutely accurate, from summer 2019, because the last subjects won't be numerical until then, e.g. some languages] it will be OK?

Oh, no, forgotten the KS2 SATs changeover, so then we will have the first cohort who had the new KS2 SATs results, so:

• 2016 progress8 - last all non-numeric, old conversion.
• 2017 progress8 - only mixed, new conversion year.
• 2018 progress8 - current Y11, mostly numerical for the first time
• 2019 progress8 for a year or two: all numerical, progress measured from old Levels at KS2 SATs
• 2021? progress8 - numerical, measured from new-style KS2 SATs results

So non-comparability year on year until at least 2021/2022?

Don't really see the point of comparing year on year anyway. As you rightly said before the 5 broad bands are about as much granularity as you need (you might get away with deciles,but no more than that)

Don't really see the point of comparing year on year anyway.
Sorry?
What ?
Time series data are some of the most powerful sets as they are what show the development and impact of changing situations and policies
(eg the removal of lead from fuel significantly improving city school results)

The point is, if you want to use data to intelligently compare schools - and to see what is working (are the children there making better progress, or not) and what is not - you absolutely need reliable time series.

Just as an example, a school I know of was below average in last year's dataset, and average this year. If I knew that the data was comparable, i could ask reasonably insightful questions about what they had changed, whether they were simply very different cohorts or whether they had adjusted teaching practice to ensure better progress etc. However, as the basis of comparison has changed, I can only say that it is probably because they are a selective school and therefore more of their progress was between A and A* than between E and D - it potentially tells me nothing about the school itself.

I was thinking more that if a school was well below average (around the bottom 10%?) it would be scrutinized and expected to improve (or valid reasons for its underperformance accepted). So yes, you'd want to look at their following years results to ensure they had improved, but as long as they had you should be happy.
I'd be unconcerned about schools average or above, as they are serving their pupils around as well or better than the national average and within that there's almost bound to be movement - I would have thought it's unlikely that the same schools would stay at the top (except for selective schools possibly) as it's unlikely that all the teaching goes right and no problems in the cohort year on year, though I guess in a really well managed school it might. There's possibly a case to monitor below average schools too, but half the schools will always have scores lower than zero, rather fewer with the actual category ' 'below average', so it kind of depends whether you consider the acceptable range to encompass the below average category or not. They tend to have scores between around .25 and half a grade lower than average, so the question is if that's an acceptable spread or not. If it is look at them if not don't.
At that kind of level of granularity exact year on year comparisons aren't especially useful.

The thing is it's a zero sum game. If a bunch of poorly-performing schools massively improve their results, then some other schools' progress 8 scores will suffer even if they perform as well as they did the previous year.

Obviously it would help if the calculation method was the same each year, however, if you're comparing schools you can still do that by looking at the 'graph lines' for each school to see if/how they deviate from each other.

For those saying the system is flawed:
How would you improve it.
Would you get rid of P8, if so how would you compare schools.
Would you also get get of Attainment8.

For me, comparing schools should be done at a prior attainment and gender level.
E.g. if you had a very bright daughter I'd want to be comparing at that level.

stubiff
The biggest flaw in the system was Michael Gove
he "believed" he knew the answers and ignored evidence.
Therefore he moved the goalposts (once the badgers had put them down) every single year
so kids did not know if they were coming or going, nor did teachers.

evidence based policy making would be a massive leap in the right direction
as would giving policy changes time to bed in and have a measurable impact

but that would leave a lot less jobs for politicians and their cronies
so it will not happen

stubiff,

the thing is, that only works if you are comparing similar schools - so comparing two non-selective schools with children with similar prior attainment.

What the latest change has done has made it impossible to compare e.g.a selective school with a non-slective school, because progress of a high attainer from A to A* is given much higher weight [despite possible being easier to achieve] than getting a lower attainer from an E to a D.

I will not suggest that this is possibly because last year's incarnation of Progress8 didn't make selective schools look good enough for a Tory government, showing that many didn't ensure that their children made as much progress as supposedly much 'poorer' non-selective schools.

What happened to the school's P8? My Dc's school has a lower %5+ than many other schools in the county, but a P8 >0.5, putting them "well above average" and indeed above schools who have a higher Att8 and historically higher A*-C.

What was their Progress8 last year? Is it that they have historically had a low attaining cohort on entrance, but has got better than expected results from them...so good progress, but from a low base?

How would you improve it

There's one interesting suggestion in this blog that suggests a graph of deciles of attainment at KS2, and deciles of attainment in Y11, compared to the national average. So for example you can see from this graph that the school had a lot of kids in the lowest deciles on intake, but fewer in the lower deciles in Y11 and more in the higher deciles, meaning that value has been added.

teacherhead.com/2015/05/02/progress-8-looks-like-data-garbage-to-me/

Unfortunately it does rather assume that KS2 data is valid, which we know it isn't, because the assessments are crap (and they only take reading and maths anyway), the marking is crap, and because they are used to judge primary schools, totally inconsistent between schools.

Then we have the crap marking at GCSE....