Ofsted announce school report grades are bollocks and to be ignored

[noblegiraffe]

Confirming what I’ve been banging on about for ages, Ofsted have announced that school internal tracking data - the sort of ‘working at’ grades that appear on reports to parents - will be ignored in school inspections because it’s made-up nonsense.

“Too often a vast amount of teachers' time is absorbed into recording, collecting and analysing excessive progress and attainment data within schools. And that diverts their time away from what they came into the profession to do. which is be educators.

“And, in fact, with much of that internal progress and attainment data, they and we can’t be sure that it is valid and reliable information.”

www.tes.com/news/ofsted-inspections-wont-examine-internal-school-data

Maybe, just maybe, if Ofsted are no longer interested in seeing it, teachers won’t have to make it up any more?

I can, for example, set a routine spelling test on a Friday which a child will get 20/20 on, having prepared for it as homework.

On the following Monday, they will get around 15 right.

By the following Friday, they might spell the new 20 words correctly, but only 10 of the previous week's words.

In independent writing work, they may consistently spell only 4 of those words correctly.

are they, or are they not, keeping up with the curriculum? Which is the 'valid' test result? 20? 15? 10? 4?

Yes, that's the problem cantkeepaway. The national curriculum has expectations for the end of key stages, but there are up to four years in KS2 for example. Given that children don't progress in a linear fashion, even expectations for the end of each year could identify children who are 'behind' (when actually they've just plateaued) and children who are 'on track' when they had coincidently covered the areas in a test just the week before and forget it all by the next day. And just because a child doesn't quite make the 'expected standard' in their Y6 SATS doesn't mean that they're not academically capable, for example.

But I don't know what the answer is.

Who says what 'the expected score for someone on top of the curriculum' should be? Heads of department should be able to work this out. It would depend on the difficulty of the test, the timing of it within the year.

Tests should be cumulative and so although in a weekly test you would anticipate many 20/20 marks, the true curriculum test would be in maybe a month’s time. If the class is set the same parameters and they are tested on just a sample of the curriculum then it’s fair. For external exams they are told when the exam will take place so to replicate that, Id give them good notice. But for usual mini quizzes then it’s important to give them those regularly as long as they are low-stakes. Keeps the knowledge learning and retrieval ticking along.

Heads of department should be able to work this out.

Er yeah, just look at how well that went when trying to predict grade boundaries for the new GCSEs.

There isn’t even an expected standard at GCSE. Grade 4 doesn’t mean that a student has met a particular standard, it means that their exam score put them in a particular percentage of the cohort.

No they aren’t being asked to predict anything. Just to suggest what they would expect a student to score on a test that they set from the curriculum that they have taught. I’m not talking about a grade either.

Cauliflower,

Can you be absolutely clear about the difference between 'predict' and 'suggest what they expect'?

To me, those mean the same?

More critically, the lack of standardisation in the method you suggest is a fatal flaw.

So department A sets tests on the curriculum they have taught in which they think the students in their school should score 25/30. and they do, reliably, week after week. Brilliant.

Except that, come the end of year 11, all the students get 3s. Which is no perhaps what a parent might be led to expect from an 'internal tracking' score of 25/30, week after week.

Meanwhile, in school B, the department interprets the curriculum in a different way. They set different tests, and expect their pupils to get, say, 10/20. Parents are unhappy with this, as this seems so low compared with 25/30, but they go along with it. The pupils scoring 10/20 weekly get 7s at the end of GCSE courses.

How has this helped anyone?

Noble,

So do you genuinely believe that all teachers, and all parents, should remain absolutely in the dark in terms of how well a pupil is doing, until the 'big reveal', probably first after mocks and then after real GCSEs in year 11?

Or should teachers have internal tracking that they don't share?

Or should everyone go on 'gut feel' until the end of Y11?

Or what?

I will answer your questions but you are coming across a bit aggressive - I hope that’s just the style?

“Predict” is suggesting what you think a student will end up with, before they have actually done the test. It has previously been done with GCSE results - not usually very accurately.

With this system you set the test (cumulative) and consider when you’ve set it what percentage (for example) a student would need to achieve if they knew the curriculum you had taught them well. You take in the tests and have a look at them. Ranking them works well (another Daisy Christodolou) and then you again look at the percentage (or mark or whatever, I’ll say percentage for ease) that you think that they should have achieved in order to have understood and retained (and retrieved) what has been taught. It would be highly unlikely that every student had retrieved everything well, especially on a test which is only ever going to test on a small fraction of what has been taught.

Although ultimately they are doing GCSEs success isn’t pinned to those grades.

If a student is achieving 25/30 every week (and as I said previously this would not be the kind of test which you’d do every week because of the time it takes to set properly, moderate internally and judge effectively) and they ended up with a 3 then the school curriculum is not set correctly. That’s not a problem with testing it’s a problem with the actual teaching and curriculum.

The mark expected would change according to the difficulty of the test set in any case. Not a blanket 25/30 for example. And actually a huge amount of work needs to be done in following up after the test - students looking at where they missed points, redrafting or practising skills they didn’t demonstrate in the test. They need to be really clear why they got parts incorrect.

I think when you’re looking at it, you are wondering how it helps in pinning students against their possible end gcse results. But if the departments set exciting and deep curriculums and judge students against those, then the work they do in teaching them and assessing them etc will lead students to gain the knowledge and skills they need to do well in the final public exam.

I hope that makes sense and answers your queries.

I think we have moved away slightly in our discussions from the teacher's workload implications of Ofsted's announcement... that SLTs might be more thoughtful now and stop, or at least lessen, the endless and unrelenting demands for data drops (which has utterly changed the volume and nature of assessments and teaching) and 'accurate predictions' because 'Ofsted demands it'. I'm sceptical. but we'll see.

I agree Piggy. It’s so meaningless when it’s frequent. 3 times a year max in my opinion.

It’s impossible to set tests of a consistent difficulty and it’s hard to tell how student will do on a test until they actually sit it - that’s why grade boundaries aren’t set until the results are in. Even professional exam-setters can’t do it.

When you say ‘what you expect a student to get’ are you talking about the best student in your group, or the least able? Give an expected percentage and you are damning one with low expectations and condemning the other to failure.

Because we set in maths, we can use that to give parents a ballpark of where they are headed. We can give students the average score for a group on a test and they will know roughly where they are placed in the group for that test.
I imagine it’s much harder for mixed ability non-tiered subjects.

When I say “what I’d expect a student to get” I would mean a score which indicates they know the curriculum well. It’s a mark that is as objective as possible by looking at how the students have done and working out I suppose the lowest mark acceptable that still indicates that they have still understood the curriculum well. It’s not about saying to a particular student or group of students that THEIR expected mark is 10/20, but to another group that THEIR expected mark is 18/20. It’s more objectively saying, “on this particular test of the sample of curriculum taught to you over the last X months, you scored 20/30 and our department looked at all the assessments and felt that in order to show good understanding of what has been taught we would expect you to have scored 18/30. Now let’s have a look at which areas you fell down on and which bits were successful and why”

An average score is another way but it would just show you their rank within the group rather than pinning their result to how well they knew the curriculum. If the majority hadn’t worked very hard, they could still think they’d done well if they had come, say, 4th in the class. Or in an exceptionally hardworking class a student might feel dispirited coming 20th out of 30 when the teacher might feel that only the bottom 5 hadn’t reached a good enough standard for that particular test. So student number 20 had in fact done well, despite his apparently low ranking.

It forces them to compare with each other rather than the standard set. If you see what I mean.

Now let’s have a look at which areas you fell down on and which bits were successful and why”

This is a valuable approach to any test feedback. Putting an arbitrary pass mark on the test seems odd, especially when you’re essentially plucking that pass mark out of the air. And what does that pass mark even mean in the long term?

Why would it be arbitrary? You’re deciding on a mark that you and your department feel, from knowing the level of difficulty of the curriculum you’ve taught and from looking at all the assessments completed across a year group. You’re indicating as a team of professionals the mark above which you feel shows the curriculum has been understood.

Plucking a figure out the air is not happening in this situation - I mustn’t have explained myself very clearly.

What does the pass mark mean? That the student has demonstrated for that test that the curriculum has been well understood. Or not. As the case may be. Long term? Well if every assessment that the student did was well above the mark that the heads of departments decided indicated the curriculum was well understood then it would indicate that the student was doing well. What else could it mean?

What everyone wants is a system whereby you know the GCSE grade you’re going to get. And that can never happen - all you can do is measure against the curriculum you are teaching.

student was doing well.

And where do you set the level of ‘doing well’? Grade 4? 7? 9?

In my school we have two year halves and two top sets, two set twos and so on. Set one will be getting grade 6-9, set 2 grade 4-7 in the end. On a tricky algebra topic test the average mark for set one could be 70% and for set two 30%. Where do you set the pass mark? 50%? Oh well done set one, you have all passed. Oh dear set two, you have all failed. You can’t set it at 30%, clearly they have not ‘understood the curriculum’, but we would not expect them to do much better as it’s tricky algebra and they are not headed for a grade 7-9.

Do we make the test easier so we can have a more reasonable-sounding pass mark? But that won’t challenge the top set....

Set two are doing well in that they are headed for pass+ at GCSE but you wouldn’t think it if you suddenly started slapping pass marks on tests, because clearly they don’t understand that much of the curriculum - grade 4 at GCSE was 21% on higher. Unless you want the pass mark for tests to be 21%?

grade 4 at GCSE was 21% on higher. Unless you want the pass mark for tests to be 21%?

If a particular test was hard enough then 21% might be appropriate as a pass mark.

Like any test, you wouldn’t set the pass mark randomly.

With sets you put pass marks at different levels according to where you are aiming. Or set different tests altogether if you are teaching the curriculum in a different way.

If you are setting different pass marks for different sets for the same test then it isn’t a pass mark is it?

It’s basically levelling tests. Looking at a test, trying to decide how difficult it is (this takes years of experience and is something I’m better at than my HOD as I’ve been teaching way longer), and grading it.

It’s bollocks. They can’t even do it accurately for GCSEs and they are professional exam-setters

I don’t think you’ve understood what I’ve been explaining. Not grading and not levelling, looking at how well the curriculum is known according to where you’re aiming for. That’s it. I’d say giving a class average is pretty rubbish for the reasons I gave, but that’s up to your school. None of it is an easy science. How do you think it’s best to assess a curriculum?

And what I’m trying to point out is that question is unanswerable. When we do tests kids will ask ‘what was the pass mark?’ Or ‘what grade will this mark get me?’ And I tell them ‘there is no pass mark and it doesn’t work like that’. Because it doesn’t. I’ve been teaching over a decade, know my stuff and I can’t answer those questions which means it is wrong to ask them. It is not the case that I’ve simply not thought hard enough or don’t know the curriculum well enough, it really doesn’t work like that.

You want the pass mark to be set to show that the child has understood the curriculum and is doing well. There’s no planet where ‘21%’ would be seen as an acceptable answer week after week and yet that’s what it is in the end. In the meantime they’ll have hard tests on algebra, easier tests on ratio, tests on geometry that are hard or easy depending on how good you are at visualisation. Fail. Pass. Fail. Is reducing the scores to a binary actually useful if the kid is going to get a 4?

I think it’s useful, yes.
And classes I teach feel it is, and so do parents. I guess that’s what counts in the end. When students get a test result they should have some understanding of how well they did, beyond a comparison with their peers in my opinion.

And the kid who gets fail fail fail fail fail fail, how do they feel? How useful is it for the kid who will always pass?

I wouldn’t use the word pass or fail because it’s not passing or failing. It’s reaching or not reaching (yet) the standard set.

You deal with the student(s) not reaching the standard set like you would in your normal day to day teaching. First putting in the background work of having mistakes as interesting points of development and areas to practise and improve on. Everyone can get better with practice so modelling this and expecting it, rewarding it etc. Then analysing where students are going wrong and creating exercises and re-explaining it so they get it.

A student getting 10/75 in your class knows they haven’t done well even if there is no expected standard and you say it doesn’t mean anything. In my class I might say that the standard I was expecting was to get over 25 on this particular test. But in the next test out of 75 which was much easier I might say that the expected standard was to reach 62. Then the student has something to pin their mark against.

A student not doing well would know how far off they were, rather than just thinking they hadn’t done well. A student getting 62/75 might think they had done amazingly unless the test was contextualised for them and they knew that in fact it was much easier than the previous test.

I expect all pupils to be able to get 100% on the tests I've set - I've taught them 100% of the material. They never do though. Have they all failed??

I ask pupils not to share scores while in my lesson. What is brilliant for one kid is awful for another. Someone always ends of demoralised (needlessly). It's because all tests except gcse/ a level are formative NOT summative.

Additionally, I've seen two major and multiple minor curriculum changes to n 7 years of teaching at nearly every level. It doesn't stay static long enough for me to make the sort of judgements you're asking for. Different schools teach different schemes/ exam boards and cover material in a different way. Every time I change schools it's almost like I'm in my first year of teaching again.

Honestly, I get what you're saying, but it just won't work practically speaking. Some cohorts do better than others. Expectations for everyone have to be high (it's in the teaching standards that we all follow). It just sounds like you don't know how the system works to understand your suggestion isn't compatible with accepted practice and how it actually works...

cauliflower I still don't get what happens when students don't reach the 'standard set' : say, a low ability group...do you just set a different standard? Do you really have this much autonomy at your school cos I sure don't!

Sorry, are you talking to me here physicskate ?