Ofsted announce school report grades are bollocks and to be ignored

[noblegiraffe]

Confirming what I’ve been banging on about for ages, Ofsted have announced that school internal tracking data - the sort of ‘working at’ grades that appear on reports to parents - will be ignored in school inspections because it’s made-up nonsense.

“Too often a vast amount of teachers' time is absorbed into recording, collecting and analysing excessive progress and attainment data within schools. And that diverts their time away from what they came into the profession to do. which is be educators.

“And, in fact, with much of that internal progress and attainment data, they and we can’t be sure that it is valid and reliable information.”

www.tes.com/news/ofsted-inspections-wont-examine-internal-school-data

Maybe, just maybe, if Ofsted are no longer interested in seeing it, teachers won’t have to make it up any more?

@noblegiraffe

Im not selling anything. Im sharing the fact from a parental perspective it has been a useful exercise for two of mine. Whether
Or not you agree, that has been our experience.

What i have also said is that in my experience, present attainment can be measured and doing that repeatedly over time assuming the teacher chooses a reasonable method gives an absoltely essential overview of progression. The GCSE paper is just one option but there are many others, dont like that one pick another that suits you.

But my main concern is without any significant and meaningful data (however that is garnered) many children with SEN will have a significantly harder job getting support than they currently do.

Its easy to focus on one bit and miss the bigger picture.

noble is absolutely correct. You cannot measure progress. You can only show what a student is or is not able to do at fixed points.

Green - you should read Daisy Christodolou for more information on this.

And yet Ofsted say that internal data is so unreliable they won’t bother looking at it.

If all these methods are out there and just super and you can take your pick, why does Becky Allen (who knows quite a bit about data) think they don’t exist?

I think were arguing semantics.

If you show where a child is at a spot in time, then later you test and they can do more, then you can see progress.

If a child can complete 50% of an ed psych test one year and 80% two years later, its not unreasonable to say they've made 30% progress on that test. If is acceptable in a tribunal court im damn sure it has value

Problem isnt data is what you choose to measure and how. I can see why Ofsted say internal data is unrealiable. (The maths teacher who uses my maths results from adding HTU homework for example). But thats why a variety of measures is useful

Thanks for clarifying Ta1kinpeace. I'm about to appeal a refusal to assess my DS3. Part of the problem is school have recorded him as being more able than he is. I understand how and why that's happened, but it does create a problem for me.

I have to confess that as a parent, the most useful data I have received over the years is the setting. Because every half year or so they sit all the year in some sort of setting test (or tests, sometimes it is two or three tests during the half year, although it is usually mostly if not all on the final exam for the end of the year), and then from that they take the top 28 or so for set 1, the next 28 or so for set 2 etc. (they are supposed to only move DC up or down one set at a time, but I definitely know DC who have dropped more than one set). That means I know how my DC are performing relative to cohort by which set they get put in each half year. It has always been far more useful than these weird targets (set by some external agency, and which make absolutely no sense for art). It usually does roughly corollate with the grades given, but the grades given are usually only intelligable by reference to the sets (so if I didn't have the sets, I probably wouldn't understand them).

It does make things very stressful for the DC, given the way they can yo-yo between sets (a bad day means a dropped set, but then next half year, they will usually go up again), but it has always been the most reliable way of getting a feel for how my DC are doing (except in subjects in which they are not set, where I end up feeling much more clueless)

Given that maths is being used as the example - DD (Year 8) had a setting test mid November on the first two chapters of Mymaths (mostly 2c/3c) and will have another in mid January on chapters 3 and 4. The results for those two tests will be averaged and will be the score that determines which set she is put in after February half term, with the sets next going to change at the start of Year 9 based in the end of year exam in Year 8. That does not mean that the individual teacher cannot set tests for individual classes more frequently (eg at the end of one chapter), but that is a test she designs herself, and it does not count towards setting. The setting test is sat by everybody in the year (although there may be exceptions for some in the bottom set where the teachers judge it may be too demoralising).

Getting a top mark on an end of topic test doesn't mean they've made any progress on the previous topic. So just because their 'working at' grade is high, it's not reflection of the past or the future.

Science is taught in topics in the vast majority of schools. They revisit topics and build on them from previous years, so progress can only be viewed from that perspective. And no one is comparing how a child did with the same topic in year 7 compared to year 9/10.

Progress is also NOT linear!! There are so many factors affecting a child's ability and capacity to learn at any given time (issues at home, with friends, extracurricular, hormones etc...) that 'data is king' just perpetuates the myth that progress is linear when it's anything but!!!

So true, kate : my DS just got a '9' in a physics test (year 10) and a 4 in biology. He's doing combined science, so i think that may be said to reveal where his 'weknesses lie' (biology) , if it weren't for the fact that he got a 4 in a previous physics test and a 6 in biology! Useful for showing what topics he needs to revisit, but certainyl not a useful way to measure progress!

Getting a top mark on an end of topic test doesn't mean they've made any progress on the previous topic

That is true, but if every setting test covers two topics, and there are two setting tests, that are a two setting tests for a set move, then that surely if you get put in set 1 after two setting tests that should show you are doing well relative to cohort, that gives you some real data. If you don't because one test is good and the other isn't, it might be because you had a bad day, or because you didn't understand topics 1 and 2. If the test shows you lost most of your marks in a particular topic, and you get put down, and your best friend doesn't tha suggests you are weak on a particular topic, and she isn't.

Of course topics build on on another, so that certain Year 8 topics build on year 7, for example. But so what. If one does badly on an end of topic test in Year 8, it might be because one never understood the work in Year 7 in the first place, or because one didn't understand the build, but bottom line, what that test shows is that one is not performing at the level expected of Year 8, which is what a parent wants to know.
What a parent wants to know (and therefore hopefully what Ofsted should be testing for, and it is the only thing Ofsted should be testing for) is how is my child doing in terms of what is reasonable to expect at this point.
It is like a journey to the top of a mountain. Sure, the really tough cliff at the top of the mountain might prove impossible for me to climb, no matter how well I have navigated the bottom slopes. But if I am struggling with the bottom slopes, and are not far enough up the path, I won't even get as far as the cliff face at the top. If I get up the bottom slopes well, then I am clearly in reasonable shape to attempt the cliff face.
I don't believe any parent is asking for confirmation that their kid will be able to climb the cliff face of GCSE, based in what is going on in Year 7 or 8. But surely the opposite must be true. If they are not successfully navagating the lower slopes to the point where most kids reach at this age, it will be that much harder (not impossible, but that much harder), to attempt the cliff face when that time comes. And if most kids after having this level of preparation actually then do navigate the cliff face, it is not unreasonable to mark certain points on the journey as adequate or not adequate.

And clearly kids have different strengths and weaknesses. I have one kid who handles algebraic equations with ease, and is most likely to fall down on the geometric type, whereas another who struggles with the abstract nature of algebraic equations, but can see geometric relationships instantly. Very different kinds of mathematicians. Both can do very well, ultimately, but both also need to know which types of questions they will almost certainly lose marks on, and need to work on, even if they both get the same marks in a test (a 9 or 8 or whatever). Because I have some mathematical background I can see this, but if I didn't, I might ideally want the teacher to be able to tell me that, but I suspect it is likely they will not be able to (although topic tests should help). I might also ideally want to have some idea of the ratio of algebraic type questions to geometric type questions in the syllabus. So long as it is kept in roughly the same ratio all the way through, then my kids are less likely to face major changes. If Year 7 and 8 is all geometry and then GCSE is all algebra, sure, what my kids get at Year 7 and 8 is not going to give a good picture of how they will do at GCSE. But if the subjects in Year 7 and 8 are well chosen because they will build to GCSE, then some sort of averaging of topic tests should give me a reasonable picture of how well they are prepared for the GCSE. And surely 100-200 kids is likely to be somewhat statistically significant, in which case how that cohort does on a setting test should give me a reasonable feel for how my DC are positioned (unless the teaching is exceptionally poor, or brilliant, when they will be lower or higher, correspondingly).
Isn't this the sort of data that should be being produced?

What a parent wants to know (and therefore hopefully what Ofsted should be testing for, and it is the only thing Ofsted should be testing for) is how is my child doing in terms of what is reasonable to expect at this point.

This is exactly what Ofsted have just said they won’t be looking at. Because ‘What is reasonable to expect at this point’ is a question that can’t be answered - there are no age-related expectations in secondary school.

This is exactly what Ofsted have just said they won’t be looking at. Because ‘What is reasonable to expect at this point’ is a question that can’t be answered - there are no age-related expectations in secondary school.*

And here you have the mismatch between what parents want to know, and what Ofsted is prepared to give. I do not believe that "What is reasonable to expect at this point’ is a question that can’t be answered", and I am willing to bet that just about every other concerned, involved parent doesn't believe it either.

That is the nub of it.

If you cannot produce a reasonable set of expectations for what my kid can get out of your school, then all I can conclude is that you are a glorified babysitting service. When I sign up to a course , in anything I am interested in, or anything work related, or whatever it is, I have a set of expectations as to what I will know at the end of that course, and what I will be able to do. If I didn't know the first thing about first aid before my first aid course, I expect to have been trained to a certain level from my course, especially if my colleagues at work are then relying on my having gone through the course so as to be able to administer the needed first aid. Clearly the courses need clear goals. It is no point suggesting that with a simple first aid course I will end up trained like a doctor. But I expect to have a certain level of knowledge at the end of it, and one that will equip me for the next level up first aid course. Some people on the course will have mastered absolutely everything, some only to a more limited extent. Some people might have not mastered enough to be useful as a first aider at work, and I would hope that the system would make sure they are not accredited as such, as otherwise lives could be at stake. Ditto with everything else that involves continued learning (even if the consequence of failure are less disastrous). Why is it only schools who are not able to produce any meaningful idea of what can and cannot be reasonably be expected to be learnt in a given year, and how well that has been learnt. And if schools can’t why then does the government force me to force them to go to school every day, when they would rather not, if there is no point, because nobody has the faintest idea of what knowledge or skills or anything schools impart to my children over and above what they would learn if they were left to their own devices at home.

It is incredibly frustrating as a parent of secondary kids - you are treated like mushrroms frankly there is v little feedback from school, I think DD is underperforming as she is to some extent ignored as she is not a squeaky wheel she basically gets on with things. She is in danger of being switched off because of this lack of individual interest. How are we meant to know where things are at before its too late

And not only is what the teachers on here are saying worrying but it goes against my experience, particularly in maths and sciences (which I know something about), and where my kids have had some pretty poor teachers over the years (as well as some brilliant ones, and it is just luck of the draw). There was fair bit of resistance from my son in Year 7, when he ended up with a pretty appalling teacher in maths, to come to me rather than to his mates. And it took a while to persuade him that with an honours degree in theoretical physics and applied maths, I am not just mum, and I just might be at least as equipped to understand Year 7 and 8 maths as his teacher (even if I had to find out what was on the syllabus, and the methods they preferred by using the textbook). And I know when I have made a reasonable fist of it when suddenly they start getting good scores in the tests whereas they had a couple of appalling ones before I stepped in. And I have also run into situations where it is clear to me that there was a rationale to what was being taught at KS3 (for both my kids, I tried to explain to them in Year 7/8, where they were asked to simply memorise the period table and the various reactions of groups, about electron shells and electron sharing so as to make some sense of why the groups behaved as they did, only for them both to be baffled by this, only for it to click for DS when he hit GCSE, when it actually gets taught – my conclusion, although I don’t regret telling them, because I think it is important to understand that there is some logic to It, but that they weren’t at a stage where they could realistically grasp electron shells, just as I cannot realistically expect my Year 11 to grapple with probability wave collapses). And because I do understand the science and maths that is being taught, and have been forced at times to follow the syllabus quite closely, I am comfortable that in the case of my kids, and maths and science, I could map them both against the syllabus, and against what is reasonable for them to know, and having now some knowledge of what is taught at GCSE, I reckon I could make a reasonable fist of mapping my KS3 child against where she needs to be to do well in GCSE in maths and science. I cannot do that for English or Geography (although if I put a bit of work into it, despite my having dropped geography in Year 8 myself, I might be able to use the textbooks sufficiently to have a pretty reasonable idea, because it is not that far from statistics and various other subjects I have studied), but English I cannot. But I certainly do not think it unreasonable for an English teacher, trained in the subject, to be able to do what I can do for maths and science. So why is it that I feel I can do it, and yet all the actual teachers claim they can’t. I would never go into teaching because I cannot do, and do not want to do, crowd control, and handling disruptive behaviour. But that is not what is being measured here, it is learning outcomes, why is this so impossible?

So why is it that I feel I can do it

Basically it’s the Dunning-Kruger effect.

A few hundred kids is NOT statistically significant in a cohort of hundreds of thousands.

You can do that for an individual child - understand every nuance of their understanding and grasping of a subject; but when you teach 200 or more children a week, it is virtually importable to have this nuance of understanding for each of them. Tests can be useful to feedback to the teacher where kids haven't understood something, but the curriculum time is limited (particularly in ks3) for the teacher to do anything about it but hope the child will go back and try to improve their understanding.

The issue is there is a HUGE range of expected progress during secondary school. Some kids will make masses of progress and others will stall - progress is NOT linear for an individual. They will stall and stagnate at times and make leaps and bounds at others - this means short term data (from which many teacher's jobs are judged) is not a realistic reflection of a cohort.

I get that parents want to know if their kid is doing ok. But data (which to be fair is not standardised across the system except at the end of the gcse and a level courses) is not a reflection of 'progress' but performance, generally on too short a time-span to be indicative of progress.

Most parents (and teachers) don't have a strong enough grasp of statistics and the data performance measures of the new GCSEs to be of much help.

I've personally been shouted at by a parent when I gave a 'working at grade' of a 7 (which was based on the last topic) when his mother insisted he should have been 'working at' a 9 because he had scored much better in previous assessments. It was ridiculous because at that point we'd made up the grade boundaries!!! Which is the only thing that CAN happen at ks3 because there are no national measures. So how can data based on nothing be meaningful?

Hmm,

Wikepedia summarises the Dunning-Kruger effect as follows:

"In the field of psychology, the Dunning–Kruger effect is a cognitive bias in which people of low ability have illusory superiority and mistakenly assess their cognitive ability as greater than it is. The cognitive bias of illusory superiority comes from the inability of low-ability people to recognize their lack of ability. Without the self-awareness of metacognition, low-ability people cannot objectively evaluate their actual competence or incompetence"

So what noblegiraffe would seem to be saying, is that because of my low ability, I do not have the self-awareness to understand that I am out of my depth in terms of assessing my kids, and this is why, unlike those wiser and more intelligent than I am, I do not doff my forelock to my betters in teaching and accept that it is not reasonable to be able to give an educational prognosis.

Oddly enough, the research for the Dunning-Kruger effect went as follows, again according to Wikepedia:

“: Dunning and Kruger tested the hypotheses of the cognitive bias of illusory superiority on undergraduate students of introductory courses in psychology by examining the students' self-assessments of their intellectual skills in logical reasoning (inductive, deductive, abductive), English grammar, and personal sense of humor. After learning their self-assessment scores, the students were asked to estimate their ranks in the psychology class. The competent students underestimated their class rank, and the incompetent students overestimated theirs,”

Of course, this research assumes that in fact there Is a true class rank that can be objectively determined, against which the estimate of each member of the class was compared. According to noblegiraffe and others, such an objective test is impossible, because progress is not linear, and therefore there is no way of objectively determining the class rank in an introductory class, so in fact the incompetent students (and the competent students, for that matter) might actually have been right, and the assessors of the class wrong. The very existence of this research is predicated on exactly what is being claimed is impossible to determine – that the rank of relatively novice students can be assessed, so that such rank can be compared with where the student themselves think they are.
Whereas what I am saying is not whether my kids can accurately assess where they think they are in the class (my kids generally under assess - in fact my DS came home distraught after his first maths mock a couple of weeks ago, saying he completely messed it up, only to be emailed this weekend by his maths teacher to say he got a 9 for his overall mocks. Classic Dunning and Kruger – again from Wikepedia “Moreover, competent students tended to underestimate their own competence, because they erroneously presumed that tasks easy for them to perform were also easy for other people to perform.” Hence even if he did mess up several questions on the mocks, so did sufficent others that the result was a 9 for him). The question is can a class teacher make a reasonable assessment of a class rank at a novice stage (in psychology, or maths, or science) and have it be meaningful. The research would seem to assume that one can.

A few hundred kids is NOT statistically significant in a cohort of hundreds of thousands.

That is interesting, I wonder how many novice psychology students Dunning and Kruger asessed in order to determine that their effect was statistically significant across the teaching universe? Universities usually do not have casts of hundreds of thousands of novice psychology students whose rank they have assessed (so it can be compared against the self estimate of such students, after a battery of tests has been run).

You can do that for an individual child - understand every nuance of their understanding and grasping of a subject; but when you teach 200 or more children a week, it is virtually importable to have this nuance of understanding for each of them.

I think this is probably closer to it. Teachers are required to teach too many kids to be able to teach properly, i.e. to cater for individual nuances, and spend so much time dealing with discipline and disruption and the rest of it, to really get to know their students, especially the quieter ones. And because they are not allowed the kind of time they really need, they are forced to make things up, to cover up the fact that they have absolutely no idea. I remember showing up to a parent/teacher evening in Year 3, and it was quite clear when we said DD’s name to the primary school teacher, she had absolutely no idea who DD was. When she then looked down at her row of data, she looked up and said DD was doing fine. No doubt she was so busy dealing with the disruptive kids and the precocious over achievers that she just didn’t notice DD, doing her best to stay under the radar and coast. And that is a teacher with under 30 kids in the class!

So what noblegiraffe would seem to be saying, is that because of my low ability, I do not have the self-awareness to understand that I am out of my depth in terms of assessing my kids

Nope, I’m saying that because you’ve assessed your kids against a textbook, that seems to have given you the false confidence to opine on the quantification of assessment of progress of of a cohort against national standards that don’t exist.

Yes you can measure class rank. But class rank tells you bog all about where the student is supposed to be.

As a parent of Y7 and Y5 children, I would be very happy to see the back of this 'progress data' being sent home. It seems to create more anxiety and unanswered questions than anything else.

But I would like to know how my children are doing and specifically what they might need help with. Is there any way for schools to supply that information to parents in a useful way?

Thehorse,

I agree.

The difficulty is that parents, students and teachers all need some information about how a pupil is doing between the end of Y6 and the end of Y11 - but there is no agreed, consistent and reliable way of obtaining or providing this information.

So on the one hand, schools refusing to provide parents with any information about whether their child is likely to obtain 8 9s, or 8 1s, or 8 5s at the end of Y11 is unreasonable - because it prevents parents and pupils from making good decisions and taking sensible actions (about subject choices, post 16 courses, about the amount of work that needs to be put in...)

On the other hand, providing huge quantities of 'apparently accurate but in actual fact meaningless' data is time-wasting, and can be damaging, especially if decisions and actions are taken on the assumption that this data is accurate and meaningful..

Noble and others - what would be a sensible way forward? An 'expected' level / knowledge for each year / couple of years that is nationally agreed? It is all very well to say that the status quo is wrong, but what would a better solution, which provides meaningful and actionable information to teachers, students and parents alike, look like?

Well they can tell you how they have scored in tests and what they would expect someone to score on that particular test if they were keeping on top of the curriculum. And they can tell you strengths and weaknesses and therefore areas to work on yes.

what they would expect someone to score on that particular test if they were keeping on top of the curriculum

But is the test set at the right level for that part of the curriculum (it is possible, indeed easy, to set a test that everyone gets 100% on, and alternatively a test of the same part of the curriculum in which virtually nobody scores above 10% - see the first round of the 9-1 GCSE Maths trial papers)? Who says what 'the expected score for someone on top of the curriculum' should be? and how long after direct teaching of that part of the curriculum is the test valid? Immediately? A week later? A month later? A year later? Is the test set after revision and re-teaching in class? Or unannounced?