Evidence teachers may be biased in their teacher assessments?

[DiamondAge]

It seems there is evidence that teachers may be biased in their teacher assessments, and that children are "stereotyped by 7".

Here's a link to the full paper, along with a link to a summary.

The suggestion here is that teacher assessments might be less accurate than, for example, externally examined tests or child completed tests and that teacher bias may be at least partly responsible for the disparities found in achievement between girls v boys and also for pupils needing FSM or having SEN compared to other pupils.

I've just settled down to read the full paper but would like to know what parents and teachers think - does this research match your experiences / make sense to you and if so what would you do about it?

I will read this as I think it's very interesting.

My experience - ds1, in year 4, did their tests a couple of weeks ago and he had a really hard time. He had a cold, he was very tired, and he struggled with the maths exam, quite badly. He is working on a 4b sublevel, but got just a 3a - and his teacher, in his wisdom, decided to keep him on a 4b sublevel because he exercised his judgement, he knows DS very well, and knew that his performance was affected by not feeling well. An external examiner would have not been able to make that kind of allowance.

Having said that, teachers are people, they are not robots. They know the children (hopefully) and we cannot expect them to forget all they know about a child for the sake of one 'exam'.

I absolutely think teachers are biased.

DDs tutor grades DD much higher than her teachers who have pigeon holed and stereotyped her.

Everyone's biased. So I fail to see how it would be possible for teachers not to be biased.

Research 'cognitive bias' and you'll find heaps of info on this.

There's a good book on this but I can't remember it's name.

What would I do about it?

• stop performance related pay. Because this encourages teachers to inflate grades.

• have more standardised tests available to schools, I guess produced by the govt and free to schools, which are marked by class teachers but provide an objective not subjective level.

• make targets based on IQ (cats tests type things) as opposed to last years level + 2 (or 3) sub levels . This will catch out children who are behind their potential.

These are good ideas Brilliant.

As a parent, I worry about performance related pay for teachers. I think it will do the system much more harm than good.

Of course teachers are people too, and people have biases. It's part of being human.

The first idea would stop the introduction of additional bias into the system. The second two ideas would help teachers recognise the natural bias that they already have because they are human beings.

If teachers have inflated their grading it does absolutely no good in the long run because of external SATs tests! Also good schools sample check to ensure the work is graded consistently across school. The new curriculum, which a year 4 child is working on, is not graded a, b, c any longer so you should not have this next year. A paid tutor is more likely to inflate grades than a school - they want parents to have value for money! What curriculum are they teaching anyway? The new one?

Performance related pay is not all about the performance of the children, as given in a final grade. If teachers have other responsibilities, these can have performance targets too. Why is it wrong to expect teachers to perform well? Every other job holder has to? Also the performance criteria will be on progress made by the children. It is crude to use final outcomes. A cohort might be very bright or very average so final attainment is not necessarily a measure of good teaching. Lesson quality is also part of performance so performance pay and link to grades that you are making is not so clear as you think. Also, increments are given, or withheld, according to performance. It is not a huge amount of pay.

I think it is a very poor school that would allow teachers to inflate grades thinking they would achieve their increment! Not where I am a Governor anyway!

Also, you don't fatten a pig by weighing it! So you don't measure progress by continually testing. Performance pay is designed to enhance the quality of teaching to ensure the children make the most progress they can.

You're a governor? You sound a bit naive. I assume most of your information comes from the HT?

Governors don't get much of a true picture of what happens at the coal face. They only get the edited version that's presented to then at governor meetings.

Why don't you ask a teacher how many of their pupils came up with inflated grades.

Teachers are biased, they have gone for the system to work (I don't think it is right). They have to level individual testa and pieces of work which levels were never ever meant for. Most teachers I know do their best but it is difficult.

Performance related pay had already shown to have little or a negative effect on jobs requiring higher skills. Even the project started in the states that gove wanted has been hit with controversy due to teachers inflating grades.

Anyway, everyone has bias. Teachers have bias but then standardised tests can misrepresent a child's abilities. Even externally marked work can be marked with bias.

I'm not sure there is a solution to that unless it is really severe.

I agree bias is inevitable and as HayFeverHell stated awareness is key.

Working in a LA for years meant attending training to help people recognise their prejudices. I know teachers have enough to do, but so did everyone working in social services etc. In fact the training was compulsory because trying to deal with prejudice was prioritised over work load.

I'm not sure how bias in this instance compares to prejudice with respect to disabilities for example, however if the bias is significant, as this report suggests, then awareness seems a good starting point.

I had the opposite to StrictLyison. My son did the assessment tests. The teacher told us he did well in one of the subjects, but she had decided not to give him that grade as she didn't really think he was at that level.

If it's subjective, then of course it's going to be biased.

BrillianDayForTheRace think about it .

The Y1 teacher inflates the level so
The Y2 teacher further inflates the level so
The Y3 teacher is forced to inflate by this point the child is supposedly working at the expected level for Y6 and there's no going back...
What happens in Y6 when the child isn't anywhere near? Any teacher inflating levels isn't only hurting the child but they are also shooting their colleagues and school in the foot. It's why schools moderate across the school/key stage.

Yes I've known teachers who inflate levels but it was before performance related pay was even thought of never mind introduced ... The teachers reason .,. She didn't want the parents to be disappointed if their child hadn't reached expected levels (because she'd been upset when it happened to her child) crazy! She also wrote glowing reports for every child for the same reason! It was an absolute nightmare for the next teacher. She now works in a call centre!

By inflates the level I mean claims pupil have made progress / expected progress when they haven't.

This only applies to a few pupils in their class. So they get away with it.

Their colleagues can't complain without looking bad.

It's also in the HTs interest. Because if pupil is making no progress it's a massive problem.

But expected progress isn't good enough for performance related pay... Children must make better than expected progress.

I've been in the position where I've had to tell parents their child is at a lower level than the previous teacher reported and guess who they blame and who they think is a wonderful teacher...

It definitely isn't in the head's interest to inflate levels because schools are judged on progress based on end of KS results. An inflated level lower down means that the school won't show value added.

If a school wants to game the system, it's in their interest to set a low baseline and inflate the final year. I think this is some of the tension between primaries and secondaries when kids move up.

Since SATs are externally marked in year 6, I assume the secondaries are trying to set a "conservative" baseline.

Except the final year is externally marked so the school have no control over it.

I just said that.

I know ...?

Add in explicit 'teaching to the test' and exam coaching and it gets more complicated...

The aim towards 'value added' points seems most noticeable IME. KS2 was so much less stressful than KS1 when the teachers, especially reception, seemed to want to highlight any small difficulty...

Ive only read the summary and scanned the report but it throws up lots of questions.
How did they select the children included in the study?
Is it a national picture or local to the IeO?
How did they match cognitive tests to teacher assessment as they measure completely different things?

I agree Mrz, in particular your last question made me wonder. On the other hand many schools set - would it be that surprising if children in lower sets have greater potential than their sets allow them to display?

And which children are more likely to be in those lower sets than the ones identified in the report?

"it throws up lots of questions."

Yes it does. I remarked on the disconnect between the 'cognitive' tests and what primary teachers teach and assess the other day (elsewhere).

However, those cognitive tests were apparently PiM (Progress in Maths) and some British Word thing so vaguely, but not necessarily well correlated with core school-stuff.