ISEB exam results - bad day or system botch up ?

[ElonanotAlone]

Do you think the ISEB results truely reflect your DC's actual ability? Just that really.

We have been fortunate to receive exam feedback from a couple of Schools with quartile cohort results and eventually our actual scores from a sympatheic and understanding school. As a consequence we have been shocked by the inconsistency of ISEB results versus our DC's known competency on a particular subject, notably English and Maths.

Our view is based on known performance on a number of verifiable asessments - a non ISEB exam eg Grammar, mocks, school exams, CAT4 outcomes, etc or teacher expectations of performance.

We have tried endlessly to rationalise the outcomes, perhaps DC had a bad day, greater competition, however, in discussion with a few at school others facing similar outcomes, we have come to ponder, was there a glitch or ISEB malfunction with the exam format questions/answers or set up issues within the inviligation centres?

Hence, curious to see if thus is an isolated school issue or one felt amongst other parents feeling disillusioned with 11+ choices we are forced to accept..

We have had a similar shock when a sympathetic school shared the scores with us and they were not in line with what our DC can do based on school reports, tutor feedback, mocks and of course the feedback received from other 11+ exam results.
Basically the ISEB scores look like they belong to another child ! It did cross our mind that login details may have been mixed up. In addition, there are all other the place anyway, with none of the usual correlations between the subjects.
Maybe the algorithm has hiccups in certain particular cases?
It’s a shame so many schools decided to use it as the main exam and we are grateful to those who did not !

This is why schools don't usually share scores.

Are these schools that have exam papers for those who are successful in the ISEB and if so did your DC sit them?

If your head teacher thinks your child scored lower than they should have done they can speak to the schools in question and plead your case. That said most schools I've spoken to said they look carefully at the school reference and if there is a discrepancy between expectation and outcome they speak to the school.

The ISEB isn't the same as a CAT test and I'm not sure an ISEB "score" can be compared.

Every year parents complain about the process but you never hear of children who have very low CAT scores getting into top schools.

I'm sorry. It's a brutal process. We didn't get our top choices but please don't let your DC believe there is a glitch in the system. It makes it impossible to move on, won't change anything and if they are bright I'm sure they will thrive wherever they go.

Sorry to hear about this -

There should be a good correlation between CAT4 and ISEB. Both are made by GL (although some cognitive tests are by CEM).

CAT4 by GL

Verbal Reasoning – the ability to express ideas and reason through words is essential to subjects with a high language content, and the most obvious skill picked up by traditional assessment.

Non-verbal Reasoning – problem-solving using pictures and diagrams; skills which are important in a wide range of school subjects, including maths and science-based subjects.

Spatial Reasoning – the capacity to think and draw conclusions in three dimensions, needed for many STEM subjects, but not easily measured by other datasets.

Quantitative Reasoning – the ability to use numerical skills to solve problems, applicable well beyond mathematics.

ISEB by GL

English – 25 minutes (reading comprehension, sentence completion, spelling and punctuation)

Mathematics – 50 minutes (in line with Year 5 of the UK National Curriculum)

Non-Verbal Reasoning – 32 minutes (shape analogies, classes like, horizontal codes)

Verbal Reasoning – 36 minutes (common words, antonyms, word combinations, letter transfer and number codes)

Based on these, the obvious differences are English (could be covered by VR) and Spatial Reasoning (could be covered by NVR).

It could just be a bad day -
We know of a DC - went for tests in a few grammars - got top/ almost top scores/secure allocation except for one school
Got scholarships/ offers from super super selective indies
What happened to that one grammar school?
It was one of first test - likely the first
DC did not finish a few sections -
Nerves?
It was a bad day -

Agree with @LondonMummer try to plead your case to your prep's Head

It hasn't made any sense for us either. Son's CAT average is 132, and he's really, really strong in maths. Can do lateral-thinking/algebra scholarship questions in a blink, beats his older sister at her top-set maths homework, etc. However, he often struggles with, and hates, English comprehension.

Apparently he did better in English than maths at ISEB. This just doesn't happen with his exam scores, ever.

He didn't even get an interview at either of the two selective schools he went for. It's the total opposite of what we were led to expect.

@ElonanotAlone The same happened to us. One of my DC's ISEB scores sounded like it belonged to another child. It wasn't a bad day but a car crash which doesn't make sense at all given they are always top scoring in this particular subject. I have sent you a PM.

Did you get the raw scores but not the cut-off line for each school? It could have been that all the schools had a lower entry line this year and your son was just under the line.

In answer to your original question OP-we got an expected result based on ISEB and know of various others who did too. Those expected to get into certain schools were offered places and the ones who did not were ones for whom those schools would have been a stretch. So I am not aware of any specific ISEB glitches myself.

That said though, I am sure that every year with ‘normal’ 11+ papers there are DC who don’t get offered places, and tutors and prep heads who are shocked based on the child’s performance to date.

I don’t know how much success anyone would have with challenging the ISEB system though. I am aware of inconsistencies between schools, especially with preps who allowed dc to spread the sections over a couple of days whilst state school dc had to do all four back to back in one morning. Was that unfair? Possibly, but within ISEB guidelines apparently. So they’re unlikely to do anything about it if parents complained.

But unfortunately sometimes (with any exam - I work in secondary exams admin and it happens all the time) it really is just bad luck on the day or with one particular exam section. Sometimes it is nerves - a feeling of pressure that this really is ‘the real thing’ and everything is riding on it - which you simply don’t get in any classroom test or mock exam (or CAT assessment). This could well have been the case with 11+ ISEB too especially if multiple schools were riding on it. Sometimes just one question at the beginning of a formal exam utterly unsettles the student and throws their whole performance off kilter. Or they know a subject is perceived as their ‘best’ and then panic about not doing as well. It’s sad but it does happen.

I hope your DC ends up at a school they are happy with in the end.

We’ve had the same experience. ISEB scores don’t reflect my child who has been on the top table since reception. We didn’t over tutor but had once a week tutoring since mid- year 4. Plus we did Atom for 1 month. We didn’t even apply to a very selective school as a more rounded school would have suited better. Oh well.. I’ve decided not to over think it. Our local state is a fab/outstanding school and very oversubscribed so we are lucky to have got it.

It’s good to reflect but I feel very important to now look forward to the school your child will be going into. Keeps morale up and they don’t feel they are going to a second rate school. My two pence.

Such good advice from @FlyingPandas - remember the CATs test underlying innate potential (in theory) whilst the ISEB tests the actual curriculum in Maths and English. My son scored 140 on maths CAT and then 107 in a school Atom test under exam conditions - maths went from top to bottom of the four scores. I think questions came up on areas he was weaker in or hadn't yet learned.

Do also remember that tutors will always give you a sense of what how a child will perform on a good day rather than a bad one.

Mostly though children have ended up where you would broadly expect. There are always exceptions but nothing to me indicates a widespread error in the testing - which you can be sure would have been picked up and addressed by schools across the country.

@ElonanotAlone Thank you for starting this thread! In my most humble opinion there has been a mess up in the ISEB exams.

Senior school admissions teams are obviously going to defend the situation because they are used to being criticised for selecting one child over another. That is their right in a normal year, when children sit a written paper, because they can compare children on exactly the same exam: the same English comprehension.

ISEB however has a large data pool of questions which SHOULD be age specific for 11+. The randomiser then selects from that data pool. In theory that is all well and good, as long as the data pool has been correctly loaded with age specific questions.

The problem is that ISEB and their staff were all working remotely, so were the checks and balances put in place when they updated the data pool and when they ran the algo for the results?

Another mum has experienced the same and has commented on another thread that at her DS's school the English teacher administering the test looked over a child's shoulder and commented afterwards that in their opinion it was a 13+ paper while all the other children had 11+.

Any educationalists will know that there is a correlation between different tests. So therefore yearly CAT4 tests, attainment tests, CEM tests etc gives a school a clear indication of a pupils overall ability and performance.

I totally accept that children can have good and bad days, which is why the admissions staff will always argue their point. However in this years ISEB we are not seeing a child having a bad day across all 4 sections of the ISEB, we are seeing one or two of the sections being total anomalies .

The head of our prep brought up the ISEB issue independently to us and to one other family a few weeks ago. His opinion having spoken to one or two other heads was that the issue was very much English orientated at the time, but that may have evolved since.

What we are seeing, reading the comments above and on other chats, and from some head teachers, is that for some pupils the 'range" in 4 ISEB results is very large. 3 scores at 130 and one at 110, range of 20. Clearly a large 'range' will be more easily identified in pupils who normally score highly in a CAT score. This does not mean that a pupil who scores 110-115 in a CAT score has not been effected by the ISEB problems, but the 'range' might be less and so less obvious to the Heads or the educationalists.

I know people will come on here to rubbish this discussion, please don't, there maybe a problem or there may not, but until more mums start sharing their experiences we will not know.

Our experience with ISEB has been similarly disappointing and not reflective of DC's ability or strength.
DC got low score in strong subject and great score in the least strong subject!
The other two scores were in between, but definitely on the low side of what was expected with no correlation in between.
we have other examples now of DC's friend who have gotten their lowest marks in their respective strong subjects (not always the same subjects).
It just does not seem to be making a lot of sense, and of course no further details are available.
ISEB may be ok as a pre-screening test followed by a full-on test, but as a sole exam very much less so. it's mostly a matter of clicking after all !
DC's friend got offers from all the ISEB schools, but none from the non-ISEB schools- admittedly less selective, so parents are concerned they might have gotten accidentally in a school they won't be able to keep up with !

My DD had said the Maths test was extremely difficult. She was especially shocked that even the practice questions were so hard, and the test started hard and was hard all through. Whereas for some other students, practice questions were very easy and test started easy and went to difficult as it should have been. Our tutor also said he got the same feedback from his other students that they suspect something was not in line this year with the ISEB ( at least for the Maths part). My DD’s strongest area had always been Maths and based on feedback we received from one school, she did the most poorly in Math. Having said this, she still got nice offers from 4 different schools, two of which are very selective. So we are not in a position to complain. However, she missed her chance at her most preferred choice due to this.

I don’t think it is unreasonable to ask whether their system was robust enough to deal with the very large increase in candidates and venues.

For us though, I think the problem was mostly that it was an exam that exposed our child’s weaknesses (tendency to make silly mistakes on easy questions, no experience with NVR which we weren’t expecting to feature in the process at all) without giving them an opportunity to show where they shine (can cope with the toughest questions at the end of maths papers and strong at the extended thought needed to produce good creative writing and comprehension answers). We also took the schools’ ‘no special preparation required’ advice too literally, and thought we were pretty clever doing 5 days on Atom, only to discover that children at prep schools had been using Atom for months.

Thankfully we are happy with our state option but the lack of transparency and feedback is not helpful.

What most people don't realise is that ISEB tests are progressive in nature i.e. the questions gets harder or easier on the basis of whether you answered the preceding question correctly or incorrectly.

Understandably harder questions carry more marks than the easier ones.

If a child answers some of the initial questions incorrectly then they will only get easier questions later on which means they might score less marks even if rest of their answers are correct.

This year many more schools relied upon the ISEB results for initial screening which under normal circumstances they don't. Therefore there was much more chance of rejection if DC doesn't perform well in just one exam.

It was not fair and I understand how you might be feeling.

@ElonanotAlone
So glad you have raised this matter. I posted on another thread on the very subject, because I genuinely think the algo did not work.

For our part, DS is bright and has been since reception, CATS score averaging 135, a little genius in Maths, been to prep school and prepared for exams; no anxiety or evident stress; DS loves exams and was very excited - though not so much for the interview process.
So he met thresholds for the Sutton grammar schools and was offered Tiffin ( really pleased) but we also attempted Hampton as DS headmaster promised scholarship there - the headmaster wanted us to also apply to Westminster, Kings, St Paul’s, and some others.. anyhow DS only made it to WL.. I called the school and found out his results by quartile ... I guess 3 things from the discussion: either the results are incorrect; either DS had a terrible terrible time or he was handed over the hard set which some other parents refers to. DS did say that the English comprehension questions were weird, there was a technical glitch on the day.. could this be the problem ??

I feel sorry for all the bright DC that did not get anywhere this year... I know that my DS prep school is going to have to deal with some serious questioning. I am not hearing good news especially with the 13+ process !

If you feel wronged by, please start a petition / challenge the process officially... as I quote from above: “ it is not unreasonable to ask if the system was robust”.

You know in your heart of hearts if this does not feel right /match you/headmaster/tutor’s expectation.
Am just glad the state schools did not use ISEB !!
I hope you all get to the bottom of this issue.

@Sterou but the state schools did use ISEB? Everyone had to sit the same exam. Some under less favourable conditions perhaps as there was little consistency in the way it was administered...

Lonestar123 what state schools used ISEB?
I am not aware of them?
Yes it is adaptive and also dependent on age.

Also you can not make mistakes early on as otherwise it is harder to reach the more difficult questions.

Our child did best on 2 subjects, average on 1 that is usually his best, and poor on 1 that is usually fine.

I would be interested if some people got actual marks as we were led to believe that they would not divulge this? Thanks

And I agree that state school students doing them in one go as opposed to prep school students doing them over 2 days is not fair!

@tryalittle I know a prep that did them over 4 days...yes one each day, in small groups each time !

My DS said at the time is that fair ? No!

@ElonanotAlone we were given the actual SAS that the school we applied to received from ISEB.
apparently schools are not allowed to share them with parents under their contract with ISEB, but some will share
As I understand the schools only receive the scores, not the actual transcripts.

@TW79
I honestly think most people understand it is adaptive.

• what is not clear is how many questions need to be answered correctly or incorrectly before unlocking the next set of (harder or easier) questions

-how the marks allocated vary from one set of questions to the other
so basically how quickly does it all spiral upward or downward ?

• can the questions go back up again, or down again... does it keep adapting or does it adapt only on a couple of key occasions

-Are the harder questions really within the year 5 curriculum as advertised ?

in the absence of transcripts, past papers and marking schemes, it's all a black box.

as someone also pointed out, was the system robust enough for the huge increase in applicants this year ?

This is before I even get to the huge difference it makes to sit a test in November compared to sitting it in January, particularly following the preceding schooling disruptions !!!
and as others have said sitting the test over a number mornings rather than all on one day.

@11plusNewbie My DS school told us the following: The adaptive nature kicks in the first couple questions itself. If a DC answers the first few questions wrongly then there is high chance that they will not get the harder questions later on.

School specifically told my DS cohort to pay particular attention to the first few questions specially in Maths and English section. Most of the questions are from Y5 curriculum however like in all tests one should be ready to receive some questions which are more challenging (read Y6/Y7) level. These questions will carry more marks and gives an edge to whoever answers these correctly.

As far as past papers are concerned, these are available to buy online and my DS practiced on these in school.

My knowledge is based on whatever my DS school told us last year. I am not defending the iSEB approach and I agree that there is lack of transparency and clarity and it should be more.

Not sure that’s a general prep experience. I know of two preps where all 4 were doing consecutively over a morning.

@catwoman1980 our DS prep did it all in one day, but one I know did it as I said over 4 days. Your are right most preps did it in one day or split on two from what we hear from friends.

@Lonestar123
The states schools we applied to had their own papers. My understanding is that only Indies signed up with ISEB.

The point I was trying to make, based on personal experience with DS is simple: How come DS met thresholds for the most Competitive Grammar Schools in the area, but not the likes of Hampton which relied on the ISEB output, when his Headmaster had categorically assured us a scholarship and his CAT scores are 99th percentile nationally 2 years in a row?

My DS got 2 subjects at top quartile and 2 subjects at 4th quartile =non-sense.
So like I said, no longer affects DS, Hampton was a back-up option, but happy to support the group as this is shocking news.

@Sterou sorry, misunderstanding as to what you meant with regards to state schools and the ISEB. Ignore me :)