CAT4 Year 5 Result Interpretation

[nasrinconfused]

Hello all,

This is confusing me a lot. Yesterday I received year 5 CAT scores and am very confused about how to interpret them. Are the below excellent, good or bad scores? How do they compare to the averages? Can someone pls advise?

Just typing the scores from the second row which are relevant:

Verbal 141
Quantitive 127
Non-verbal 136
Spatial 126
Mean 133

They'll have sent instructions on how to interpret them, surely?

Hi OP, the mean score is 100, which means that your child’s scores are in the very high range across the board.

Universally this puts you in the 95% percentile for the mean averages.

It gets a bit different with context as I'm quite sure the school in question expects at least 125 rather than the average across all students of 100.

This school also has the highest CAT4 scores according to atom learning.

Either way it's still good.

You can't really revise but maybe expose your DC to the areas he's weakest in from a non exam perspective. More spatial optimisation problems like certain physical/computer games but also map navigation etc. But generally this is for the school to adjust work.

The mean score is top 1 percent

Yea but also no unfortunately.

Within that school the range is 125-145 (as below 125 is intervention level) and as it's selective they ensure this at entry.

It means allowing for normal distribution the score above is average in that cohort.

Parents in Y5 are especially anxious as they're going to be doing the 11+ by the end of the year against external candidates also gated to above 125.

So yes the score is good but that's why people are asking. As there are kids at the higher end getting a 141 clean sweep in this reasoning in the same year group.

Thanks all for your responses, gives me some context.

141 is 99.7% percentile
133 is 99% percentile
the 126/127 will be somewhere around 95%/96% percentile.

Great marks overall. Are mean marks of 133 common?

Maybe should have exapanded on the last comment.
141 is 99.7% which is give or take 3 in a 1,000 kids.
Do Cat4 actually give that mark to 3 in a 1,000 kids?
Just I seem to see a fair few 141's knocking about on here and they should be rarer than hen's teeth going off what I'd expect.

I'm with you on this as DS got 141 SAS in pretty much all too -- there has to be a cohort wide approximation that assumes a low attainment in the population not doing CAT4.

In selective schooling these scores are normal so I'm not sure where the average is being pulled down.

Generally why I also ignore the "top 1%" statements as the actual kids in close proxmity are all scoring within a very limited range of 130+

If you think long term for the same kids they'll need to score 99% at GCSE, and 95% at A-Level to maintain the school's historical averages so maybe these kids are all the top 1% in small number of schools.

But then what is top? Is it just being good at tests?

@elij @Jonny234

CAT scores are normed against a broad segment of the population when setting the distribution curve for scores including state and private schools in representative proportions as well as other key demographic characteristics to mirror the UK population. The norming process is similar to that done for IQ scores so a CAT score of 133 is top 1% of the academic ability range relative to the general population. The GL-assessments provide all this information online if you want to look into the details.

Why you see so many on mumsnet is because people with high scoring children like to talk about it and the education boards are filled with lots of parents targeting highly selective education talking about CAT scores.

Also, CAT scores are used by state and private schools. They are part of progress tracking and predictive grades.

141's "in pretty much all" is great going. Fantastic. Congrats to your DS.

After seeing your relevant comments and following my own observations I'm kinda seeing both sides. I know a selective indie London school with what must be mainly top 10%-20% chosen on ability is about the equivalent mean average on Cat4, so that bit i'm fine with. I see that with my own eyes.

Then I wonder if the assumption around the normal distribution which holds around a mean of 100 does not apply in a school with say 120 average because the distribution is skewed right or left a bit. Perhaps the profile is flatter?

Perhaps it's as simple as parents with higher grades being more proud, willing to share, or similar. If you have a DC on a 130+ average I'd guess people are more willing to publicise it than if they had say a 99 average.

My last post should have stated @elij , apologies

@MomFromSE I agree with what you say, that adds context.

Many thanks to both.

An academically selective school will have a higher average score than the general population by definition as they are selecting and admitting by academic ability @Jonny234

Thanks @MomFromSE and again agree.

Your comment from earlier has me thinking now, what was the lowest mark of the 4 tests for your DS?

For DS it was spatial reasoning which brought down his average I think it was 127 or so. The rest were maxed.

@elij sorry what school in question? do you know which school OP's DC goes to?

I can't really speak for OP but we are at WUS and the screenshot looks like the result section of the parent site.

(It's quite a unique looking site as it was developed in house)

Yes this is WUS. We have the same report for DS.

The way they norm the tests is that they give them to a (hopefully) representative sample of subjects, calculate the mean of the raw scores, calculate the standard deviation of the raw scores, then rescale the scores so that mean is 100 and the standard deviation is 15. The inference that 141 represents the 99.7th percentile, i.e., is achieved by only 3 in 1000 subjects, is true only if the underlying distribution of scores is actually Gaussian (Normal). It is not. It is well documented that the distribution of these scores is leptokurtotic, colloquially, "fat-tailed". The actual distribution of scores has far more "extreme values" and "outliers" than a Normal distribution would.

When they norm the tests, what they do not do is find the 3rd highest raw score in 1000 and assign a value of 141 to that raw score.

Remember from statistics:

• You can calculate a mean of any set of numbers or distribution.
• You can calculate a standard deviation of any set of numbers or distribution.
• You can only infer confidence intervals for specific distributions.

Transforming a raw score so the average is 100 with a standard deviation of 15 points is how IQ scores are done which is what I said. I’m not sure what you are disagreeing with if anything @user149799568

GL explains how they do their sampling to match the UK population and it involves thousands of pupils. Each subtest has a standard deviation of 15 though the average of the 4 tests doesn’t.

Where is it well documented that cat scores are leptokurtotic? I’d be interested to read that.

@MomFromSE thanks for that. Absolutely great scores, you must be very proud. I think I need to get my DC on Atom, etc.

On the subject of the distribution I have looked at both your and @user149799568 comments.

I was just thinking around the angles this morning. The 141 score cut off point is an arbritrary decision, and I can see various reasons for this, notably if scoring did continue to say 161 then a DC could score 1212 and 1612, average 141, and that would be a lot different than 4*141's across the board.

In cutting off at 141 there is a cumulative area after this. Then I thought what would the distribution look like if the cutoff was at 121? Well we'd have a regular distribution profile to 120, and then 121 would be those actually scoring 121 and all the rest 121+ (circa 10% ish of the whole population). So the distribution to 120 would be normal, and then at 121 there'd be a massive spike. Scoring 121 would be far more likely than 120.

Intuitively the same will happen at 141. Although we think of 141 as 3/1000, this score in reality picks up all those above. I'd guess 141 picks up maybe 5-8/1000 marks which at a top indie with top 10% of the population could be 5-8/100. Again I'd expect 141 will be far more popular score then 140.

I think calling the distribution leptokurtotic is probably correct, but fat-tailed appears incorrect. Simply speaking it's just a spike at 141.

They cut off at 141 for a few reasons. In general for tests of this nature, once you get beyond 2/3 standard deviations the results are less reliable given how rarely those scores will occur in the population. Most modern IQ test also stop at 145 for that reason. The second reason is as a practical assessment tool rarely do you need to assess intelligence beyond this.

So for instance, CAT4 scores are used by schools (including most state schools) to track pupil progress. So, if your child has a CAT score of 125 for example but their academic progress / performance doesn't reflect where they statistically would expected to be based on that score, it will be a red flag to the school for an intervention. Going beyond beyond circa 3 standard deviations isn't really useful here.

Similarly, IQ tests are diagnostic tools. Spiky profiles can reveal underlying learning differences that are being partially masked due to high intelligence etc. However, they aren't designed specifically to 'test' extreme intelligence for its own sake.

Stopping at 141 should still produce a normal distribution of scores. A slightly high number of extreme scores would impact the standard deviation not whether it falls into a normal distribution curve or not. There are a variety of shapes a normal distribution curve can take. A leptokurtotic distribution is very extreme; intelligence doesn't follow that distribution in the general population and I've never seen anything suggesting CAT4 scores have anything besides a normal distribution curve including the same standard deviation of 15 as most IQ tests.

If you are really interested there is a lot more technical information at the link below:

support.gl-assessment.co.uk/media/2794/cat4-uk-technical-report.pdf

@MomFromSE you wrote:
Why you see so many on mumsnet is because people with high scoring children like to talk about it and the education boards are filled with lots of parents targeting highly selective education talking about CAT scores.

While the selection bias inherent in the parents who participate on mumsnet and the parents who are more likely to volunteer their DC's scores does sometimes make it seem like 130+ scores are barely above average, it's not the only reason that 141 scores pop up on here so frequently.

Raw scores on IQ tests are fat-tailed and norming them with linear transforms preserves those tails. I don't have the citation to hand, but I seem to recall that, on the Stanford-Binet test, the actual frequency of a nominal 3 standard deviation above average score was about three times what you would have expected from a Gaussian distribution. If the CAT4 is similar, I would expect about 1% of the population to get 141 on a given test. Though, again, I agree that it does seem like a much higher percentage of our biased sample.

@Jonny234 while you are correct that the probability of getting a 141 on the CAT4 is higher than getting a 140 because the normed scores are capped at 141 so all higher scores get pushed into 141, the 0.3% probability being tossed around on this thread for a 2.73 standard deviation right tail (41 / 15 = 2.73) is actually a cumulative probability, i.e., the probability of 141 or above, not of exactly 141. Using an ideal Gaussian with mean 100 and standard deviation 15, assuming that all rescaled raw scores between 139.5 and 140.5 would be reported as 140 and all scores above 140.5 would be reported as 141, you would expect to have about 0.08% of scores at 140 and 0.35% of scores at 141. If CAT4 scores are leptokurtotic consistent with Stanford-Binet IQ scores, we might see 0.24% of scores at 140 and 1% of scores at 141.

Those are an excellent set of CAT scores. Year 6 CAT scores are very predictive of GCSE results, 81% correlation according to the research:

www.sciencedirect.com/science/article/abs/pii/S0160289606000171?via%3Dihub