The view from here - Finland

[mrz]

So often Finland is held up as an example of "good" education ...
www.tes.co.uk/article.aspx?storycode=6278622

They are the headings from the report identifying the flaws in the process math

I have to confess that when I see a page in a report with the text

'Why PISA isn't the whole truth

GOVERNMENT
CENSORSHIP
PROTECTING YOU FROM REALITY'

I sort of expect the next bit to be about flying saucers.

In a manner of speaking, I am not altogether wrong in the case of this particular piece. The critique that followed that page is garbled and incoherent and conspicuously lacking in references to studies of PISA by reputable researchers.

However, it seems to me that all of the bullets do in fact indicate something about the students in the individual countries, or suggest questions about what schools in different countries are doing -- for instance, why do students in the Netherlands try to answer all items even if they resort to guessing towards the end of the time period, while Greek students plod do not?

WRT the point 'Uncontrollable Variables' and the assertion that there are issues such as
'sampling, exclusions, response rates, test taking habits, culture, and language are quantitatively important.

• the subtitled TV programs, differences in genetic equipment, pre-scholar education, and extra-scholar environment'

that render the test meaningless --

Cutting through the clunky English and assuming the phrase 'genetic equipment' means inherent intelligence levels, my response to this one is - well DUH. Children come from different homes and parents can afford different levels of support for their children's education, children inherit different intellectual capacities from parents and are raised with or without certain values that enable them to succeed in school. If that is to be taken seriously as a critique of PISA, then it is equally a critique of all attempts to extract data from assessments of student progress.

The truth is out there
There are much better critiques out there.

Interestingly the report is from the University of Helsinki

even PISA says that there test were never ment to be taken as a bench mark to produce league tables from.

Each time PISA tests it is on a different area of study

It is not really a report. More of an opinion piece. A lot of it seems to be lifted from one of the references, the Eliane Gautschi article.

It also contradicts itself -- see the page with other international comparisons such as PIRLS, TIMSS, ICCS, and the Reading Literacy Survey 1984-1995. Finland performed very well on all. PISA isn't meaningless.

al my finnish relatives seem to study till they are about 82

Then get divorced a lot.

I didn't say that PISA is meaningless.

I said that it is being used to do something that it was never meant to do.

Sorry, BoneyBack -- I should have made clear the 'meaningless' comment was for Mrz.

I agree with you PISA is used for purposes it was not designed for and to further a good few educational policy agendas.

Where did I say that it was "meaningless" math? I'm simply pointing out that the data comparison isn't as straightforward as the popular press would have us believe.

List of critical points plus
'Conclusions

• The accuracy of country rankings is largely overestimated.
• It is clear from the outset that little can be learned when something as complex as a school system is characterised by something as simple as the average number of solved test items. '

and also mention of 'flaws in the process' plus issues with the exclusion of SEN students seem to indicate that you find the PISA test itself and its use as a means of comparing educational outcomes a waste of time/meaningless..

I don't think anyone has tried to say the popular press does a good job at translating items like PISA into interesting news without giving them some slant that was never intended by PISA itself. Politicians seem to be very fond of using PISA results to point out how well their plans for reform are based on evidence from PISA or some other international test and journalists tend to rely heavily on what the press officer feeds them unfortunately.

here is a press release from the Dept of Education as an example of one particular politician taking PISA and running with it. It has a definite POV. A journalist who was too lazy-- close to deadline to read beyond what the press officer fed them could simply arrange the release into paragraphs interspersed with quotes and hey presto, there's their article.

As an example of the press release being used for the purposes of backing up a Minister's particular agenda, consider the following quote selection from the PISA report:
Autonomy matters when combined with Accountability
Within countries where schools are held to account for their results through posting achievement data publicly, schools that enjoy greater autonomy in resource allocation tend to do better than those with less autonomy. However, in countries where there are no such accountability arrangements, the reverse is true.
In countries that use standards-based external examinations, students tend to do better overall.'

Finland is notable for the freedom to develop curriculum on the part of individual schools, lack of accountability of individual schools to the state, allowing teachers to operate with a lot of autonomy and according to their own professional judgement, and its lack of state-wide exams, yet the heading this quote follows reads:
Have clear, transparent and proportionate assessment and accountability systems.
As you can see, the bit about autonomy mattering has been dropped and the lesson PISA has taught is now all about assessment and accountability systems .
An interested journalist with time to spare might dig and find that the press release heading seemed to indicate a direction for British educational policy that was at odds with the best practices used elsewhere. But smart journalists with time to spare know that a piece on education isn't going to make their reputation for them.

You missed it math it made the broadsheets 2 years ago followed by concerns over following Finland and the US down the academy/free school route.

'I don't think anyone has tried to say the popular press does a good job at translating items like PISA into interesting news without giving them some slant that was never intended by PISA itself.'

I saw the coverage. I found it interesting. Irish coverage was interesting too.

There was less coverage of the 2009 results than those of 2000 in the UK. The 2003 results weren't counted because of issues with the sample size. This commentator liked Finland

This is a fairly accurate assessment of the misuse of PISA, except that it was the 2003 results and not the 2000 ones that were in question due to sample size.
'The Daily (hate) Mail in particular NEVER let the facts get in the way of a good rant' - one good comment under the piece.

An interesting study on media and PISA 2006 and policy, by the Univ of Edinburgh. The tone and content of coverage tended to be predictable and to follow party political lines. This particular test was the first that didn't have questions hanging over it since 2000 and it was therefore the first reliable test to show Britain's relative decline from 2000.

The 2009 test again showed stagnation and relatively better performances by other countries -- to a certain extent it was old news by that stage. It was used by Gove as noted above (i.e. to proclaim that Britain needed to follow the example of Finland and other high performers, and then to tout a policy that was poles apart from that of Finland and other high performers, instead following the example of the US).