In my opinion: PISA can help us to improve our learning system

PISA (the Programme for International Student Assessment) is a major international research project that began in 2000 and collects data from 15-year-olds at three-year intervals in a growing number of countries (65 in 2009).

Its main focus is on the measurement of the knowledge and skills that young people need for participation in society. PISA's long-term importance will probably rest on what is learned about the factors that facilitate and impede the acquisition of such knowledge and skills.

However, the aspects of PISA that typically receive most attention are those related to where countries' overall performance places them in international league tables.

Results for the most recent cycle of PISA revealed remarkably large declines in the performance of students in Ireland in reading and mathematics but not in science.

The changes in performance, especially in reading, have given rise to a good deal of negative commentary on the basis that they represent real declines in the knowledge and skills of students. There are, however, several reasons for treating such a conclusion with caution.

First, if declines in proficiency on this scale had occurred, one would expect that there would be corroborating evidence from other sources (for example, national assessments or the results of public examinations). No such evidence exists.

Secondly, some decline was to be expected because of changes in the population (and therefore, the PISA samples) between 2000 and 2009.

These changes include an increase in the numbers of young people in the system who speak neither English or Irish at home, a rise in the number of students with special needs eligible for the PISA sample and an increase in the proportion of 15-year-olds enrolled in the Transition Year Option.

Thirdly, although there have been some changes in the educational experiences of students in the various cohorts since 2000 it seems unlikely that the impact of these changes would have been substantially negative or necessarily negative at all.

Fourthly, questions have been raised, in recent work by the Educational Research Centre and Statistics Canada, about the scaling procedures used in PISA and about how change is measured over time (for example, over-reliance on a small number of test items common to all cycles).

Finally, there is some evidence for a hypothesis that students in PISA 2009 may have taken the exercise less seriously than their counterparts in previous cycles due, perhaps, to a certain amount of 'survey fatigue' and the proximity, for almost all participants, of the more 'high stakes' Junior Certificate.

Notwithstanding the reservations raised here and recalling the relatively positive findings on science, PISA has confirmed the existence of a number of challenges facing Irish education.

These include: the association between social background and achievement which, although weaker than on average across other countries, is still large; the number of students (especially boys) with low levels of reading proficiency; and the fact that students in Ireland continue to perform less well on real-life aspects of maths problem-solving than they do on other mathematical tasks.

PISA can not only help to identify challenges to our education system, it can provide us with indicators of the extent to which such challenges are being addressed. A lesson of the recent cycle of PISA is that other indicators may also be needed.

- Peter Archer, Acting director of the Educational Research Centre

Irish Independent