Don’t make a mountain out of PISA scores

Education reformers increasingly point to one piece of hard evidence that American schools are failing and in need of shock therapy: PISA scores.

The Program for International Student Assessment (PISA) is a standardized test administered by the Organization for Economic Cooperation and Development (OECD). Every three years, a representative sample of 15-year-olds takes the test in approximately 70 countries. The PISA covers reading, math, science and problem solving; the focus (PDF) of the 2012 survey was on the capacity to use mathematical concepts in real world contexts. The ostensible purpose of PISA is to enable policymakers to gauge how students in their countries are acquiring cognitive skills and to identify high-performing educational systems that may offer useful policy lessons.

The 2012 PISA results for the United States, according to U.S. Secretary of Education Arne Duncan, reveal “a picture of educational stagnation” that justifies the need for reforms such as the Common Core educational standards, national tests, value-added teacher evaluations and charter schools. Other education reformers — including Joel Klein, a former chancellor of New York City schools; Michael Barber, chief education officer of Pearson; and former Florida Gov. Jeb Bush — have issued similar warnings about the United States’ lackluster PISA scores.

While U.S. policymakers should care about the quality of education for American students, they shouldn’t place too much value on these scores. In failing to account for factors outside school and by intensifying a testing mentality that educators worldwide increasingly recognize as harmful, the PISA can lead to poor policy recommendations with long-term consequences. 

PISA, of course, has its share of supporters. In their book “Endangering Prosperity: A Global View of the American School,” Stanford economist Eric Hanushek, Harvard political scientist Paul E. Peterson and German economist Ludger Woessmann make the case for Americans to take the PISA results as seriously as other countries do.

Cognitive human capital, their argument goes, helps determine long-term economic prosperity. And according to them, a math test of 15-year-olds provides a reliable indicator of this capital. They write, “Math appears to be the subject in which accomplishment in secondary school is particularly significant for both an individual’s and a country’s future economic well-being.” Furthermore, math is a subject that appears well suited to cross-country comparisons.

On the basis of these scores, advocates argue that American students are performing unsatisfactorily. In 2009 just 32 percent of eighth-graders in the U.S. were deemed proficient in mathematics, and only 7 percent performed at an advanced level. The U.S. is 18th in advanced math achievement, just ahead of the United Kingdom, Italy, Russia, Latvia, Croatia and Kazakhstan. According to the authors of “Endangering Prosperity,” this is proof that the U.S. suffers from an “educational malaise” that will be “extremely costly both for the next generation and for the country as a whole.”

But there are problems with what PISA claims to measure and the kinds of policies it supports.

PISA advocates often exaggerate the usefulness of data provided by a test of 15-year-olds. For instance, “Endangering Prosperity” projects economic growth rates based on an improvement in PISA test scores. For example, if the United States matches Singapore’s score, the average American will earn approximately $300,000 in the year 2085, whereas if the United States reaches only Germany’s score, then the average American will earn $150,000. Such projections make too simple assumptions about eighth-grade test proficiency leading to economic growth and the sustainability of such growth. It also fails to explain how the United States, with its mediocre test scores, has prevailed in the global economy. 

“Endangering Prosperity” also neglects to consider the influence of factors outside school, such as student poverty or wealth. The authors explain that a country’s education system has the primary public responsibility for facilitating the acquisition of knowledge and skills. This may be partly true, but according to the OECD, socioeconomically advantaged children perform, on average, the equivalent of one year of formal schooling better than other children. The problem with the United States’ PISA scores may be poverty, not the educational system.

Also, the PISA framework can shift educational priorities in a negative way.

The test’s advocates offer policy proposals under the banner of education reform. In regards to teacher policy, this means merit pay, the elimination of teacher tenure and the weakening of teachers’ unions. In regards to school choice, it means vouchers, charter schools and online courses. And in regards to accountability, it means support for annual testing, education standards and tests for grade promotion or graduation.

But such proposals can unduly burden a country, which must often redesign its whole system to prepare students to excel on standardized tests. The U.S. began this process with the No Child Left Behind Act and has accelerated it with President Barack Obama’s Race to the Top initiative. As NPR education reporter Anya Kamenetz documents in her new book, “The Test: Why Our Schools Are Obsessed With Standardized Testing — But You Don’t Have to Be,” high-stakes standardized tests over the past decade appear to have stunted children’s spirit, demoralized teachers, narrowed discussions about education reform to improving scores and dampened creativity that can lead to economic innovation. “The way much of school is organized around these tests makes little sense for young humans developmentally,” she writes. “Nor does it square with what the world needs.”

Furthermore, PISA measures — and thereby incentivizes the teaching of — only a small fraction of what contributes to a meaningful life. Professors Heinz-Dieter Meyer and Katie Zahedi elaborate this point in an open letter to PISA head Andreas Schleicher. “PISA takes attention away from the less measurable or immeasurable educational objectives like physical, moral, civic and artistic development, thereby dangerously narrowing our collective imagination regarding what education is and ought to be,” they write. They also worry about the influence of educational corporations affiliated with PISA that may prioritize making a profit over educational or democratic considerations.

Education reformers seem to want to outdo one another in warning what will happen if the United States does not improve its PISA scores. But such alarmist rhetoric can cloud judgment and lead policymakers to invest limited financial resources in testing rather than education, in designing an educational system geared toward standardized exams rather than nurturing the talents and interests of children.

America’s goal should be to provide all children the type of education provided by the finest private schools, with qualified teachers, small classes, sports programs and a curriculum that includes languages, math, science, history and the arts. This, more than pushing test scores into the spotlight, will help us out of our educational malaise.