Test Scores Poor Indicator of Students’ Life Outcomes and School Quality: New Consensus?

According to Education Next, “Jay P. Greene is endowed chair and head of the Department of Education Reform at the University of Arkansas and a senior fellow at the Manhattan Institute.” This Arkansas “Department of Education Reform” epitomizes the far-right think tank posing as a university department. For years, Greene has been at the center of education “reform” orthodoxy, and yet today he is questioning one of its primary tenets—the use of short term test score gains as the primary measure of school improvement.  While federal policy in education as prescribed in the new Every Student Succeeds Act continues to prescribe annual testing and the requirement that states develop plans to turn around schools that can’t quickly raise scores, there seems to be growing consensus in the academic world about problems with accountability that is purely test-based.

Here is what Professor Greene posted on his blog on Tuesday:  “I’ve written several times recently about how short term gains in test scores are not associated with improved later life outcomes for students. Schools and programs that increase test scores quite often do not yield higher high school graduation or college attendance rates. Conversely, schools and programs that fail to produce greater gains in test scores sometimes produce impressive improvements in high school graduation and college attendance rates, college completion rates, and even higher employment and earnings. I’ve described at least 8 studies that show a disconnect between raising test scores and stronger later life outcomes.” Greene devotes the rest of his post to describing a new study that replicates these findings.

Greene concludes: “It’s time that people start paying a lot more attention to this pattern of a disconnect between short term test score gains and long term life outcomes. We can’t just dismiss this pattern as a fluke… If we think we can know which schools of choice are good and ought to be expanded and which are bad and ought to be closed based primarily on annual test scores gains, we are sadly mistaken.  Various portfolio management and ‘accountability’ regimes depend almost entirely on this false belief that test scores reveal which are the good and bad schools. The evidence is growing quite strong that these strategies cannot properly distinguish good from bad schools and may be inflicting great harm on students.”

Paul Tough’s new book, Helping Children Succeed, describes educational research that confirms Greene’s concerns about test-based accountability.  Tough explains the research of Kirabo Jackson: “What he found was that some teachers were reliably able to raise their students’ standardized-test scores year after year… But Jackson also found that there was another distinct cohort of teachers who were reliably able to raise their students’ performance on his non-cognitive measure.  If you were assigned to the class of a teacher in this cohort, you were more likely to show up to school, more likely to avoid suspension, more likely to move on to the next grade… Jackson found that these two groups of successful teachers did not necessarily overlap much.” (pp. 69-70)  Tough is describing teachers and schools that build intrinsic motivation and that attend to challenging students, connecting them with other students, and building autonomy. And Tough describes schools that are supportive, not punitive: “When kids feel a sense of belonging at school, when they receive the right kind of messages from an adult who believes they can succeed and who is attending to them with some degree of compassion and respect, they are then more likely to show up to class, to persevere longer at difficult tasks, and to deal more resiliently with the countless small scale setbacks and frustrations that make up the typical student’s school day.” (p. 73)

This week  the National Education Policy Center at the University of Colorado released a policy brief that cautions about condemning schools that continue to struggle rapidly to raise test scores even as they are in the midst of concerted reform and turnaround.  NEPC’s brief warns:

“Attempts to dramatically turn around schools to show quick improvements in student outcomes are often counterproductive, resulting instead in school conditions associated with persistently low performance. Many quick school turnarounds, like those initiated via the federal School Improvement Grant program, were associated with unintended, negative outcomes such as high teacher turnover, large numbers of inexperienced teachers, administrative instability, poor school and classroom climate, and socioeconomic segregation.”

NEPC instead endorses comprehensive reforms that address the multifaceted issues that are known to affect test scores: “Part of the challenge in turning around schools is that outside-of-school factors likely account for twice as much of the variance in student outcomes as do inside-of-school factors. Accordingly, the community schools approach—one of the most prominent and research-based approaches to sustained reform—addresses the academic, social-emotional, and health needs of children as well as the capacity to systemically meet these needs in communities of concentrated poverty.”

Academic reforms—beyond the social and health reforms that surround children in a wraparound Community School—also take time: “Research offers strong caution against claims of miraculous school change.  Instead, changing a school’s culture and practices in sustainable ways that improve student learning takes years of commitment by all the stakeholders in the school… Effective schools have stable leaders who support teachers…. Effective schools have teacher leadership that’s distributed through the school and that facilitates a continuous improvement cycle…. Effective schools meaningfully engage families and the community.”

The NEPC brief concludes: “Policies that demand rapid school turnaround largely ignore the complexity of reforming schools for sustainable improvement and also ignore out-of-school factors such as poverty, race, and systemic funding disparities. These mistakes arise, in part, from an imbalanced focus on test scores that can be gamed to show temporary and shallow improvements.  Instead, policies should look to a broad range of appropriate interim indicators to assess whether a school is improving.”

All this research points to the need for a radical shift in America’s domestic policy agenda. Our society will be required to reverse tax policy that has slashed public education budgets across many states and also to invest in the institutions, including public schools, that serve America’s poorest children living in concentrated and sometimes extreme poverty across our urban centers.