These days, based on schools’ aggregate standardized test scores, states brand schools as “failing.” Some school districts shut down or charterize schools that have, according to their test scores, been marked as “failing.” Some states still use their students’ test scores to constitute 50 percent of teachers’ evaluations. States publish school district report cards that award “A” grades to mostly wealthy school districts (because high test scores tend to correlate with family income) and penalize with “F” grades the poorer districts that tend to post lower overall test scores—a system that is condemning the schools in poor communities and driving segregation across metropolitan areas.
Now Sean Reardon, the Stanford University sociologist of education who is best known for tracking the resegregation of our society by income and the implications for public schools, has published a new study showing that annual standardized test scores cannot accurately measure the quality of a public school system.
Here is Reardon describing his new study: “I use standardized test scores from roughly 45 million students to construct measures of the temporal structure of educational opportunity in over 11,000 school districts—almost every district in the U.S. The data span the school years 2008-09 through 2014-15. For each school district, I construct two measures: the average academic performance of students in grade 3 and the within-cohort growth in test scores from grade 3 to 8. I argue that average test scores in a school district can be thought of as reflecting the average cumulative set of educational opportunities children in a community have had up to the time when they take a test. Given this, the average scores in grade 3 can be thought of as measures of the average extent of ‘early educational opportunities’ (reflecting opportunities from birth to age 9) available to children in a school district. Prior research suggest that these early opportunities are strongly related to the average socioeconomic resources available in children’s families in the district. They may also depend on other characteristics of the community, including neighborhood conditions, the availability of high-quality child care and pre-school programs, and the quality of schools in grades K-3.”
Test scores in grades 3-8 are, according to Reardon’s hypothesis, more reflective of the quality of the public schools: “The growth in average test scores from grades 3 to 8 can likewise be thought of as a measure of the average extent of ‘middle childhood educational opportunities’ available to children in a school district while they are roughly age 9 to 14. Given the prominence of schooling in children’s lives at these ages, these middle childhood opportunities may depend in large part on the quality of the local elementary and middle schools. They may also depend on average family resources, of course, as well as other conditions, including neighborhood characteristics and the availability of after school programs.”
Reardon continues: “I find that the two measures are largely uncorrelated; early and middle-grades opportunities appear to be distinct and separable dimensions of local educational opportunity structures… (A)lthough both dimensions of opportunity are positively associated with district socioeconomic conditions, the correlation is much weaker for the middle grades growth dimension. There are many low-income school districts with relatively high measures of growth and many affluent districts with relatively low growth.”
Reardon cautions that because the variables are enormously complex, his study cannot provide simple answers: “It is tempting to think of growth rates in test scores as a rough measure of school district effectiveness. This is neither entirely inappropriate nor entirely accurate. The growth rates better isolate the contribution to learning due to experiences during the school years… But that does not mean they reflect only the contribution of schooling. Other characteristics of communities, including family resources, after school programs, and neighborhood conditions may all affect growth in test scores independent of schools’ effects… And although poverty is systematically associated with low opportunities to learn in early childhood… poverty very clearly does not strictly determine the opportunities for children to learn in the middle grade years.”
Emphasizing that achievement gaps are likely to remain, Reardon adds: “That said, it is not clear from the patterns here that an effective school system alone can make up for low opportunities in early childhood. The large gaps in students’ academic skills between low and higher-SES (socioeconomic status) districts are so large that even the highest growth rate in the country would be insufficient to close even half of the gap by eighth grade.”
Reardon condemns the current practice in a number of states of rating and ranking schools and school districts by their students’ aggregate standardized test scores. He hopes that the conclusions of this current research will help to diminish the rapidly growing trend his own research has identified (here and here) of nearly half a century of residential resegregation by family income: “(A)ny information system that makes average test scores publicly available to parents in the hopes that a market for high test score districts will emerge and drive school improvement may instead simply create a market for high-SES districts, increasing economic segregation without improving school systems. To the extent that pubic information about school quality affects middle-and high income families’ decisions about where to live, information on growth rates might provide very different signals, perhaps leading to lower levels of economic residential and school segregation”