The National Assessment of Educational Progress (NAEP) is considered the most reliable indicator of trends in American public education. The test is administered to samples of students and is used to track long-range trends. Nobody reports on the NAEP scores of specific students. Nobody judges schools by comparative scores on NAEP. Nobody evaluates teachers based on their students’ NAEP scores. NAEP has never been part of the accountability scheme imposed by No Child Left Behind.
Diane Ravitch explains what the NAEP is: “We have only one authoritative measure of academic performance over time, and that is the National Assessment of Educational Progress, known as NAEP (pronounced ‘nape’). NAEP is part of the U.S. Department of Education. It has an independent governing board, called the National Assessment Governing Board. By statute, the governing board is bipartisan and consists of teachers, administrators, state legislators, governors, business people, and members of the general public.” (Reign of Error, p. 44)
Daniel Koretz, the Harvard expert on the construction and use of standardized tests and the way testing is distorted when the scores are used for high stakes school accountability (to compare and judge schools and teachers), explains why the NAEP scores are respected as an accurate measure of the overall trends in U.S. public schools: “NAEP… is considered a very high-quality test. NAEP scores are not susceptible to inflation because teachers aren’t held accountable for scores and therefore have no incentive to engage in NAEP-focused test prep. And NAEP scores are there for the taking. In math and reading, NAEP is administered every two years, and the scores are available to anyone on the web.” (The Testing Charade, p. 57)
The most recent NAEP scores were released in late October, for the first time since 2017. For the NY Times, Erica Green and Dana Goldstein describe the results: “America’s fourth and eighth graders are losing ground in their ability to read literature and academic texts…. The average eighth-grade reading score declined in more than half of the states compared with 2017, the last time the test was given. The average scores in fourth-grade reading declined in 17 states. Math scores remained relatively flat in most states.”
Of course, the papers have been filled with a lot of hand wringing—blaming the teachers—despairing the decline in our young peoples’ attainment. Bill Mathis believes all this misses the point. Mathis served as a design consultant for the National Assessment of Education Progress. He is currently a member of the Vermont State Board of Education and the managing director of the National Education Policy Center. Mathis recently shared his analysis in a pithy column which first appeared in the Vermont Digger. He has given me permission to reprint it here:
Put plainly, standardized tests have no meaningful relationship with economic development and they are poor definers of learning needs. Nevertheless, the NAEP is a valuable outside way of examining trends.
The scores dropped across the nation — which tells us one important thing. The causes are not found in local or state initiatives. Something bigger is at play. Since the scores themselves do not tell us why they are low, we have to look at broad contemporary events and circumstances. This means looking at the research and related social and historical events.
Such is the case with NAEP. The strongest predictor of standardized test scores is poverty.[ii] In this latest release, the biggest drops were among disadvantaged students. Sean Reardon at Stanford has compiled a data base of all school districts in the nation and found that test scores are most affected by this single construct.[iii]
He goes on to note that schools are highly segregated by class and by race. In fact, society is showing signs of resegregating.[iv] Resolving these gaps is our first threshold issue. High needs children are concentrated in high poverty schools which are, on average, less effective than schools with lower poverty. In a vicious cycle, poor schools are provided lesser resources. Compounding the problem, the Census Bureau tells us the wealth gap has sharply increased across the nation. Many schools across the nation have not recovered from the 2008 fiscal crisis and the federal government has never provided the promised support for needy children.
Regardless, the schools were mandated to solve the test score problem. The trouble was that the policymakers got it backwards. Poverty prevents learning. It is the threshold issue. Without resorting to what we knew, the dead horse was beaten once more with the No Child Left Behind Act. We adopted the Common Core curriculum, punished schools, and fired principals and teachers whose misfortune was being assigned to a school with a high concentration of needy children. It was literally expected that a child from a broken home, hungry and with ADHD would be ready to sit down and learn quadratic equations. Nevertheless, the test-based school accountability approach emerged and still remains the dominant school philosophy. While it is claimed that successful applications exist, the research has not been found that says poverty can be overcome by beating the dead horse. The irony is that the tests themselves show that a test based system is not a successful reform strategy.
Regardless of the dismal results, there is some reason to be optimistic. Policy researchers from across the spectrum agree that test based accountability has not been successful. On one end are Diane Ravitch and David Berliner who point to the lack of support provided to schools. On the opposite end of the spectrum, Michael Petrilli of the conservative Thomas B. Fordham Institute agrees. They further agree that we must attend to social and emotional learning.
We live in troubled times. We face pathological shooters, communal activities are waning, our political establishment is wobbly, and basic economic well-being is threatened. We must certainly prepare the younger generations to be ready for the workforce, and that means keeping a sufficient number of independent measures of academic achievement, geared to the needed skills of society. Yet, while we teach fundamentals, our most important obligation is to prepare all of our children to enhance the values of our heritage, guided by equality and democracy, as our paramount and universal values.
Thankfully. The public gets it. But it will not be solved by beating a dead horse.