Continued Misuse of Test Score Data to Rate and Rank Schools

Zachary Smith, the Plain Dealer‘s current data wonk, just published another article ranking Ohio’s schools, Ranking Ohio Public High Schools from 1 to 823, based on Ohio’s 2022 state school report card’s Performance Index.  A few weeks ago he ranked all of the state’s school districts by the same Performance Index.

Ohio released its annual state school report cards in mid-September and changed its ratings from A-F to a five star system.  At the time, the Plain Dealer‘s Laura Hancock reported that because the state’s schools were dealing with COVID all of last year and the federal government, therefore, eliminated demands for school improvement plans, the state would not calculate an overall summative rating for schools and school districts: “This year the Ohio Department of Education is not offering an overall rating for each school and district, due to the reprieve on sanctions. In future years, there will be an overall star rating.”

This year the stars were awarded in five categories: Achievement, Early Literacy, Graduation Rate, Progress, and Gap Closing.  A new category was added, “College, Career, Workforce and Military Readiness,” but there was were no stars assigned this year in this area due to ongoing COVID recovery.

While the state did not assign any overall summative grade for schools and school districts this year, reporter Zachary Smith discovered—in the state’s description of the category of “Achievement”what he considers an overall way to rank the state’s schools. In its description of measuring Achievement,  the state lists a “Performance Index” number for each school and school district.  The state says it calculates the Performance Index based primarily on aggregate standardized test scores: “The Achievement component represents the number of students who passed the state tests and how well they performed on them. This component includes three additional performance indicators —the Chronic Absenteeism Improvement Indicator, End-of-Course Improvement Indicator and Gifted Indicator.” Based on Performance Index scores, Smith and the Plain Dealer have been ranking the state’s school districts and high schools.

The problem is that test scores are known to reflect a community’s family economics more than they measure the quality of a school or school district. I wish Smith would go back a couple of years to look at Rich Exner’s profound 2019 Plain Dealer article on Ohio’s school report cards. See How Closely Ohio School Report Card Grades Trend with District Income. Here is an example of one of Exner’s bar graphs, which profoundly depict the story.

Two weeks ago, Jack Schneider, a professor at the University of Massachusetts, Lowell and Joel Boyd, the superintendent of the Lowell Public Schools, explained that the correlation of standardized test scores with family income has been an issue from state to state through the past two decades since No Child Left Behind thrust us into school accountability based on standardized test scores: “As research indicates, test scores are highly indicative of the inequalities that afflict our communities, and are not a valid basis for determinations about overall school performance… Scholars have repeatedly shown (that) the leading predictors of student standardized test scores are demographic variables like family income and parental educational attainment.”

Schneider and Boyd show how test scores this year particularly are likely to reflect the disparate economic realities for families during the COVID pandemic: “Imagine that in one school community, students were insulated from the worst effects of the pandemic. Parents were able to work from home, oversee remote schooling, and offer additional support. Young people felt safe, and their families remained intact. Family resources were deployed for educational purposes and enrichment. The pandemic was a challenge, but one that was mitigated to a significant degree.  In the other school community, students felt the effects of the pandemic acutely. Family members became sick, were hospitalized, and may have even died.  Working in so-called essential fields drew caregivers away from home during the day. Internet was often slow and unreliable, and students competed for quiet space with siblings. Young people felt vulnerable, frightened, and isolated.”

The No Child Left Behind Act, formulated in 2001 and signed into law on January 8, 2002, restructured public education by demanding quantitative, standardized-test-based school accountability and by using sanctions to punish the public schools struggling to raise aggregate scores.  Gail Sunderman was one of NCLB’s early critics as the lead author of NCLB Meets School Realities, published for the Harvard Civil Rights Project in 2005, in collaboration with James Kim and Gary Orfield. Sunderman is now a research scientist in the Department of Teaching and Learning, Policy and Research and director of the Maryland Equity Project at the University of Maryland School of Public Policy.

Sunderman reminds readers that No Child Left Behind was replaced by the Every Student Succeeds Act in 2015, but the new law still requires states to rate their schools and put the lowest scoring schools on corrective action plans. Here are Sunderman’s concerns today about the damage wrought by state school rankings: “There’s… evidence that state school rating systems often reflect personal and ideological preferences of state leaders… States with a more liberal orientation… are more likely to incorporate indicators related to school quality and indicators of student success, such as growth measures, while states with a more conservative leaning maintain a focus on student test scores…  While school rating systems may be a practical means to a political end, their educational value is questionable. Despite the proliferation of school rating systems, there is very little peer-reviewed, empirical research on their effects on student performance, and school and teacher practices…  Summative ratings also tend to obscure the well-documented relationship between student achievement scores and demographic variables, most notably race and socioeconomic status. An analysis of the Maryland five-star rating system, for instance, examined why no high-poverty schools earned a five-star rating, but when the researchers adjusted ratings to account for economic disadvantage, the number of five-star schools increased.”

Sunderman concludes: This inability of summative school ratings to distinguish school performance from student demographic variances disproportionately harms schools serving marginalized children and inflates the quality of schools serving wealthy and white students.”

This blog previously covered Ohio’s 2022 school ratings here.

Advertisement

New Reports Confirm Persistent Child Poverty While Policymakers Blame Educators and Fail to Address Core Problem

On Tuesday, the Cleveland Plain Dealer published a stunning analysis, by the newspaper’s data analyst Rich Exner, of the school district grades awarded by the state of Ohio on the state report cards released last week.  The new report cards are based on data from the 2018-2019 school year.  I encourage you to follow the link to look at Exner’s series of bar graphs, which, like this one, present a series of almost perfect downward staircases, with “A” grades for school districts in communities with high median income and “F” grades for the school districts in Ohio’s poorest communities.

The correlation of academic achievement with family income has been demonstrated now for half a century, but policymakers, like those in the Ohio legislature who are debating punitive school district takeovers, prefer to blame public school teachers and administrators instead of using the resources of government to assist struggling families who need better access to healthcare, quality childcare, better jobs, and food assistance.

Ohio’s school district grades arrived this week. At the same time, and with less fanfare, arrived a series of reports on the level of federal spending on children, reports documenting that, as Education Week‘s Andrew Uifusa explains: “The share of the federal budget that goes toward children, including education spending, dipped to just below 2 percent of the nation’s gross domestic product in 2018—the lowest level in the decade.”

On Tuesday of this week, the Urban Institute released a new report detailing trends in federal spending on children’s needs: “(O)verall spending on children represents a relatively small share of total federal spending, and that share is dwindling. In 2018, overall federal spending on children younger than 19 fell from recent years to about $6,200 per child.  Education and other discretionary spending categories saw the steepest declines last year, as they were squeezed by growing spending on health and retirement programs, as well as interest payments on the national debt.”  Further, federal spending on children is growing thin in particular areas as children’s needs compete with one another: “Increased mandatory spending on health programs for children and adults is putting pressure on education spending and other discretionary spending on kids. In 2018, federal spending on education dropped by $1.9 billion.  This is part of a long-term trend, as 2018 federal spending on elementary and secondary education was 48 percent below peak spending during the recession (in 2010) and 14 percent below pre-recession spending (in 2008).”  “As spending exceeds revenues year after year, the national debt will continue to climb… Under current policies, interest payments on the debt are projected to exceed spending on children in the next few years.”

A new report this month from the Center for Law and Social Policy (CLASP), Children and Families in Trouble, examines the persistence of child poverty and the federal government’s failure to address it: “Poverty in the United States continued a sluggish decline in 2018, falling to 11.8 percent, with children and young adults still experiencing the highest rates.  Child poverty (ages 0-18) and young adult poverty (ages 18-24) remained unacceptably high at 16.2 percent and 15 percent respectively with alarmingly large racial and ethnic disparities in poverty.  Young children, under age 5, remain the poorest of all, at 17.7 percent….”  “Racial disparities are persistent, stark, and caused by structural factors… Black and Hispanic children are more likely to be poor (29.5 and 23.7 percent respectively) compared to 8.9 percent of non-Hispanic white children, despite high levels of work among their families.”

CLASP reports relatively high levels of employment among families with poor children, but problems with the kind of work available, the wages, and the conditions: “More than two-thirds of poor children (70.3 percent) live in households with at least one worker. Low wages, inadequate hours, and underemployment mean that work still does not pay a family-sustaining wage for millions of households. While unemployment remains near historical lows, a substantial share of low-income workers is employed part time involuntarily, meaning they would prefer to be working full time but are unable to find full-time work or get sufficient hours from their employer. Low-wage jobs predominate in the fastest-growing sectors, such as retail and food service. Such jobs are characterized by few benefits; unstable and unpredicable schedules; and temporary or part-time status.”

In the 13th annual release, last week, of its proposed Children’s Budget, First Focus on Children summarizes several areas in which Congress needs to support children with increased spending:

  • “Almost 80 percent of eligible 3-5 year old children lack access to Head Start programs.
  • “The Federal Government is not fulfilling 55 percent of its funding commitment for Individuals with Disabilities in Education Act (IDEA) grants.
  • “Of the households on the waiting list for housing assistance, 60 percent are families with children.
  • “75 percent of poor families in the U.S. who are eligible for cash assistance do not receive it.
  • “Nearly 83 percent of children who receive free or reduced price lunch during the school year do not have access to the summer meals program.”

The Trump administration has now also proposed a new “public charge rule” which would eventually deny green cards and application for citizenship to members of immigrant families who use public benefits. The new rule will apply in the future to the possible citizenship of today’s infants and children in these families.  In its recent report CLASP highlights special problems for immigrant children if, at the end of a 60 day posting period, the rule goes into effect (on October 15, 2019): “Among children, 425,000 more were uninsured in 2018 versus 2017, reversing a decades-long trend toward greater coverage. This concerning reversal, including a significant worsening among Hispanic children and among young children… likely reflects multiple attacks on health insurance coverage for people with low incomes. Notably, the Trump Administration is waging ongoing efforts to undermine the ACA and Medicaid access, and a hateful anti-immigrant agenda… (is) causing a chilling effect on immigrant families’ access to public programs.”

In late August, the National Education Policy Center (NEPC) highlighted “Six Ways Trump ‘Public Benefits’ Policies Harm Children.” NEPC’s newsletter examines how the Trump administration’s proposed new rule would constrain opportunity for children in vulnerable immigrant families: “On August 12th the Trump administration proposed a new rule to change the criteria considered when the U.S. government decides whether to extend visas or grant permanent residency (‘green cards’).  These criteria—which are inextricably tied to a history of bias in the immigration process—have long included evidence about the likelihood of the immigrant becoming dependent on public benefits. But the approach that is now used focuses on cash benefits, such as Supplemental Social Security (‘disability’) or Temporary Assistance for Needy Families (‘welfare’).  The proposed rule will expand that to the main non-cash benefits used by immigrants: the Supplemental Nutrition Assistance Program (SNAP), or food stamps; Medicaid; and housing vouchers and other housing subsidies.”  NEPC continues: “(Seventeen) states plus DC have brought two lawsuits against the administration, alleging that the rule redefines the term ‘public charge’ inconsistently with Congress’ intent in the Immigration and Nationality Act; that it violates constitutional equal protection guarantees by effectively targeting immigrants from poorer areas in Asia, Latin America, and Africa; that it infringes on states’ rights to protect their own residents; and that it punitively, arbitrarily and capriciously targets immigrants for using public benefits programs that are used by about half the country’s residents.” While school breakfast and lunch programs are not directly affected, “current policy automatically enrolls students in the federal free and reduced-price school meal program if their families receive food stamps… Accordingly, if immigrant families avoid SNAP, (their children) are less likely to receive the meals.”

My reason for quoting all of this information about persistent child poverty is to make the needs of America’s poorest children visible. The bar graph produced by the Plain Dealer‘s Rich Exner clearly shows that child poverty affects academic achievement. Policy makers, however, in the spirit of test-based, sanctions-based school accountability, are instead determined to impose punishments on the school districts serving poor children. They imagine that if they shift the blame onto teachers, nobody will notice that they are themselves failing to invest the resources and power of government in programs to support the needs of America’s poorest children.