Continued Misuse of Test Score Data to Rate and Rank Schools

Zachary Smith, the Plain Dealer‘s current data wonk, just published another article ranking Ohio’s schools, Ranking Ohio Public High Schools from 1 to 823, based on Ohio’s 2022 state school report card’s Performance Index.  A few weeks ago he ranked all of the state’s school districts by the same Performance Index.

Ohio released its annual state school report cards in mid-September and changed its ratings from A-F to a five star system.  At the time, the Plain Dealer‘s Laura Hancock reported that because the state’s schools were dealing with COVID all of last year and the federal government, therefore, eliminated demands for school improvement plans, the state would not calculate an overall summative rating for schools and school districts: “This year the Ohio Department of Education is not offering an overall rating for each school and district, due to the reprieve on sanctions. In future years, there will be an overall star rating.”

This year the stars were awarded in five categories: Achievement, Early Literacy, Graduation Rate, Progress, and Gap Closing.  A new category was added, “College, Career, Workforce and Military Readiness,” but there was were no stars assigned this year in this area due to ongoing COVID recovery.

While the state did not assign any overall summative grade for schools and school districts this year, reporter Zachary Smith discovered—in the state’s description of the category of “Achievement”—what he considers an overall way to rank the state’s schools. In its description of measuring Achievement,  the state lists a “Performance Index” number for each school and school district.  The state says it calculates the Performance Index based primarily on aggregate standardized test scores: “The Achievement component represents the number of students who passed the state tests and how well they performed on them. This component includes three additional performance indicators —the Chronic Absenteeism Improvement Indicator, End-of-Course Improvement Indicator and Gifted Indicator.” Based on Performance Index scores, Smith and the Plain Dealer have been ranking the state’s school districts and high schools.

The problem is that test scores are known to reflect a community’s family economics more than they measure the quality of a school or school district. I wish Smith would go back a couple of years to look at Rich Exner’s profound 2019 Plain Dealer article on Ohio’s school report cards. See How Closely Ohio School Report Card Grades Trend with District Income. Here is an example of one of Exner’s bar graphs, which profoundly depict the story.

Two weeks ago, Jack Schneider, a professor at the University of Massachusetts, Lowell and Joel Boyd, the superintendent of the Lowell Public Schools, explained that the correlation of standardized test scores with family income has been an issue from state to state through the past two decades since No Child Left Behind thrust us into school accountability based on standardized test scores: “As research indicates, test scores are highly indicative of the inequalities that afflict our communities, and are not a valid basis for determinations about overall school performance… Scholars have repeatedly shown (that) the leading predictors of student standardized test scores are demographic variables like family income and parental educational attainment.”

Schneider and Boyd show how test scores this year particularly are likely to reflect the disparate economic realities for families during the COVID pandemic: “Imagine that in one school community, students were insulated from the worst effects of the pandemic. Parents were able to work from home, oversee remote schooling, and offer additional support. Young people felt safe, and their families remained intact. Family resources were deployed for educational purposes and enrichment. The pandemic was a challenge, but one that was mitigated to a significant degree.  In the other school community, students felt the effects of the pandemic acutely. Family members became sick, were hospitalized, and may have even died.  Working in so-called essential fields drew caregivers away from home during the day. Internet was often slow and unreliable, and students competed for quiet space with siblings. Young people felt vulnerable, frightened, and isolated.”

The No Child Left Behind Act, formulated in 2001 and signed into law on January 8, 2002, restructured public education by demanding quantitative, standardized-test-based school accountability and by using sanctions to punish the public schools struggling to raise aggregate scores.  Gail Sunderman was one of NCLB’s early critics as the lead author of NCLB Meets School Realities, published for the Harvard Civil Rights Project in 2005, in collaboration with James Kim and Gary Orfield. Sunderman is now a research scientist in the Department of Teaching and Learning, Policy and Research and director of the Maryland Equity Project at the University of Maryland School of Public Policy.

Sunderman reminds readers that No Child Left Behind was replaced by the Every Student Succeeds Act in 2015, but the new law still requires states to rate their schools and put the lowest scoring schools on corrective action plans. Here are Sunderman’s concerns today about the damage wrought by state school rankings: “There’s… evidence that state school rating systems often reflect personal and ideological preferences of state leaders… States with a more liberal orientation… are more likely to incorporate indicators related to school quality and indicators of student success, such as growth measures, while states with a more conservative leaning maintain a focus on student test scores…  While school rating systems may be a practical means to a political end, their educational value is questionable. Despite the proliferation of school rating systems, there is very little peer-reviewed, empirical research on their effects on student performance, and school and teacher practices…  Summative ratings also tend to obscure the well-documented relationship between student achievement scores and demographic variables, most notably race and socioeconomic status. An analysis of the Maryland five-star rating system, for instance, examined why no high-poverty schools earned a five-star rating, but when the researchers adjusted ratings to account for economic disadvantage, the number of five-star schools increased.”

Sunderman concludes: This inability of summative school ratings to distinguish school performance from student demographic variances disproportionately harms schools serving marginalized children and inflates the quality of schools serving wealthy and white students.”

This blog previously covered Ohio’s 2022 school ratings here.

Advertisement

Five Reasons Why the Lawsuit Filed by Over 100 School Districts to End Ohio’s EdChoice Vouchers Is So Urgently Important

As I listened to a webinar last week about the Vouchers Hurt Ohio lawsuit filed by more than 100 school districts to demand that the court finds our state’s EdChoice Voucher program unconstitutional under the Ohio Constitution, I challenged myself to name five succinct reasons this lawsuit is so urgently important.  Here is my five point argument against Ohio’s EdChoice Vouchers.

First — EdChoice Vouchers were premised on a false assumption—that standardized test scores are a measure of school quality.

In fact aggregate standardized test scores reflect primarily our society’s family economic inequality from school district to school district. In the fall of 2019, the Plain Dealer’s data wonk, Rich Exner, created a series of bar graphs to demonstrate the almost perfect correlation of school districts’ letter grades on the state school district report card with family income.  Standardized test scores are not a good measure of school quality.

By basing voucher eligibility for students on their school’s low standardized test scores as reflected in the state school report cards, legislators who initiated the voucher program premised the idea of vouchers as an escape route for students from public schools instead of developing a strategy to support the public schools that serve our state’s poorest children.

In research summarized in a newsletter last October describing the National Education Policy Center’s new Price of Opportunity Project, NEPCs executive Director Kevin Welner explains that school achievement gaps as measured by standardized tests instead reflect larger opportunity gaps:  “Those of us who work in or with schools never question the enormous impact that a teacher or school can have on a student. But this essential truth coexists with another truth: that differences between schools account for a relatively small portion of measured outcome differences. That is, opportunity gaps in the U.S arise primarily outside of schools. This should not be a surprise. Poverty, concentrated poverty, and racialized poverty are pervasive features of America. School improvement efforts cannot directly help children and their families overcome decades of policies that perpetuate systemic racism and economic inequality. When children are born in the United States, their educational and life outcomes can all be predicted based on their parents’ education, income and wealth. Compared to the Scandinavian countries and other so-called Western democracies like Canada, Spain, Australia, and New Zealand, American children are inordinately trapped in intergenerational poverty. Inequality in the U.S. is stark and enduring.”

Second — Ohio school districts serving masses of poor students lose urgently needed school funding to vouchers. The students who remain in district public schools suffer program losses when other students extract essential school funding to pay private school tuition.

In Ohio, EdChoice Vouchers, funded by school district deduction, have taken money year after year from the budgets of local school districts serving masses of Ohio’s poorest children. Until last summer’s budget bill, the Legislature made the school districts themselves pay for the vouchers out of their local budgets and therefore increase class size, or cut the number of counselors and school nurses, or shutter the school library in the public school.  At last week’s Vouchers Hurt Ohio webinar, Cleveland Heights-University Heights school board member, Dan Heintz, explained that of the 22 Ohio school districts most severely penalized by the EdChoice Voucher program, all 22 are majority poor, and 19 are majority African American. In these school districts which desperately need increased services for their students, the loss of school funding to vouchers has pared down programming for the students who remain in the public schools.

Third — Many school districts lose more money to vouchers than they receive for voucher students in state foundation basic aid.

The state has awarded to the local school districts basic foundation aid for each of the voucher students, but the amount of each voucher in many cases has significantly exceeded the amount of per-pupil state aid coming to the district. In many of these districts which primarily serve students in poverty, the state has been extracting more money for each voucher student to carry out of the school district budget than it spends on each student who remains in the district’s public schools.  Many of the students taking vouchers out of the school district budget have never been enrolled in the district’s public schools and have always attended private schools. Heintz reminded viewers that in his district, 95 percent of students taking vouchers have never been enrolled in the school district.

The lawsuit, as filed, delineates the losses in public school funding for Heintz’s district, one of the plaintiff districts: “The Cleveland Heights-University Heights City School District… is expected to receive from the state of Ohio a total of approximately $5.6 million in foundation funding for Fiscal Year 2022 to educate the 5,000 students who attend its schools. The state of Ohio, however, will pay out over $11 million for private school tuition to the approximately 1,800 EdChoice Voucher recipients residing within the Cleveland Heights-University Heights City School District in Fiscal Year 2022. In other words, approximately twice as much public funding will be paid in Fiscal Year 2022 for private school tuition for CH-UH residents as the foundation funding allotted to the entire student body of the Cleveland Heights-University Heights District.”

Fourth — EdChoice Vouchers have exacerbated white flight.

In a number of school districts, EdChoice vouchers have accelerated racial segregation, as white students have taken the vouchers to attend private schools.  At the press conference where the lawsuit was announced, a member of the Richmond Heights Local Schools Board of Education, Nneka Jackson explained how, in her Cleveland suburban school district, EdChoice Vouchers have increased racial isolation: “If someone tells you this is about helping poor minority children, hook them up to a lie detector test ASAP and stand back because the sparks are going to fly… About 40 percent of Richmond Heights residents are white. Before the EdChoice private school voucher program, about 26 percent of the students in the Richmond Heights School District were white and 74 percent were students of color. Today, after EDChoice, Richmond Heights is three percent white and 97 percent students of color. Private schools are allowed to discriminate, plain and simple, based on disability, disciplinary records, academic standing, religion and financial status. These are often proxies for race and other protected characteristics. Ohio is essentially engaged in state-sponsored discrimination in admissions and retention. You know who can’t do this? Public schools. Common schools.”

Fifth — By expanding the EdChoice Voucher program in the FY2022-2023 state budget, the legislature made it impossible to fully phase in the new Fair School Funding plan it passed in the same state budget.

The state ended school district deduction finding for EdChoice Vouchers in the FY 2022-FY2023 state budget and will fund the vouchers out of the state’s public school foundation budget.  In the very same the FY 2022-FY2023 state budget, the legislature passed a new Fair School Funding Plan. By expanding EdChoice Vouchers, the state has rendered itself unable to fully phase in the Fair School Funding Plan.

The lawsuit declares: “Under House Bill 110, in the state budget bill recently passed by the General Assembly, effective June 30, 2021, the EdChoice Program has been greatly expanded both in terms of eligibility for the EdChoice Program and its scope. Indeed, HB 110 eliminates previous limits on the number of EdChoice Program vouchers the Ohio Department of Education can approve… Because public funds are finite, funding EdChoice Program Vouchers out of the foundation funding designated for public school districts inevitably depletes the resources designated by the legislature for educating Ohio’s public school students… HB 110 initially incorporated the salient features of the… Fair School Funding Plan, a bipartisan effort to fund Ohio’s public schools adequately and equitably as required by the Ohio Supreme Court in DeRolph v. State…. However, due to the ballooning effects of the EdChoice Program, the enacted version of HB 110 funded only up to one-third of the increases required by the proposed Fair School Funding Plan over the next two fiscal years.”

The Vouchers Hurt Ohio lawsuit challenges a state program that has exacerbated systemic racial and economic injustice.