Continued Misuse of Test Score Data to Rate and Rank Schools

Zachary Smith, the Plain Dealer‘s current data wonk, just published another article ranking Ohio’s schools, Ranking Ohio Public High Schools from 1 to 823, based on Ohio’s 2022 state school report card’s Performance Index.  A few weeks ago he ranked all of the state’s school districts by the same Performance Index.

Ohio released its annual state school report cards in mid-September and changed its ratings from A-F to a five star system.  At the time, the Plain Dealer‘s Laura Hancock reported that because the state’s schools were dealing with COVID all of last year and the federal government, therefore, eliminated demands for school improvement plans, the state would not calculate an overall summative rating for schools and school districts: “This year the Ohio Department of Education is not offering an overall rating for each school and district, due to the reprieve on sanctions. In future years, there will be an overall star rating.”

This year the stars were awarded in five categories: Achievement, Early Literacy, Graduation Rate, Progress, and Gap Closing.  A new category was added, “College, Career, Workforce and Military Readiness,” but there was were no stars assigned this year in this area due to ongoing COVID recovery.

While the state did not assign any overall summative grade for schools and school districts this year, reporter Zachary Smith discovered—in the state’s description of the category of “Achievement”—what he considers an overall way to rank the state’s schools. In its description of measuring Achievement,  the state lists a “Performance Index” number for each school and school district.  The state says it calculates the Performance Index based primarily on aggregate standardized test scores: “The Achievement component represents the number of students who passed the state tests and how well they performed on them. This component includes three additional performance indicators —the Chronic Absenteeism Improvement Indicator, End-of-Course Improvement Indicator and Gifted Indicator.” Based on Performance Index scores, Smith and the Plain Dealer have been ranking the state’s school districts and high schools.

The problem is that test scores are known to reflect a community’s family economics more than they measure the quality of a school or school district. I wish Smith would go back a couple of years to look at Rich Exner’s profound 2019 Plain Dealer article on Ohio’s school report cards. See How Closely Ohio School Report Card Grades Trend with District Income. Here is an example of one of Exner’s bar graphs, which profoundly depict the story.

Two weeks ago, Jack Schneider, a professor at the University of Massachusetts, Lowell and Joel Boyd, the superintendent of the Lowell Public Schools, explained that the correlation of standardized test scores with family income has been an issue from state to state through the past two decades since No Child Left Behind thrust us into school accountability based on standardized test scores: “As research indicates, test scores are highly indicative of the inequalities that afflict our communities, and are not a valid basis for determinations about overall school performance… Scholars have repeatedly shown (that) the leading predictors of student standardized test scores are demographic variables like family income and parental educational attainment.”

Schneider and Boyd show how test scores this year particularly are likely to reflect the disparate economic realities for families during the COVID pandemic: “Imagine that in one school community, students were insulated from the worst effects of the pandemic. Parents were able to work from home, oversee remote schooling, and offer additional support. Young people felt safe, and their families remained intact. Family resources were deployed for educational purposes and enrichment. The pandemic was a challenge, but one that was mitigated to a significant degree.  In the other school community, students felt the effects of the pandemic acutely. Family members became sick, were hospitalized, and may have even died.  Working in so-called essential fields drew caregivers away from home during the day. Internet was often slow and unreliable, and students competed for quiet space with siblings. Young people felt vulnerable, frightened, and isolated.”

The No Child Left Behind Act, formulated in 2001 and signed into law on January 8, 2002, restructured public education by demanding quantitative, standardized-test-based school accountability and by using sanctions to punish the public schools struggling to raise aggregate scores.  Gail Sunderman was one of NCLB’s early critics as the lead author of NCLB Meets School Realities, published for the Harvard Civil Rights Project in 2005, in collaboration with James Kim and Gary Orfield. Sunderman is now a research scientist in the Department of Teaching and Learning, Policy and Research and director of the Maryland Equity Project at the University of Maryland School of Public Policy.

Sunderman reminds readers that No Child Left Behind was replaced by the Every Student Succeeds Act in 2015, but the new law still requires states to rate their schools and put the lowest scoring schools on corrective action plans. Here are Sunderman’s concerns today about the damage wrought by state school rankings: “There’s… evidence that state school rating systems often reflect personal and ideological preferences of state leaders… States with a more liberal orientation… are more likely to incorporate indicators related to school quality and indicators of student success, such as growth measures, while states with a more conservative leaning maintain a focus on student test scores…  While school rating systems may be a practical means to a political end, their educational value is questionable. Despite the proliferation of school rating systems, there is very little peer-reviewed, empirical research on their effects on student performance, and school and teacher practices…  Summative ratings also tend to obscure the well-documented relationship between student achievement scores and demographic variables, most notably race and socioeconomic status. An analysis of the Maryland five-star rating system, for instance, examined why no high-poverty schools earned a five-star rating, but when the researchers adjusted ratings to account for economic disadvantage, the number of five-star schools increased.”

Sunderman concludes: This inability of summative school ratings to distinguish school performance from student demographic variances disproportionately harms schools serving marginalized children and inflates the quality of schools serving wealthy and white students.”

This blog previously covered Ohio’s 2022 school ratings here.

Advertisement

Money Follows Child in Ohio Budget—Cleveland Public Schools Learn They Lose Millions

All spring through the 2014-2015 biennial Ohio budget debate in Columbus, the legislature was provided printouts of the implications of the budget for the state’s 631 school districts.  The only catch is that the printouts counted charter schools as part of their public school districts for budgeting purposes.  Nobody—no legislator, no school superintendent, no school board member, no citizen—could tell how much money the public schools in any school district would receive once money followed some children to charter schools.

Critics questioned whether there might not be school districts that appeared in the printouts to benefit from additional state aid or at least stay even but would actually lose state funding when the money for the charters was broken out.  Computer runs that would reveal the truth did not appear before the budget was passed by the legislature and signed in to law by Ohio’s Governor John Kasich on June 30.

Months later and a couple of weeks into this school year, the Ohio Legislative Services Commission released the data.  The September 6 Plain Dealer shared the news:  “Estimates for how much the state would deduct from districts for students attending charter schools were not available when the budget passed June 29….  The most dramatic case those estimates reveal is the Cleveland school district, which has no aid increase from the 2012-2013 school year to the current school year under the budget, but much higher deductions for charter students.  Depending on how you calculate it, the district will end up with $3 million to $4.5 million less for its students, after the state deducts a greater share for charter schools.”  In other words Cleveland, one of the nation’s poorest big city school districts, had been told its state aid would stay even in the upcoming budget cycle, only to learn this week that it will lose millions of dollars it had counted on.  And this after Cleveland voters passed a 15 mill levy last November to replace the millions cut by the state in the 2012-2013 budget.

There are ample reasons to be concerned about the emergence and growth of Ohio charter schools.  A recent report by the Columbus Dispatch describes Charter Schools’ Failed Promise. Our state is reputed to be among the weakest in its regulatory oversight of charters, with many earning state-issued grades of D or F on the report cards issued by the Ohio Department of Education.  To his credit, Cleveland Mayor Frank Jackson tried to create a Cleveland Transformation Alliance with the power to authorize only quality charters and to put the rest out of business, but it turned out that the legislative fine print denies the Transformation Alliance any real power to regulate Cleveland’s charters.

Of additional concern, however, is the allegation made by reporter Stephanie Simon in a Reuters investigative report last February, Class Struggle–How Charter Schools Get Students They Want, that one of Cleveland’s top-rated charters is controlling its test scores by selecting its students.  According to Simon,  a boy at the top of the waiting list for the Intergenerational Charter School was required to take a two-hour entrance examination.  The principal  then told the mother the child “wasn’t academically or developmentally ready for third-grade–even though he was enrolled in the third grade at his local public school, where he remains.”  Simon continues: “A spokesman for the Ohio Department of Education said charter schools are obligated to admit students into the grade they would attend at their neighborhood school, regardless of skill.”  Simon continues: “The community authorizer that supervises Intergenerational Charter said that it is confident the school’s admissions policy is legal but that it will review the policy.”

Simon’s article describes the many ways charter schools across the United States cream-skim the most promising students and those whose parents bring the most savvy to the application process; she alleges that the sometimes subtle ways charters select students leave behind students with special needs, English language learners, and homeless and other children living in extreme poverty.  These students are the most expensive to educate  In Cleveland this year we see the state budget punishing the public school district which is required to serve all children.

In a fascinating analogy, former middle school life sciences teacher Anthony Cody blogged last winter that charter schools exist as organisms in a symbiotic arrangement.  He warns that parasitism, in which one set of organisms are “helped at the expense of the other,” cannot survive if the parasites kill off their hosts.

White Hat Charter Scandal Continues in Ohio

This week, for the first time in my memory, the Cleveland Plain Dealer reports extensively on the scandal of White Hat Management, the charter school operation owned by powerful Akron political contributor, David Brennan. In Funds are at Issue in Charter School Suit the newspaper reports on a case that has now been appealed to the Ohio Supreme Court.

Several White Hat schools, trying to pull out of the network, sued White Hat in 2010 to find out what equipment belongs to the management company and what belongs to the school. The plaintiff schools won at the trial court and appellate level, but White Hat continues to appeal the case. At issue is that White Hat, a privately held corporation, is not required to explain how much it spends on salaries or equipment. Not even to its own appointed school boards or to the staffs at its schools.

What the Plain Dealer does not point out this morning is that the public’s interest is a little different than the interests of the boards of the schools trying to leave the network.

White Hat is known to take 96 percent of the state revenue provided for the schools, but White Hat is not required to document publicly how much goes into educating the children and how much is taken as profit by owner David Brennan.

The attorney for White Hat, Charles “Rocky” Saxbe is reported in today’s paper as explaining “that the company already has provided as much information as state law and the Ohio auditor’s office require.” That is of course the problem: the state of Ohio has failed its obligation to protect the public from waste and to protect children from low performing schools. Charter school boards appointed by Charter Management Organizations are not in a position to provide adequate oversight over the corporations that appoint them.

The other problem, as the Akron Beacon Journal reported on June 29 is that David Brennan, a huge political contributor, got a boost of $1,400 per child in Basic Aid in the biennial budget that our governor signed into law on June 30. This is far higher than the increase in state aid for traditional public school districts or for other charter schools, even those posting far higher test scores than Brennan’s schools.

In a 2010 pastoral letter sent to the President and Congress, the governing board of the National Council of Churches wrote: “We believe that democratic operation of public schools is our best hope for ensuring that families can secure the services to which their children have a right. On balance, we believe that if government invests public funds in charter schools that report to private boards, government, not the vicissitudes of the marketplace, should be expected to provide oversight to protect the common good.”

For more about troubles in Ohio’s charter schools, read this new report from Innovation Ohio: Unfair Funding: How Charter Schools Win & Traditional Schools Lose.