In mid-August, this blog posed the following question: “How has standardized, test-based school accountability changed the way we understand public schooling?” Here is how that post answered the question: “The most basic critique of accountability-based school reform is that its frame is entirely quantitative. School accountability based solely on aggregate standardized test scores fails to measure the qualitative process of education as experienced by students and practiced by teachers.”
Despite the growing critique of high stakes, test-based school accountability, the 2015 federal Every Student Succeeds Act, which replaced No Child Left Behind, requires states to rate their schools and intervene in the schools where aggregate student test scores have not risen significantly. For years, Ohio has assigned “A” through “F” letter grades for each school and school district, based primarily on standardized test scores. Last Thursday, however, Ohio released a new state report card evaluating each of the state’s 610 school districts and each individual public school by substituting a five-star system for the letter grades.
Because the U.S. Department of Education has relaxed—during and immediately following the pandemic—the demand that states develop correction plans to turn around the lowest scoring schools, Ohio will grant schools another year of the pandemic-driven reprieve on the imposition of state-imposed improvement plans. Neither will the state aggregate the school ratings into one overall summative score for each school and school district during this year. The Plain Dealer’s Laura Hancock reports, “This year the Ohio Department of Education is not offering an overall rating for each school and district, due to the reprieve on sanctions. In future years, there will be an overall star rating.”
A new five-star rating system is the key change this year. Ohio has rated schools with up to five stars in six categories. According to Hancock, “The Ohio Department of Education created a 17-page guide to understanding the report cards, which shows how stars will be used….” Here are the six categories on which schools are being rated and the method for computing the rating, according to Hancock:
- “Achievement: This component represents whether student performance on state tests met previously established thresholds. It also considers how well students performed on tests overall…
- “Early Literacy: This area measures reading improvement and overall proficiency scores for students in kindergarten through third grade.
- “Graduation: This measurement looks at the four-year adjusted cohort graduation rate and the five-year cohort graduation rate…
- “Progress: This measurement looks at the growth students are making based on their past performances…
- “Gap Closing: This component measures the reduction in educational gaps for student subgroups based on income, race, ethnicity, or disability.
- “College, Career, Workforce and Military Readiness: This component looks at how well prepared Ohio’s students are for future opportunities, whether training in a technical field or preparing for work or college. This is a relatively new measurement and the full data won’t be completely reported until the 2024-2025 school year. There also won’t be a star ranking for this area.”
In its new rating system, the Ohio Department of Education leaves in place a system based on the assumption that school quality can be measured accurately and summarized with a quantitative methodology. Interestingly, four of the six categories in Ohio’s new system depend on a school’s or a school district’s aggregate test scores, which have for years been highly correlated with a school population’s overall family income. The new five-star method is assumed to be better than the assignment of letter grades even in our age of emojis, where people are quite comfortable with inferring a clear meaning from a visual display of symbols like stars.
A serious problem with the new ratings is that it is utterly unclear whether and how the ratings in any way measure what educators are doing differently from district to district. I looked at the 17-page guide to interpreting the scores. In the section describing the “Early Literacy” measure, the guide explains: “The Early Literacy Component measures reading improvement and proficiency for students in kindergarten through third grade.” It is based on students’ third-grade “Language Arts Proficiency” test score, how many students are promoted to fourth grade, and “two consecutive years of data to evaluate how well schools and districts are doing at providing supports needed to help struggling readers become on track with their reading.”
The 17-page guide does not acknowledge the research of Sean Reardon, the Stanford University educational sociologist, who comments on the opportunity gaps that come to school with children as they enter Kindergarten: “We examine… test score gaps because they reflect… differences in access to educational opportunities. By ‘educational opportunities,’ we mean all experiences in a child’s life, from birth onward, that provide opportunities for her to learn, including experiences in children’s homes, child care settings, neighborhoods, peer groups, and their schools. This implies that test score gaps may result from unequal opportunities either in or out of school; they are not necessarily the result of differences in school quality, resources, or experience. Moreover, in saying that test score gaps reflect differences in opportunities, we also mean that they are not the result of innate group differences in cognitive skills or other genetic endowments… (D)ifferences in average scores should be understood as reflecting opportunity gaps….”
There is nothing in the guide to the Early Literacy measure on the new Ohio State School Report Card that acknowledges the early gaps in preparation for reading that children present as they enter Kindergarten. Surely the new Ohio School Report Card’s Early Literacy measure is as much a measure of young children’s experiences outside of school—parents who read with them, and exposure to enriched child care, Pre-Kindergarten, and public library story hours—as it may be to their in-school experiences before they take the third-grade Language Arts Proficiency test that is so key to this measurement.
There are several reasons the Ohio Department of Education chose not to create one overall summative rating for each school and school district this year. The pandemic affected school districts differently with some districts forced to use more online services during COVID-19 upswings and with widely disparate access to the internet and home computers among the state’s children. State officials imply that they want this year’s five star ratings to be a helpful guide for school districts. But a reporter, grasping the public’s hunger for comparisons, found a way to rank the districts in order merely by adding up each district’s total number of stars and publishing the state’s school districts in order from top to bottom. The Cleveland Plain Dealer reporter begins his report: “While the new Ohio school report card gives a star rating to various performance categories, there is no overall performance grade assigned for this year. So cleveland.com calculated the total score for all 607 districts… to show which schools scored the best across the board.”
I predict that, even without assigning an overall letter grade for schools and school districts, Ohio’s new, much touted five-star rating system will continue to promote educational redlining across Ohio’s metropolitan school districts. Prospective home buyers will read the five-star system the same way they have been reading the “A” through “F” letter grade school district rating system. They will continue to want to live in the school districts with the most stars, and the system will, thereby, exacerbate economic and racial segregation as people who can afford it continue to move to pockets of privilege in exurbia. After all, in a follow-up report, the Plain Dealer‘s Jeremy Pelzer notes that “‘suburban, higher-income districts in Northeast Ohio and around the state, not surprisingly, generally received higher report-card ratings…. A dozen school districts in Ohio received perfect scores across the board, including four in Northeast Ohio: Aurora City School District, Chagrin Falls Exempted Village Schools, Highland Local Schools in Medina County, and Solon City Schools.” All are higher-income exurbs.
Ohio’s new school rating system appeared just a week after the Washington Post‘s Valerie Strauss published an overall critique of school accountability as measured quantitatively primarily by a district’s aggregate standardized test scores. Strauss reminds readers that, “For several decades now, education policymakers have been obsessed with data-driven accountability—usually with standardized test scores as the key metric. The approach has failed to achieve any of the goals supporters have championed, such as closing the achievement gap, and has instead brought us things like pep rallies to get students excited to take standardized tests and methods to evaluate teachers based on the scores.”
Strauss publishes a piece written by two Northwestern University researchers, both sociologists, who have been evaluating our society’s obsession with ranking and rating. Simone Ispa-Landa and Wendy Espeland declare: “We are a nation obsessed with lists and rankings, not just for dishwashers and other consumer products. We track our steps, rate our sleep, and go to hospitals with the ‘best ratings.’… In our research, we find that, across institutions, school leaders are pressured to devote enormous time and energy to ‘improving the numbers,’ even when this comes at the expense of making changes that, in private, they acknowledge would be far more impactful for students. Because rankings and other measures change how school leaders do their work and make decisions, current accountability policies have far-reaching implications for school discipline and student mental health at a moment of intense national crisis in child and youth well-being… We should acknowledge that one-size-fits-all metrics do not fairly measure what matters most in many schools…. We should reward schools for innovation, for creating programs that will take time to evaluate. Simple numbers promote simple solutions and can prevent promising programs with long-term positive implications from taking root. Before we head into another school year, let’s look at dismantling the ranking systems that are burdening our administrators… and preventing authentic improvement.”
Ohio’s brand new school report cards—still based largely on each school’s aggregate standardized test scores—neglect to reflect the experiences created by fine educators who meet students where they are and help them experience educational opportunity in classes that are respectful, challenging and emotionally safe.