Yes! Rethinking the Value of Testing and of Graduation Tests, Ohio Joins More Progressive States

At its meeting on Tuesday, the Ohio State Board of Education discussed ways to reduce standardized testing along with the urgent need to amend the state’s current demand that high school students pass an overly tough set of end-of-course exams in order to qualify for high school graduation. The board had already eased the graduation requirement for the class of 2018. Now its members have agreed to ask the legislature to add an alternative path to graduation for students in the classes of 2019 and 2020.

The Plain Dealer‘s education reporter Patrick O’Donnell explains: “Statewide requirements that students score well on state tests in order to earn a diploma took effect with the class of 2018, this year’s senior class. But worries about a graduation ‘apocalypse’ or ‘trainwreck’ because of low scores led the board and state legislature to ease the requirements earlier this year, just for the senior class… After debate the last few months, board members now want to extend the same exemptions for the classes of 2019 and 2020… Those include graduating, even if state test scores are poor, by reaching some career training goals, having strong attendance or classroom grades as seniors, doing a senior capstone project or working at a job or on community service.”

On Tuesday, the Ohio State Board of Education also discussed ways to reduce the overall heavy test burden on students and teachers: “The state school board is asking the Ohio legislature to wipe out three items that add a testing burden to teachers and students—the high school English I exam, WorkKeys tests for some career training students, and requirements that some tests be given just to evaluate teachers.  State Superintendent Paolo DeMaria and an advisory panel he appointed recommended these and other changes to the board in June, after statewide outcry over the time spent on standardized testing in schools… Board members voted nearly unanimously for the three reductions Tuesday afternoon… (T)he board and DeMaria agreed that the state needs only the high school English II exam, usually given to sophomores, to meet the federal requirement for an English test in high school. They also agreed strongly with DeMaria’s recommendation to wipe out tests that are given just to measure the effectiveness of teachers.  Districts often give a pre-test at the start of the year, then another at the end of the year, to see how much a teacher taught over the year.”

O’Donnell adds that State Superintendent DeMaria recommends eliminating a number of other tests considered extraneous by his advisory panel.

Ohio’s beginning steps to cut back on the standardized testing that has dominated schools since 2002, when No Child Left Behind became federal law, reflect a broader trend, according to Monty Neill and Lisa Guisbond of the National Center for Fair and Open Testing (FairTest).  FairTest just released a major report, Test Reform Victories Surge in 2017: What’s Behind the Winning Strategies?, which summarizes the effects of broad public opposition to over-testing and some relaxation of federal pressure now that No Child Left Behind has been replaced by the Every Student Succeeds Act: “Widespread opposition to the overuse and misuse of standardized testing is producing a marked shift in attitudes about high-stakes assessments and, increasingly, state and district practices… The drumbeat of concerns includes: the amount of testing; the time it consumes; the outsized consequences for students, teachers and schools attached to test scores; the negative impacts on educational equity for low-income and minority students; and the damage to teaching, learning and children’s futures from the testing fixation.”

FairTest’s report is particularly scathing about the damage for young adults when failure of state-mandated tests denies them a high school diploma: “For tens of thousands of students who don’t drop out but stay in school and complete their other high school graduation requirements, exit exams unjustly confer the status and diminished opportunities of high school dropouts. The National Research Council of the National Academy of Sciences concluded that the graduation tests have done nothing to lift student achievement but have raised the dropout rate. Since 2012, the number of states that had or planned to have standardized high school exit exams has plunged from 25 to 13.”

FairTest adds that “seven states have made their elimination of graduation testing retroactive,” creating the opportunity for students previously denied diplomas in Georgia, South Carolina, California, Alaska, Arizona, Texas, and Nevada to apply for the diplomas they were denied as long as they successfully completed all other graduation requirements.

Public opinion has been changing as it has been more widely understood that “passing” cut scores on standardized tests are in many ways aspirational, not realistic. Cut scores that determine children’s futures have not been based on some kind of scientifically determined amount of knowledge children must master; instead they have been set by politicians for the purpose of driving teachers to work harder and faster.  High stakes standardized testing has been particularly punitive for students who start much farther behind.

Here is Daniel Koretz, the Harvard University professor whose new book, The Testing Charade: Pretending to Make Schools Better, exposes the damage inflicted by high stakes testing: “One aspect of the great inequity of the American educational system is that disadvantaged kids tend to be clustered in the same schools. The causes are complex, but the result is simple: some schools have far lower average scores…. Therefore, if one requires that all students must hit the proficient target by a certain date, these low-scoring schools will face far more demanding targets for gains than other schools do. This was not an accidental byproduct of the notion that ‘all children can learn to a high level.’ It was a deliberate and prominent part of many of the test-based accountability reforms…. Unfortunately… it seems that no one asked for evidence that these ambitious targets for gains were realistic. The specific targets were often an automatic consequence of where the Proficient standard was placed and the length of time schools were given to bring all students to that standard, which are both arbitrary.” (pp. 129-130)

Advertisements

Sorting Out the Debate About Educational Accountability

The watchword for the last quarter century’s school reform has been accountability: holding schools and school teachers accountable for quickly raising students’ scores on standardized tests. Sanctioning schools and teachers who can’t quickly raise scores was supposed to be an effective strategy for overcoming educational injustice. Test-and-punish has enabled us at least to say we’ve been doing something to hold schools accountable.

The politics of this conversation are pretty confusing—all going back to the federal education law, the 2001 No Child Left Behind Act (NCLB), and the debate about its replacement, the 2015 Every Student Succeeds Act (ESSA).  There was bipartisan agreement in 2001-2002 when NCLB was debated, passed, and signed into law that our society could close racial and economic achievement gaps by testing all students and then demanding that schools quickly raise the scores of underachieving students. In 2015 when Congress debated the law’s reauthorization, accountability-hawk Democrats stood by test-and-punish accountability; many Republicans, led by Senator Lamar Alexander instead pushed to expand states’ rights by lifting the heavy hand of the federal government and allowing states to design their own plans to improve so-called failing schools. Worrying that removal of universal testing would let schools off the hook, the Civil Rights Community has stood by NCLB’s testing plan. Many have continued to assume that universal testing exposes achievement gaps and that the exposure will motivate politicians and educators to address racial and economic disparities.

Test-and-punish school reform has been at the center of a conversation between Republican Senator Lamar Alexander, the chair of the Senate Health, Education, Labor and Pensions Committee, and Republican Education Secretary Betsy DeVos.  An article by Caitlin Emma published over the weekend by POLITICO examines the history of No Child Left Behind vs. the Every Student Succeeds Act as a background for looking at how policy around school accountability has been evolving in the Trump administration. Emma describes the new ESSA, passed by a Republican Congress in 2015 and designed to return at least some authority for accountability back to the states. But Democrats prodded by Civil Rights leaders and some Republicans have stood by federally imposed accountability: “Critics… worry whether states will adequately track and provide equal opportunities for at-risk kids…. (Even) former Republican Rep. John Kline… an architect of the measure, has said he’s worried states are now getting away with testing plans that violate a key requirement of the law—that states administer the same test to all students annually.  The provision is critical (Kline believes) so that states are forced to report the performance of all students and the results for poor and minority students are not hidden from view, as they were for decades before federal testing requirements were enacted.”

Emma explains: “The Every Student Succeeds Act, which passed in 2015, was widely viewed by Republicans as a corrective to the federal overreach that followed… No Child Left Behind.”  Emma reports that last summer, when Jason Botel, an official in Betsy DeVos’s Department of Education began reviewing the states’ applications for federal funds under the ESSA, Botel demanded that before he would approve some states’ plans, they must toughen their standards and demand more.  Powerful Republican Senator Lamar Alexander, who had—during the 2015 reauthorization—supported a return of control to the states, formally complained to Betsy DeVos—“furious that a top DeVos aide was circumventing a new law aimed at reducing the federal government’s role in K-12 education. He contended that the agency was out of bounds by challenging state officials, for instance, about whether they were setting sufficiently ambitions goals for their students.”

For many of us who have, for fifteen years, closely followed educational accountability as mandated under No Child Left Behind and the Every Student Succeeds Act, the entire debate seems wrong-headed and bizarre.  I am writing about those of us who care deeply about expanding opportunity for children segregated in schools where poverty is highly concentrated— schools where intense segregation by poverty is overlaid on segregation by ethnicity and race. The schools these children attend have, under federal policy, been derided by accountability hawks as “failing” schools.  Widespread blaming—of schools and school teachers—now dominates discussions of school reform even as sociologists increasingly document that family and neighborhood poverty pose overwhelming challenges for these children and their schools.

Much of the confusion and rancor arises because the public debate about school accountability conflates two very different questions:

  • Should the federal government be involved at all in telling states what to do about education?
  • Is test-and-punish accountability an effective strategy for improving public schools and closing opportunity gaps?

The original federal education law, the 1965 Elementary and Secondary Education Act, addressed the first question as a response to the needs of children in primarily southern states, where schools serving black children had been underfunded and inadequate for generations. There are similar problems of inequity across cities today and forgotten rural areas. Poor children and children of color segregated in particular areas remain under served. The debate about this first question involves states’ rights vs. what has come to be accepted (by many of us) as the federal government’s responsibility to protect the rights of all children and ensure they are all well served. It is a heated question that remains underneath much of the debate about school reform.

The second question involves the strategy Congress chose for reforming schools in the 2001 No Child Left Behind Act. Congress blamed teachers and schools and devised a law that was supposed to force schools and teachers to work harder and faster to improve test scores in schools where achievement lagged when all children in each state were tested on a single standardized test.  It is becoming clearer all the time that when Congress jumped behind test-and-punish accountability, it chose the wrong strategy.  A long and growing body of research demonstrates that test scores are far more aligned with a school’s aggregate economic level than with the work of the teachers or the curriculum being offered to students. Economists like Bruce Baker at Rutgers University also document enormous opportunity gaps as these same public schools in our nation’s poorest communities receive far less public investment than the schools in wealthy suburbs, schools serving children whose families also invest heavily in enrichments at home.

Here is just some of the prominent research from the past ten years that tries to answer the second question.

In 2010, Anthony Bryk and educational sociologists from the Consortium on Chicago School Research at the University of Chicago described the challenges for a particular subset of schools in Chicago, Illinois that exist in a city where many schools serve low income children. The Consortium focused on 46 schools whose students live in neighborhoods where poverty is extremely concentrated.  These “truly disadvantaged” schools are far poorer than the norm. They serve families and neighborhoods where the median family income is $9,480. They are racially segregated, each serving 99 percent African American children, and they serve on average 96 percent poor children, with virtually no middle class children present. The researchers report that in the truly disadvantaged schools, 25 percent of the children have been substantiated by the Department of Children and Family Services as being abused or neglected, either currently or during some earlier point in their elementary career. “This means that in a typical classroom of 30… a teacher might be expected to engage 7 or 8 such students every year.”  “(T)he job of school improvement appears especially demanding in truly disadvantaged urban communities where collective efficacy and church participation may be relatively low, residents have few social contacts outside their neighborhood, and crime rates are high.  It can be equally demanding in schools with relatively high proportions of students living under exceptional circumstances, where the collective human need can easily overwhelm even the strongest of spirits and the best of intentions. Under these extreme conditions, sustaining the necessary efforts to push a school forward on a positive trajectory of change may prove daunting indeed.” (Organizing Schools for Improvement, pp. 172-187)

Then in 2011, Sean Reardon of Stanford University released a massive data analysis confirming the connection of school achievement gaps to growing economic inequality and residential patterns becoming rapidly more segregated by income. Reardon documented that across America’s metropolitan areas the proportion of families living in either very poor or very affluent neighborhoods increased from 15 percent in 1970 to 33 percent by 2009, and the proportion of families living in middle income neighborhoods declined from 65 percent in 1970 to 42 percent in 2009.  Reardon also demonstrated that along with growing residential inequality is a simultaneous jump in an income-inequality school achievement gap among children and adolescents.  The achievement gap between students with income in the top ten percent and students with income in the bottom ten percent is 30-40 percent wider among children born in 2001 than those born in 1975.

In The Testing Charade, a book published just last month, Daniel Koretz of Harvard University blames test-and-punish accountability for enabling our society to pretend that we have been overcoming educational inequity at the same time we avoid making the public investment necessary even to begin addressing the problem: “One aspect of the great inequity of the American educational system is that disadvantaged kids tend to be clustered in the same schools. The causes are complex, but the result is simple: some schools have far lower average scores…. Therefore, if one requires that all students must hit the proficient target by a certain date, these low-scoring schools will face far more demanding targets for gains than other schools do. This was not an accidental byproduct of the notion that ‘all children can learn to a high level.’ It was a deliberate and prominent part of many of the test-based accountability reforms…. Unfortunately… it seems that no one asked for evidence that these ambitious targets for gains were realistic. The specific targets were often an automatic consequence of where the Proficient standard was placed and the length of time schools were given to bring all students to that standard, which are both arbitrary.” (pp. 129-130)  “If we are going to make real headway, we are going to have to confront the simple fact that many teachers will need substantial supports if they are going to markedly improve the performance of their students… And the range of services needed is broad. One can’t expect students’ performance in schools to be unaffected by inadequate nutrition, insufficient health care, home environments that have prepared them poorly for school, or violence on the way to school.” (p. 201)

The second question involves the overall direction of education policy, and it is important because we desperately need a better strategy. Blaming and punishing the schools with the lowest scores—by closing “failing” schools or privatizing them or firing their teachers and principals—has only further undermined the public schools in the poorest neighborhoods of our big cities without addressing the opportunity gaps the tests identify.

Today’s Republican tax slashing agenda will only further reduce public investment in education.  And we are likely to keep on blaming the victims.

Daniel Koretz: More Detail from “The Testing Charade” on Cheating Scandal in Atlanta

Back in 2015, I watched when part of the trial of the Atlanta school teachers—accused of erasing and correcting their students’ test scores—was televised on C-Span (see here and here). And two weeks ago I read Daniel Koretz’s new book, The Testing Charade, a book about what happens when high stakes punishments are attached to any social indicator. I read Koretz’s book pretty much without emotion or judgment—as an academic exercise to understand his argument against the high stakes that policy makers have used as a threat to drive teachers to work harder and raise test scores faster. I didn’t focus on the sections about the cheating scandals.  After all, I imagined, the scandals have just become a part of history.

Then on Wednesday evening, I watched Lisa Stark’s report for the PBS NewsHour about the 9 Atlanta school teachers and principals who are appealing their criminal convictions to clear their names and avoid stints in prison for participating in what is said to have been a 44-school cheating scandal driven by Superintendent Beverly Hall, who won awards when test scores rose miraculously quickly in Atlanta’s schools. Hall died before her own involvement could be adjudicated.

Daniel Koretz, the Harvard professor whose new book explores the Atlanta cheating scandal (among cheating scandals in Washington, D.C, Pennsylvania and many other places) as among the widespread consequences of our test-and-punish regime of school reform, spoke briefly in Lisa Stark’s report. In his book he attributes the problem to what social scientists call Campbell’s Law. Here is Koretz’s definition: “The more any quantitative social indicator is used for social decision making the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” (p. 38)

Koretz explores the issue far more deeply in his new book than he did in Wednesday night’s short clip for the NewsHour. My feeling two years ago that the Atlanta educators’ criminal convictions were unfair and what, as I watched the PBS report, I recognized as my feeling of relief two weeks ago when I read Koretz’s book—that an expert scholar confirmed my own sense of injustice in Atlanta—sent me back again yesterday to Koretz’s book.  Here is some of what he didn’t have time to say in Wednesday’s report for PBS.

“One aspect of the great inequity of the American educational system is that disadvantaged kids tend to be clustered in the same schools. The causes are complex, but the result is simple: some schools have far lower average scores—and, particularly important in this system, more kids who aren’t ‘proficient’—than others. Therefore, if one requires that all students must hit the proficient target by a certain date, these low-scoring schools will face far more demanding targets for gains than other schools do. This was not an accidental byproduct of the notion that ‘all children can learn to a high level.’ It was a deliberate and prominent part of many of the test-based accountability reforms… Unfortunately… it seems that no one asked for evidence that these ambitious targets for gains were realistic. The specific targets were often an automatic consequence of where the Proficient standard was placed and the length of time schools were given to bring all students to that standard, which are both arbitrary.” (pp. 129-130) Koretz continues: “(T)his decision backfired. The result was, in many cases, unrealistic expectations that teachers simply couldn’t meet by any legitimate means.” (p. 134)

In Atlanta, Koretz describes the situation at Parks Middle School, as it was portrayed by Rachel Aviv in a New Yorker profile of the Atlanta cheating scandal.  Koretz explains: “This is the school where Damany Lewis and Christopher Waller worked. Aviv documented the way in which Waller choreographed an increasingly large and well-organized cheating ring… Why did Lewis and others do this  At least in Lewis’s case, it was not because he was comfortable cheating. Quite the contrary…  Then why? In a nutshell, because their only other choice was to fail—not when compared with reasonable goals but when held to Hall’s and NCLB’s entirely arbitrary targets. Parks is located in a terribly depressed neighborhood. Half the homes are vacant. Students call the neighborhood ‘Jack City’ because of all the armed robberies. Very few of the students come from homes with two parents. Aviv reported that some students came to school in filthy clothing and that Lewis told students to drop dirty laundry in the back of his truck so that he could wash clothes for them. Some of the parents were dysfunctional because of drug use. During the years leading up to the cheating scandal, Parks had made real progress. A new principal renovated the school and worked on both refocusing students on academics and building a sense of community. Using funds that Hall’s administration had obtained, the school implemented after-school and tutoring programs. However, this simply wasn’t enough, given how fast scores had to rise to meet Hall’s demands. Lewis told Aviv that he had pushed his students harder than they had ever been pushed and that he was ‘not willing to let the state slap them in the face and say they’re failures.'” (pp. 77-78)

Besides leaving 9 Atlanta teachers and principals with criminal convictions, what has been the ultimate outcome of all this test-and-punish for society as a whole including our children? “It’s no exaggeration to say that the costs of test-based accountability have been huge. Instruction has been corrupted on a broad scale. Large amounts of instructional time are now siphoned off into test-prep activities that at best waste time and at worst defraud students and their parents. Cheating has become widespread. The public has been deceived into thinking that achievement has dramatically improved and that achievement gaps have narrowed. Many students are subjected to severe stress… Educators have been evaluated in misleading and in some cases utterly absurd ways. Careers have been disrupted and in some cases ended. Educators including prominent administrators, have been indicted and even imprisoned. The primary benefit we received in return for all of this was substantial gains in elementary-school math that don’t persist until graduation.” (p. 191)

Koretz concludes: “Reformers may take umbrage and say that they certainly didn’t demand that teachers cheat. They didn’t, although in fact many policy makers actively encouraged bad test prep that produced fraudulent gins. What they did demand was unrelenting and often very large gains that many teachers couldn’t produce through better instruction, and they left them with inadequate supports as they struggled to meet these often unrealistic targets. They gave many educators the choice… fail, cut corners, or cheat—and many chose not to fail.” (p. 244)

Harvard’s Daniel Koretz Indicts High Stakes Testing in “The Testing Charade”

Daniel Koretz’s new book, The Testing Charade: Pretending to Make Schools Better, is a scathing indictment of our society’s test-and-punish school regime, formalized in the 2002 No Child Left Behind Act and continuing in the most recent version of the federal education law, the Every Student Succeeds Act.  Koretz, the testing specialist, is not so critical of standardized testing itself as he is of the high stakes sanctions that Congress attached to the annual tests in No Child Left Behind—punishments that have driven massive pressure on educators that has ruined our public schools:

“Pressure to raise scores on achievement tests dominates American education today. It shapes what is taught and how it is taught.  It influences the problems students are given in math class (often questions from earlier tests), the materials they are given to read, the essays and other work they are required to produce, and often the manner in which teachers grade this work. It determines which educators are rewarded, punished, and even fired. In many cases it determines which students are promoted or graduate. This is the result of decades of ‘education reforms’ that progressively expanded the amount of externally imposed testing and ratcheted up the pressure to raise scores.” (p. 1)

Daniel Koretz’s biography at the Harvard Graduate School of Education describes him as an expert on educational assessment and testing policy, and the book describes in considerable detail just how high stakes punishments for schools and teachers have corrupted the results of the tests themselves, narrowed the curriculum, and degraded teaching.

But my deepest interest in the book is Koretz’s depiction of how the testing that was supposed force teachers and schools to better serve poor children, raise their test scores and close achievement gaps has instead truncated opportunity for the very children it was supposed to help. How has test-and-punish narrowed the curriculum to basic reading and math in the poorest schools, and how has it forced teachers to focus on test-prep and coaching instead of enrichment?  How has test-and-punish forced the closing or charterizing of schools in poor neighborhoods? How has evaluating teachers by their students’ test scores resulted in firing principals and teachers in the poorest schools and exacerbated staff turnover?  And what about the children being held back in third grade due to a test score—even when they may be making real progress in reading and the adolescents denied a high school diploma?

Under current federal law, students and schools are given credit for proficiency only when children reach benchmark proficiency scores. A fourth grader who advances during the school year from a first to a third grade reading level will still fail to achieve the fourth grade cut score. Neither the child nor the teacher will be given credit for the child’s improvement: “One aspect of the great inequity of the American educational system is that disadvantaged kids tend to be clustered in the same schools. The causes are complex, but the result is simple: some schools have far lower average scores…. Therefore, if one requires that all students must hit the proficient target by a certain date, these low-scoring schools will face far more demanding targets for gains than other schools do. This was not an accidental byproduct of the notion that ‘all children can learn to a high level.’ It was a deliberate and prominent part of many of the test-based accountability reforms…. Unfortunately… it seems that no one asked for evidence that these ambitious targets for gains were realistic. The specific targets were often an automatic consequence of where the Proficient standard was placed and the length of time schools were given to bring all students to that standard, which are both arbitrary.” (pp. 129-130)

Reformers decided that, if sufficiently pressured to raise test scores, teachers would be able to do so: “(T)hey acted as if… (schools alone could) largely eliminate variations in student achievement, ignoring the impact of factors that have nothing to do with the behavior of educators—for example, the behavior of parents, students’ health and nutrition, and many characteristics of the communities in which students grow up.” (p. 123-124) Koretz explains at length and in detail the ways that teachers and principals whose jobs are threatened have resorted to raising scores—coaching for the test, drilling on materials likely to be covered, and in some cases where the pressure was greatest, cheating by erasing and correcting scores.

Koretz quotes Linda Darling-Hammond’s characterization of test-and-punish school accountability: “the kick the dog harder model of education reform.” And he explains: “If we are going to make real headway, we are going to have to confront the simple fact that many teachers will need substantial supports if they are going to markedly improve the performance of their students… And the range of services needed is broad. One can’t expect students’ performance in schools to be unaffected by inadequate nutrition, insufficient health care, home environments that have prepared them poorly for school, or violence on the way to school.” (p. 201)  He suggests first that we stop judging all students and schools by benchmark scores. We must “set goals based on students’ growth, not the level of their performance.” (p. 235)

In the Washington Post, Valerie Strauss interviews Koretz about his new book, and she publishes an excerpt.

While I have emphasized the sections in which Koretz shows test-and-punish hurting the schools that serve the poorest and most vulnerable children, Koretz is a testing expert, whose primary interest is how high stakes punishments attached to a regime of universal testing have corrupted the entire operation of public schools: “Reformers may take umbrage and say that they certainly didn’t demand that teachers cheat. They didn’t, although in fact many policy makers actively encouraged bad test prep that produced fraudulent gains. What they did demand was unrelenting and often very large gains that many teachers couldn’t produce through better instruction, and they left them with inadequate supports as they struggled to meet these often unrealistic targets. They gave many educators the choice I wrote about thirty years ago—fail, cut corners, or cheat—and many chose not to fail.” (p.244)

Koretz joins a growing number of critics who indict test-and-punish school accountability. What is significant about this book is the thorough and relentless critique by a testing expert who carefully and sometimes technically dissects the evidence.