Repeating My Recommendation: Please Read Daniel Koretz’s Book, “The Testing Charade”

How has high stakes testing ruined our schools and how has this strategy, which was at the heart of No Child Left Behind, made it much more difficult to accomplish No Child Left Behind’s stated goal of reducing educational inequality and closing achievement gaps?

Here is how Daniel Koretz begins to answer that question in his 2017 book, The Testing Charade: Pretending to Make Schools Better: In 2002, No Child Left Behind “mandated that all states use the proficient standard as a target and that 100 percent of students reach that level. It imposed a short timeline for this: twelve years. It required that schools report the performance of several disadvantaged groups and it mandated that 100 percent of each of these groups had to reach the proficient standard. It required that almost all students be tested the same way and evaluated against the same performance standards.  And it replaced the straight-line approach by uniform statewide targets for percent proficient, called Adequate Yearly Progress (AYP)…. The law mandated an escalating series of sanctions for schools that failed to make AYP for each reporting group.” Later, “Arne Duncan used his control over funding to increase even further the pressure to raise scores.  The most important of Duncan’s changes was inducing states to tie the evaluation of individual teachers, rather than just schools, to test scores… The reforms caused much more harm than good. Ironically, in some ways they inflicted the most harm on precisely the disadvantaged students the policies were intended to help.”

Koretz poses the following question and his book sets out to answer it: “But why did the reforms fail so badly?”

I recommend Daniel Koretz’s book all the time as essential reading for anyone trying to figure out how we got to the deplorable morass that is today’s federal and state educational policy.  I wish I thought more people were reading this book. Maybe people are intimidated that its author is a Harvard expert on the design and use of standardized tests.  Maybe it’s the fact that the book was published by the University of Chicago Press. But I don’t see it in very many bookstores, and when I ask people if they have read it, most people tell me they intend to read it. To reassure myself that it is really worth reading, I set myself the task this past weekend of re-reading the entire book. And I found re-reading it to be extremely worthwhile.

The book divides into three parts—an introductory section of several chapters—six or seven chapters in the middle that dissect the way high stakes testing has undermined education and damaged the education of our nation’s poorest children—and some wrap-up chapters. It is the middle part that is essential. While Koretz has some ideas near the end about where we go from here, his analysis of the damage caused is the crucial part. After all, this section at the heart of the book addresses the conversational dilemma many readers of this blog must face as often as I do. What can you say to the person who doggedly tells you that a particular school is a fine school because its scores are high and another school is a failure because its test scores are so low? This person, often well-intentioned, has lived with test-based school accountability for so long that he cannot imagine there is any other way to consider school quality. And anyway, he says, standardized testing is what we have to evaluate schools, so it’s what we need to use.

Koretz explains a 40-year-old social science rule first articulated by Don Campbell, who Koretz identifies as “one of the founders of the science of program evaluation.” Here is how Campbell stated what we now call “Campbell’s Law”: “The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” The rest of the central chapters in Koretz’s book explain precisely how the use of high stakes punishments tied to low test scores has triggered Campbell’s Law. What are the high stakes punishments?  First came the school turnarounds prescribed by No Child Left Behind —firing the principal and half the teachers, closing the school, charterizing the school.  Later Arne Duncan added the evaluation of teachers by students’ test scores—and schemes rewarding teachers whose students scored well and firing the teachers whose students post low scores. Koretz summarizes No Child Left Behind’s test-and punish strategy: “The reformers’ implicit assumption seemed to be that many teachers knew how to teach more effectively but were being withholding, and therefore confronting them with sanctions and rewards would be enough to get them to deliver.”

Three chapters explore how No Child Left Behind’s test-and-punish strategy has distorted schooling itself and has undermined how teachers teach and how students learn.

  • Score Inflation: When the state achievement tests mandated by No Child Left Behind—the ones that would bring negative consequences for schools and teachers—were compared by experts like Koretz himself to another “audit” test such as the National Assessment for Education Progress (NAEP), which has no high stakes consequences, the researchers discovered that while the scores on the state test rose rapidly, NAEP scores remained flat.  Koretz comments: “(I)ncreases in scores are meaningful only if they signal similar increases in mastery of the domain.  If they do generalize to the domain, gains should appear on other tests that sample from the same domain.” He continues: “(A)ll that is required for scores to become inflated is that the sampling used to create a test has to be predictable… For inflation to occur, teachers or students need to capitalize on this predictability, focusing on the specifics of the test at the expense of the larger domain.”  And there are equity concerns here, because score inflation has occurred more often in schools serving poor students: “Ongoing work by my own group has shown… that it is not just the poverty of individual students that predicts the amount of inflation but also the concentration of poor students in a school… (S)chools with a higher proportion of poor students showed greater average inflation.” Teachers under pressure are finding a way to raise test scores without really teaching the students the material they are supposed to be learning.  Some schools have also inflated overall scores by focusing primarily on children right at the pass/fail level and paying less attention to students far behind.
  • Cheating: Koretz examines the big cheating scandals, notably Atlanta, Philadelphia, and Washington, DC.  He notes: “Cheating—by teachers and administrators, not by students—is one of the simplest ways to inflate scores, and if you aren’t caught, it’s the most dependable.” Sometimes teachers or administrators erase and change students answers; sometimes they provide teachers or students with the test items in advance; other times teachers give students the answer during the test.  And finally sometimes schools “scrub” off the enrollment rolls the students who are likely to fail.
  • Test Prep: Test prep narrows what is taught to students to the material that is tested.  Koretz identifies three kinds of bad test prep. Reallocation between subjects has been common when schools emphasize No Child Left Behind’s tested subjects—reading and math—and cut back on social studies, the arts, music and recess. Reallocation within subjects is when schools study past years’ versions of the state tests and ask teachers to focus on particular aspects of a subject.  Finally there is coaching. Schools and test-prep companies teach students to respond in a formulaic way to the format of the questions themselves. Koretz explains why all this has implications for educational equity: “Inappropriate test preparation, like score inflation, is more severe in some places than in others. Teachers of high-achieving students have less reason to indulge in bad preparation for high-stakes tests because the majority of their students will score adequately without it—in particular, above the ‘proficient’ cut score that counts for accountability purposes. So one would expect that test preparation would be a more severe problem in schools serving high concentrations of disadvantaged students…. Once again, disadvantaged kids are getting the short end of the stick.”

Two chapters in this middle section explore the ways No Child Left Behind’s test-and-punish scheme has undermined equitable access to education in the schools in areas of concentrated poverty across our cities. The law that promised to leave no child behind not only encouraged test prep and cheating in the schools whose needs were greatest, but it also set impossibly tough and largely arbitrary test score targets for those schools and an impossibly short timeline for bringing students up to those targets.  And then the federal government set out to punish the schools and the teachers unable to meet the targets.

  • Making Up Unrealistic Targets: In this chapter, Koretz explains how No Child Left Behind’s standardized cut scores and timelines were set unrealistically and arbitrarily; the consequence was to label schools in poor areas as “failing” and to subject schools in areas of concentrated poverty to a series of punishments. Here is Koretz’s short summary: “Part of the blame for this failure lies with the crude and unrealistic methods used to confront inequity.  In a nutshell, the core of the approach has been simply to set an arbitrary performance target (the ‘Proficient’ standard) and declare that all schools must make all students reach it in an equally arbitrary amount of time.  No one checked to make sure the targets were practical.  The myriad factors that cause some students to do poorly in school—both the weaknesses of many of the schools they attend and the disadvantages some students bring to school—were given remarkably little attention. Somehow teachers would just pull this off… The trust most people have in performance standards is essential, because the entire educational system now revolves around them. The percentage of kids who reach the standard is the key number determining which teachers and schools will be rewarded or punished… But in fact, despite all the care that goes into creating them, these standards are anything but solid. They are arbitrary, and the ‘percent proficient’ is a very slippery number… A primary motivation for setting a Proficient standard is to prod schools to improve, but information about how quickly teachers actually can improve student learning doesn’t play much, if any, of a role in setting performance standards… However, setting the standards themselves is just the beginning. What gives the performance standards real bite is their translation into concrete targets for educators, which depends on more than the rigor of the standard itself… We have to say how quickly performance has to increase—not only overall but for different types of kids and schools. A less obvious but equally important question is how much variation in performance is acceptable.”
  • Evaluating Teachers: In 2009, beginning with Race to the Top and later as a condition for states to qualify for waivers from the worst consequences of No Child Left Behind, Arne Duncan’s Department of Education required states to change their laws to tie a percentage of teachers’ formal evaluations to students’ test scores. Myriad problems ensued. First of all, the required tests are in reading and math. What about the other teachers? Koretz describes Florida and Tennessee, which judged teachers in non-tested grades and subjects by the scores of students who were not in their classes, and in one case not in their schools.  Other states added tests in music, art, and physical education—subjecting students to added standardized testing—just for the purpose of state teacher evaluations.  Koretz explains the problems with Value-Added Modeling to evaluate teachers; many factors affecting students’ scores cannot be traced to any teacher and any teacher’s ratings seem to be unstable over several years.

I cannot imagine exactly how our society can recover from the our terrible test-and-punish misadventure and our labeling as “failing” the institutions and teachers who serve our poorest children.  What is heartening about The Testing Charade: Pretending to Make Schools Better is the clarity with which Daniel Koretz presents our current dilemma: “We now know what many educators did.  Faced with unrealistic targets, some cut corners or simply cheated.  And perhaps because the system, in its zeal to address inequities, made the targets most unrealistic for educators serving disadvantaged kids, those kids—ironically—got the worst of it: the most test prep, the most score inflation, and apparently the most cheating.  And yet inflated scores allowed policy makers to declare victory, and the public received a steady diet of encouraging but bogus news about rapid improvements in the achievement gap…. On balance… the reforms have been a failure.”

Please read The Testing Charade.  We all need to understand and be able to explain how we’ve gone so far astray.

Advertisements

Kansas Supreme Court Declares School Funding Equitable; More Money Needed for Adequate System

Two weeks ago, the Supreme Court of Kansas found that the state’s school funding system remains unconstitutional, but gave the state a year to increase the funding. This is a relief to families, as the Court had threatened to force the legislature into a special summer session to increase school funding or shut down school altogether for the fall.  It also is a relief for those looking for justice for the state’s children because it means the Court has retained jurisdiction in the case—to ensure that the legislature will have to find enough money to provide for the needs of children in the state’s public schools.

The case of Gannon v. Kansas preceded Sam Brownback’s tax-slashing tenure as Kansas’ governor, but Brownback’s tax cuts only made matters more desperate for public school districts in Kansas, and particularly for the school districts serving the state’s poorest children.

Writing on June 26, school finance expert Derek Black explains what just happened in Kansas: “Yesterday, the Kansas Supreme Court issued its third decision in two years regarding the state’s school funding practices.  Yet again, the court found that the state had failed to meet its constitutional duty… The two big issues before the court were the equality of its financing system and the adequacy. The court found that the state had finally developed a plan that would achieve equitable access to school funding.”

The Court credits the Legislature with addressing inequity, resulting from the fact that the state has been expecting school districts to be able to raise local funding through something called a Local Option Budget (LOB).  Wealthier school districts could afford to do so; very poor districts have not been able sufficiently to supplement the state’s contribution. Black explains: “Under the prior law, not all local districts had the capacity to meet their LOB targets. The new law, according to the court, cures the problem by taking into account the percentage of at-risk students a district serves. Those with higher percentages will calculate their LOB requirement (and the funds they are entitled to from the state) differently than other districts. In short, high-need districts will receive more from the state and be expected to generate less locally.”

While The Court approved this system as the path to equity,  the issue of inadequacy of funding remains. In other words, despite that last year the Legislature raised taxes to offset the revenue catastrophe caused by Sam Brownback’s big experiment with supply-side, tax-slashing economics, the state is still suffering from inadequate revenue. Brownback had predicted that his tax cuts would grow the economy, but his hypothesis was wrong.  Now it is taking years for the state to catch up.

Reporters for the Wichita Eagle and the Kansas City Star explain the situation for the 2018-2019 school year: “The Kansas Supreme Court ruled… that a new school funding plan is still inadequate, but gave the Legislature another year to fix it. ‘The State has not met the adequacy requirement in Article 6 of the Kansas Constitution,’ the court ruling said.  But if lawmakers add money to compensate for inflation Kansas ‘can bring the K-12 public education financing system into constitutional compliance.’… The Supreme Court has previously ruled that the Legislature must meet two tests to satisfy a state constitutional mandate to provide ‘suitable’ education funding: It must be adequate, meaning that there’s enough total money in the system for schools to provide a quality education. And it must be equitable, meaning that the state resources are allocated to give poor children the opportunity to obtain an education of roughly similar quality to what’s provided in wealthy districts.”

Retaining jurisdiction over the case, the Court will consider it again on April 15, 2019, “when both sides will have to file reports on whether they think the Legislature has corrected the remaining constitutional issues.”

In Kansas the Supreme Court has provided the kind of checks and balances that are missing across many of the 26 all-Red states, whose legislators and governors doggedly pursue anti-tax dogma. That is why many far right politicians in Kansas have come to believe the Supreme Court itself is the problem. The reporters for the Wichita Eagle and the Kansas City Star quote Susan Wagle, the Senate President and a Wichita Republican: “Today the unelected bureaucrats of the Kansas Supreme Court chose to continue with the endless cycle of school litigation, leading us down the road to an unavoidable tax hike… When Kansas is on par with Nancy Pelosi’s California for sky-high property taxes and families are fleeing the state, we can thank the Kansas Supreme Court.” Senator Wagle and her colleagues are pushing for a constitutional amendment to remove court oversight and make education funding the sole responsibility of the legislature.

What the theoretical discussion of adequacy and equity of school funding misses is the impact on the daily experiences children and schoolteachers. Kansas is one of 12 states identified last November by the Center on Budget and Policy Priorities where the per-pupil school funding remained lower than before the great recession in 2008.  Several of the others—Oklahoma, Kentucky, Arizona, West Virginia, and North Carolina—are places where teachers walked out in massive protests this spring. We listened in those states to the teachers’ stories of huge classes, scarcity of counselors and support services, outdated textbooks, pared-down curriculum, and paltry, non-competitive salaries. We need to replay those stories mentally as we read about the Kansas court battle for better school funding.

On a a theoretical level, however, Kansas is a good example of the importance of checks and balances. It is a place where the judicial branch of government is putting a stop to a radical anti-tax experiment launched by the executive and legislative branches. That is how government is supposed to work.

Rand Corp. Report Says Grading Teachers by Student Scores Doesn’t Work; Ohio Law Will Diminish Use of Student Scores for Evaluating Teachers

In 2009, the Bill & Melinda Gates Foundation launched a huge project to demonstrate that evaluating teachers by their students’ standardized test scores would improve education and especially the education of “low-income, minority” students. Now the Gates Foundation has paid for a huge Rand Corporation study that showed its original experiment didn’t work. Although the Gates Foundation can move on to testing another hypothesis, its prescription for grading teachers has done immeasurable damage by injecting econometric teacher evaluation into the laws of many states. It will take a long time for the 50 state legislatures to clean up laws based on a mistake.

Chalkbeat‘s Matt Barnum describes the original plan: “Barack Obama’s 2012 State of the Union address reflected the heady moment in education. ‘We know a good teacher can increase the lifetime income of a classroom by over $250,000,’ he said. ‘A great teacher can offer an escape from poverty to the child who dreams beyond his circumstance.’ Bad teachers were the problem; good teachers were the solution. It was a simplified binary, but the idea and the research it drew on had spurred policy changes across the country, including a spate of laws establishing new evaluation systems designed to reward top teachers and help weed out low performers.  Behind that effort was the Bill and Melinda Gates Foundation… Now, new research commissioned by the Gates Foundation finds scant evidence that those changes accomplished what they were meant to: improve teacher quality or boost student learning.  The 500-plus page report by the Rand Corporation… details the political and technical challenges of putting complex new systems in place…”

The Gates Foundation not only launched a giant experiment without an adequate research base, but it also leveraged the investment of public dollars and used its own lobbying might to influence public policy. The Obama administration conditioned qualification for Race to the Top grants on the use of students’ standardized test scores in teachers’ evaluations and later made the same requirement for states to qualify for No Child Left Behind waivers.

The Washington Post‘s Valerie Strauss details the history: “Put this in the ‘they-were-warned-but-didn’t-listen’ category.”  She describes the project launched in Hillsborough County (Greater Tampa), Florida, Memphis, and Pittsburgh along with four charter management organizations: “The Bill & Melinda Gates Foundation pumped nearly $215 million into the project while the partnering school organizations supplied their own money, for a total cost of $575 million.”  Federal policy makers jumped into the mix: “The Obama administration, through its Race to the Top initiative, dangled federal funds in front of states that agreed to establish teacher evaluation systems using test scores to varying extents.  And Gates funded his ‘Empowering Effective Teachers’ project with the aim of finding proof that such systems could improve student achievement…  (M)ost states adopted test-based teacher evaluation systems.  In a desperate attempt to evaluate all teachers on tested subjects—reading and math—some of the systems would up evaluating teachers on subjects they didn’t teach or on students they didn’t have. Some major organizations questioned them, including the American Statistical Association…. And so did the Board on Testing and Assessment of the National Research Council.”

Strauss quotes the conclusion of the Rand Corporation’s huge new assessment of the experiment: “Overall, the initiative did not achieve its stated goals for students, particularly LIM (low-income minority) students. By the end of 2014-2015, student outcomes were not dramatically better than outcomes in similar sites that did not participate in the IP (Intensive Partnerships) initiative. Furthermore, in the sites where these analyses could be conducted, we did not find improvement in the effectiveness of newly hired teachers relative to experienced teachers; we found very few instances of improvement in the effectiveness of the teaching force overall; we found no evidence that LIM students had greater access than non-LIM students to effective teaching; and we found no increase in the retention of effective teachers, although we did find declines in the retention of ineffective teachers in most sites.”

What the Rand Report fails to calculate is the collateral damage. It is well known that, in Hillsborough County, Florida, the Gates Foundation suspended its study before it had been completed—leaving the school district itself to cover a significant part of the cost. But beyond Hillsborough County, the consequences were long lasting as state legislatures, lured by Race to the Top funding and the need to qualify for No Child Left Behind waivers, passed laws basing teachers’ evaluations on students’ standardized test scores. When, in December of 2015, Congress replaced No Child Left Behind with the Every Student Succeeds Act, it removed the requirement that states use  students’ test scores in teachers’ evaluations, but the laws the states had put in place to meet federal requirements remained.

For example, only last week did the Ohio Legislature act to reduce the role of students’ test scores in the state teacher evaluation system. Finally—before going on a 2018 summer recess, the Ohio lawmakers passed a new statute reducing the weight of students’ standardized tests in the formal evaluation of teachers. The law passed with bipartisan support, and it is hoped that Governor John Kasich will sign it.

Last Sunday, the Columbus Dispatch‘s Jim Siegel reported that Ohio has been basing 50 percent of teachers’ ratings on students’ standardized  test scores .  Keep in mind that it is now 2018, and Ohio, like many other states, has still been using a plan that the Rand Corporation has now declared ineffective for measuring the quality of teachers.

Siegel quotes Jonathan Juravich, the 2018 Ohio Teacher of the Year, describing the new system: “No longer… (will) student growth measures be used as a disconnected evaluation factor linked to an arbitrary weighted percentage.”

Ohio is also finally doing away with “shared attribution,” according to Siegel: “Changes include doing away with shared attribution—growth measures attributed to a group of teachers that, critics say, does not accurately measure individual performance….”

State Superintendent Paolo DeMaria is quoted describing the new law: “Most importantly, we want our teachers on a path of continuous improvement, and with these changes the system places a greater focus on improvement in teacher practices that lead to better outcomes for students.”

The Bill & Melinda Gates Foundation and Obama administration’s collaborative scheme to evaluate teachers econometricaly has undermined the morale of school teachers and contributed to a climate in which teachers have been blamed unfairly when test scores don’t rise. Contrast the Gates theory, now rejected by the Rand Corporation report, with the research of Harvard’s Daniel Koretz, who explains how the test scores—so central to the school accountability movement—don’t really measure the quality of the schools or specific teachers, but instead primarily reflect the aggregate economic level of a school’s families and neighborhood:

“One aspect of the great inequity of the American educational system is that disadvantaged kids tend to be clustered in the same schools. The causes are complex, but the result is simple: some schools have far lower average scores…. Therefore, if one requires that all students must hit the proficient target by a certain date, these low-scoring schools will face far more demanding targets for gains than other schools do. This was not an accidental byproduct of the notion that ‘all children can learn to a high level.’ It was a deliberate and prominent part of many of the test-based accountability reforms…. Unfortunately… it seems that no one asked for evidence that these ambitious targets for gains were realistic. The specific targets were often an automatic consequence of where the Proficient standard was placed and the length of time schools were given to bring all students to that standard, which are both arbitrary.” (The Testing Charade; Pretending to Make Schools Better, pp. 129-130)

Ohio is now joining other states trying to undo the damage. Writing for the Stamford Advocate, Wendy Lecker, a columnist and attorney for the Education Law Center, explains: “Technology writer Eugene Morozov coined the term ‘solutionism’: a pathology that recognizes a problem based on one criterion only… solvable with a simple, preferably technological, solution. Solutionists operate with a myopic hubris, believing that if they get their simple fix right, as the chair of Google once claimed, ‘we can fix all the world’s problems.'”

The story of America’s nine year experiment with rating teachers by their students’ test scores ought to teach us to beware solutionists with gobs of money and the power to seduce policy makers.

(This blog has tracked education philanthropy from the Bill & Melinda Gates Foundation here.)

Michelle Rhee’s D.C. Education Revolution Continues to Collapse

The Associated Press‘s Ashraf Khalil explains: “As recently as a year ago, the public school system in the nation’s capital was being hailed as a shining example of successful urban education reform and a template for districts across the country. Now…. after a series of rapid-fire scandals including one about rigged graduation rates, Washington’s school system has gone from a point of pride to perhaps the largest public embarrassment of Mayor Muriel Bowser’s tenure… A decade after a restructuring that stripped the decision-making powers of the board of education and placed the system under mayoral control, city schools in 2017 were boasting rising test scores and a record graduation rate for high schools of 73 percent, compared with 53 percent in 2011… Then everything unraveled.”

Teachers jobs were threatened if they didn’t raise test scores and increase graduation rates by passing students no matter what. Money was an incentive, with bonuses rewarding successful teachers. The Washington Post revealed that last year, while the district bragged about declining suspension rates, the reports were a fake. Many high schools were suspending students  without documentation.

Then an investigation by Washington, D.C.’s NPR station WAMU, uncovered that Ballou High School had been covering up massive student absences. Many students had been chronically absent and missed so many days of classes that mandatory course failure should have followed. But teachers failed to mark students absent, and students who were failing classes were being allowed to complete inadequate credit recovery projects so that they would pass courses and be allowed to graduate.

A district-wide investigation into practices during the 2016-2017 school year revealed that the problem wasn’t just at Ballou, but had spread district-wide. Khalil reports that, “about half of those Ballou graduates had missed more than three months of school and should not have graduated due to chronic truancy. A subsequent inquiry revealed a systemwide culture that pressured teachers to favor graduation rates over all else—with salaries and job security tied to specific metrics.  The internal investigation concluded that more than one-third of the 2017 graduating class should not have received diplomas due to truancy or improper steps taken by teachers or administrators to cover the absences.  In one egregious example, investigators found that attendance records at Dunbar High School had been altered 4,000 times to mark absent students as present. The school system is now being investigated by both the FBI and the U.S. Education Department.”

The school district responded this past winter by tightening attendance monitoring and enforcing course requirements.  In April of this year, the Washington Post‘s Perry Stein reported that, “Fewer than half of the seniors in the District’s traditional public school system are on track to receive their diplomas in June…. The city released a first batch of data in February, which showed that 42 percent of seniors attending traditional public schools were on track to graduate, while 19 percent were considered ‘moderately off track.'”

Last week Stein updated the story, reporting that 60 percent of seniors earned diplomas on time this month: “The school system said this week that 415 students who were considered ‘moderately off track’ in April received their diplomas in June. Forty students who were ‘significantly off track’ graduated… Many of the off-track students enrolled in credit-recovery courses to graduate on time.”  She adds that some students are likely to graduate after completing summer school.

Stein tracks the graduation rates by student demographics and finds them to be predictable and unfortunate: “At Banneker High, a selective-application school, 99 percent of seniors graduated this month—the highest rate in the District. The lowest rates belonged to Anacostia (42 percent), Coolidge (44 percent) and Ballou (45 percent).

In a puzzling development, Stein reports: “The D.C. Council passed emergency legislation last week allowing high school seniors who missed more than six weeks of class to receive their diplomas by discounting absences from the first three quarters of the school year.” The members of the Council are reported to have reasoned that students shouldn’t be held accountable for attendance rules that were tightened only after the graduation crisis was discovered.  Mayor Muriel Bowser has said she is not supportive of the emergency law, but she has not yet made a choice to sign or not to sign it.  Stein added : “But Bowser, whose signature is necessary for the reprieve to go into effect, has said she opposes it.  She said last week she is considering her options, and her office said Wednesday there is no update on her decision.”  If the law is signed, students will receive their diplomas late, as commencement exercises have already taken place.

What has been occurring in the D.C. public schools in recent years—allowing students to miss so much school they become chronically absent and hiding the fact that they are not in school—assigning short and easy credit recovery projects when students are failing classes—has been driven by promises to make the D.C. Public Schools a model. Michelle Rhee set up a system to pressure teachers and school administrators to by threatening to fire those who failed and paying merit bonuses to those who can make themselves look successful.  In a new book, The Testing Charade: Pretending to Make Schools Better, Harvard University’s Daniel Koretz explains why making graduation rates the primary measure of success, will taint the process and undermine the results. Koretz writes about Campbell’s Law, a well known principle in the social sciences: “The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” (The Testing Charade, p. 38)

When Michelle Rhee had been the leader of the D.C. Public Schools for several years, John Merrow, now retired from the PBS NewsHour, documented a major test score cheating scandal driven by Rhee’s demand that teachers raise test scores and her techniques for getting scores to rise: making teachers’ and principals’ evaluations, hiring and firing, and merit bonuses depend on educators’ capacity to raise scores. Later under former Chancellor Kaya Henderson, rising graduation rates became a second primary metric.  Koretz explains how Campbell’s law actually works: “(W)hen you hold people accountable using a numeric measure—vehicle emissions, scores on a test, whatever—two things generally happen: they do things you don’t want them to do, and the measure itself becomes inflated, painting too optimistic a view of whatever it is that the system is designed to improve.”(The Testing Charade, p. 38)  Koretz elaborates: “(Y)ou can take Campbell’s Law to the bank. It’s going to show up in any high-pressure accountability system that is based only on a few hard numbers.” (The Testing Charade, pp. 46-47)

Koretz predicts the consequences of the kind of school reform Michelle Rhee brought to Washington, D.C.:  cheating, lowering standards—what has happened in the D.C. graduation scandal, and excluding people with bad numbers—what may have happened in the D.C. schools when suspension rates were fudged.

Mike Rose: How “School Failure” Narrative of “A Nation at Risk” Has Undermined Public Schools

I can’t bring myself to think of Naomi Klein in the same category as Mike Rose, one of my favorite education writers. They are important but very different writers.  There is one similarity, however.  In 2007, Klein responded to Hurricane Katrina and other natural catastrophes around the world with the publication of a blockbuster, thesis-driven social science analysis, The Shock Doctrine, in which she highlighted the swift takeover of New Orleans’ public schools after the hurricane as the very definition of her idea that a crisis from a natural disaster will often be grabbed as an opportunity by business interests looking for a profit. And this week, Rose explains in a new blog post, that his extraordinary book, Possible Lives, was his own response to a “shock doctrine” crisis created by the inflammatory language of the 1983 report, A Nation at Risk.

Klein explains “the shock doctrine” in the context of what can happen to a public school system when the interests of privatization are pitched as the best response to a catastrophe: “In sharp contrast to the glacial pace with which the levees were repaired and the electricity grid was brought back online, the auctioning off of New Orleans’ schools system took place with military speed and precision. Within nineteen months, with most of the city’s poor residents still in exile, New Orleans’ public school system had been almost completely replaced by privately run charter schools. Before Hurricane Katrina, the school board had run 123 public schools; now it ran just 4.  Before the storm, there had been 7 charter schools in the city; now there were 31. New Orleans’ teachers used to be represented by a strong union; now the union’s contract had been shredded, and its forty-seven hundred members had all been fired.”  (The Shock Doctrine, p. 5)

One person who absolutely absorbed the capitalist possibilities of Hurricane Katrina was a prominent policy maker, who had by, 2010 when he spoke out about it, become our U.S. Education Technocrat-in-Chief, Arne Duncan: “I spent a lot of time in New Orleans, and this is a tough thing to say, but let me be really honest. I think the best thing that happened to the education system in New Orleans was Hurricane Katrina. That education system was a disaster and it took Hurricane Katrina to wake up the community….” A crisis. A disaster. A time ripe for throwing it all away and trying something new.

In his new blog post—to mark the 35th anniversary of A Nation at Risk, Mike Rose explains that he undertook Possible Lives as a response to A Nation at Risk—to the exaggerated, urgent, fevered language in the Reagan era report’s introduction: “Our schools are mediocre and getting worse, and their sorry state is resulting in an erosion of our economic and technological preeminence. The opening sentences build momentum toward an existential threat, the equivalent of a military attack—brought on by ourselves, by our educational failures.”  Rose continues: “(O)ne hard lesson learned from A Nation at Risk is that the way problems are represented has major consequences.  This issue of language and representation sometimes gets lost in debates about the benefits or harm resulting from specific education reforms, but I think it is centrally important. It was one of the concerns that drove Possible Lives, published twelve years downstream from A Nation at Risk.”

Rose pulls out from the opening two paragraphs of A Nation at Risk the language he describes as framed precisely to generate a sense of educational catastrophe.  He quotes from the opening paragraphs::

“Our Nation is at risk. Our once unchallenged preeminence in commerce, industry, science, and technological innovation is being overtaken by competitors throughout the world… The educational foundations of our society are presently being eroded by a rising tide of mediocrity that threatens our very future as a nation and a people… If an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war. As it stands, we have allowed this to happen to ourselves… We have, in effect, been committing an act of unthinking, unilateral educational disarmament.”

What has followed for 35 years now Rose describes as a response to an existential education crisis created entirely by language carefully chosen and narratively framed to motivate our society to do something radical.  Our response? Test and punish via No Child Left Behind. Grading schools and teachers by their students’ aggregate test scores, including A-F grades imposed by states on their school districts and individual schools. Closing so-called failing schools. And all sorts of school privatization via charter schools, and virtual schools and vouchers and tuition tax credit vouchers and education savings account neo-vouchers—all to give children an escape from their so-called “failing” public schools.

Rose responded to A Nation at Risk‘s manipulation of language and public opinion—intended to create the impression that U.S. public schools were in crisis—by traveling the country for four years, visiting America’s best classrooms, and narrating the promising story of America’s public schools and their teachers. Possible Lives was first published in 1995 and reprinted in 2006. It is one of the most inspiring books ever written about what actually happens inside our public school classrooms.  The fact that classrooms are places most of the public never visits likely contributes to people’s manipulation through crisis-driven rhetoric. While A Nation at Risk turned the public attention to an obsessive examination of outcomes based on standardized test scores and technocratic fixes—stuff that can be easily reported and statistically processed, Rose takes readers right into classrooms to watch how teachers respond to children, how they challenge adolescents to puzzle out and reason, how they design projects that fascinate students and pique their imaginations.

Looking back at A Nation at Risk, Rose summarizes the difference between the language of its introduction—designed to terrify us all if we don’t do something quick—and the rest of the report: “So there it is. 1983 and we are doomed if we don’t do something fast and decisively. Erosion. Decline. Los of Power. Assault. An act of war—against ourselves. Interestingly, throughout the rest of the report, there is little of this apocalyptic language. While the authors continue to make some questionable aims and offer some debatable solutions, there are also calls to boost the teaching profession, to increase school funding, to promote ‘life-long learning,’ and to assure ‘a solid high-school education’ for all.  But few people read the full report.  What was picked up was the dire language of the opening; and—this is hugely important—that language not only took on a life of its own, it also distorted the way many reform-minded folk implemented the (more promising) recommendations of the report.”

Rose references a recent A Nation of Risk 35th anniversary story by Anya Kamenetz on National Public Radio: “Kamentz interviewed several of the authors of A Nation at Risk and found that they did not set out to conduct an objective investigation of the state of American education, but came to the task convinced that schools were in serious decline as global competition was heating up, and therefore their job was to sound the alarm and, as one author put it, get education ‘on the front page.’ They succeeded big time.”

Rose pulls out his own warning from Possible Lives about this kind of language: “It blinds us to the complex lives lived out in the classroom. It pre-empts careful analysis of one of the nation’s most significant democratic projects. And it engenders a mood of cynicism and retrenchment, preparing the public mind for extreme responses: increased layers of testing and control, denial of new resources—even the assertion that money doesn’t affect a school’s performance—and the curative effects of free market forces via vouchers and privatization. What has been seen historically as a grand republican venture (our institution of public education) is beginning to be characterized as a failed social experiment, noble in intention but moribund now, perhaps headed toward extinction.  So, increasing numbers of people who can afford to don’t even consider public schools as an option for their children, and increasingly we speak, all of us, about the schools as being in decline. This is what is happening to our public discussion of education, to our collective vision of the schools….”  Prophetic words from a book written in 1995 and reprinted in 2006.

Rose concludes his recent blog post with this warning: “One of the big challenges we have in front of us is how to maintain momentum in addressing the inequities in our education system but to do so in a way that is analytically and linguistically precise. How can we, to the best of our ability, keep focus on the vulnerable and underserved and do so with a mix of urgency and accuracy?”

I’ll add that in Possible Lives and the rest of his fine books, Rose has not only used language with precision, but also with a sense of compassion and human understanding. He shows us what happens at school—for children and adolescents and their teachers—without emphasizing the preoccupation of the school reformers—the technocratic measures and incentives and ratings that permeate our society and that always situate our schools in perpetual data-driven competition.

Please read Mike Rose’s new blog post.  And consider adding Possible Lives to your reading this summer.

Hopeful Signs That, Just Maybe, The Times They Are A-Changin’

In Detroit…

Rochelle Riley’s column on Sunday in the Detroit Free Press begins this way: “Something miraculous happened at this year’s Mackinac Policy Conference sponsored by the Detroit Regional Chamber. For the first time that I can recall, people in charge of changing people’s lives spent most of their time focused on those in the state who need their help the most.”

Detroit’s school superintendent, Dr. Nikolai Vitti, finishing his first year on the job, made the biggest waves—by telling the truth.

Remember that in these accountability-driven times, when school superintendents pretty much have to promise to raise test scores as a requirement for getting hired, these people have to pretend they can turn a district around on a dime. When they inevitably fail, they are fired. Three years is a long time for such people to last.

And remember the history of Detroit’s schools, until 2017 under years of state takeover—run by a state government dominated by DeVos conservatives. For years Governor Rick Snyder had been able to impose emergency fiscal managers on the school district—people responsible for cutting costs but not required to be educators. Until the beginning of 2017, when an elected school board was sworn in and its members set out to find a superintendent of their own choosing.

Dr. Nikolai Vitti was their choice, and last week at the end of his first year on the job, at a conference sponsored by the Chamber of Commerce at the Grand Hotel on Mackinac Island, he confronted the dogma of Michigan’s power establishment—Rick Snyder, the DeVos family and all the rest.

The Washington Post‘s Valerie Strauss reprinted part of Dr. Vitti’s remarks: “People often ask me, ‘What were you most surprised about when you took the job and started to work in the system?’  And I often say I was shocked, horrified at the lack of systems and processes for traditional public education. Traditional public education has always been, and hopefully will always be, the vehicle for social change, for social justice, for equal opportunity in this country.  And walking into the system and seeing a lack of systems and processes is a testament to the lack of belief in what children can do.”

Vitti continued: “And there is a racist element to what has happened. Children in Detroit have been treated like second-class citizens.  When a system is allowed to be run over a decade by individuals, and it’s not about one individual, but individuals that had no track record of education reform, no local governance structure to address immediate concerns and issues by the community through an elected board… and year after year of low performance, a lack of growth, drop in enrollment, facilities that are not kept up, that would never ever happen in any white suburban district in this country.  And that is a testament of race.  Because this country would not allow that. We see signs of that in Flint and we saw signs of that in New Orleans after the flood and we have multiple examples of this.”

Vitti decried any lack of real commitment to equity: “And there is oftentimes very strong rhetoric about equal opportunity in public education.  It’s time to actually support it financially.  But even with the right policy, even when we talk about “we need accountability”….  I often say to people, when you inject the accountability, what are you doing for support?  Because for best practices for every unit of accountability, there has to be a corresponding unit of support.  And for schools that are deeply struggling basically because of poverty, there needs to be extra degrees of support. And that’s what we haven’t figured out.”

Vitti’s comments were part of a panel that included Detroit Mayor Mike Duggan and other community leaders. Detroit Free Press columnist Rochelle Riley, who was the panel’s moderator, asked Vitti to speak from the point of view of an 8-year-old third-grader and to tell the audience what the third-grader would say about his school. She reports Vitti’s answer: “I want the same thing that your child wants… I may not have your privilege. I may not have the color of your skin.  I may live in a different ZIP code. But I want the exact same thing you want for your son, your daughter, your grandchild, your niece, your nephew. That’s what I want.”

In Richmond…

In the Washington Post, Valerie Strauss reports on another sign of evolving attitudes. Richmond’s new school superintendent, originally an acolyte of Michelle Rhee and her use of test scores to evaluate teachers, has not taken to Richmond the IMPACT teacher evaluation system he helped develop for use in the schools of Washington, D.C.  Jason Kamras became superintendent of the Richmond Public Schools late in 2017.

Strauss describes how IMPACT worked in Washington, D.C. when Kamras was serving under Rhee: “Kamras helped create and implement an assessment system for teachers and other adults in D.C. schools that was on the leading edge of a movement to evaluate teachers by student standardized test scores. Those evaluations were used to reward and punish educators. Michelle Rhee, then chancellor of the D.C. school system, was a leader of that movement, and the teacher evaluation system they developed, called IMPACT, was a national model embraced in part by other states. Launched in 2009, IMPACT’s key evaluative elements were student standardized test scores and the results of classroom observations from ‘master’ teachers… For several years, all adults in a school building were graded in part on student test scores, including the custodial staff and the people who worked in the lunch room, the idea being that everybody contributed to the school’s climate.  IMPACT used methods of data analysis that were not considered especially reliable or valid for high-stakes purposes, but supporters of test-based school reform didn’t seem to mind.”

You may remember that in 2014, the American Statistical Association rejected Value-Added-Measures (VAM) evaluations of teachers by their students’ scores.  The ASA declared: “VAMs are generally based on standardized test scores, and do not directly measure potential teacher contributions toward other student outcomes. VAMs typically measure correlation, not causation: Effects—positive or negative—attributed to a teacher may actually be caused by other factors that are not captured in the model.  Under some conditions, VAM scores and rankings can change substantially when a different model or test is used….”  And in 2015, the American Educational Research Association warned strongly against over-relying on students’ standardized test scores for evaluating their teachers.

It is one thing for psychometric experts to caution about the use of students’ test scores for judging teachers, and quite another for a practicing school district leader who helped design IMPACT for Washington, D.C.’s schools to reject such a system when he later becomes a school superintendent. Valerie Strauss quotes what Kamras recently said when he was asked about whether he would bring the system he helped design for Washington, D.C. to his new school district in Richmond: “My thinking on this has evolved… The idea of high expectations for everyone—myself, teachers, students—is certainly something that I believe in and something that will be a part of my leadership here… But how that looks concretely is something we need to explore with our educators, students and families.  What I can say for certain: I am not bringing IMPACT to Richmond.”

Teachers in AZ, WV, OK, KY, and CO Turn Narrative from Test Score Outcomes to Shortage of Inputs

In the New Yorker, E. Tammy Kim summarizes the meaning of the mass walkout by teachers last week in Arizona:

“In Arizona and other states where teachers have recently gone on strike, pay is a central issue: the average American teacher earns five percent less than he did in 2009. (In Arizona, the average teacher salary fell from fifty-three thousand to forty-seven thousand dollars in that time.) But the protests are about more than salaries. In recent years, educators have been blamed by politicians and parents for an array of social problems, from bankrupt municipal pensions to low graduation rates in poor neighborhoods.  Standardized testing has constrained teacher autonomy and creativity, and charter and private schools have competed more aggressively for government funds…  Per-student spending has fallen fourteen percent in the past decade, and some two thousand classrooms have no permanent instructors.  Between 2010 and 2015, Arizona’s rate of teacher turnover was twenty-three percent in traditional public schools and thirty-three percent in charters….”  And last week it was exposed that school districts in Arizona have been importing teachers from the Philippines on J-1 temporary guest worker visas.

In Arizona the strike went on days after it was scheduled to conclude because, while Governor Doug Ducey had promised teachers a 20 percent raise by 2020, it became clear his promise had been hollow. A Republican supermajority in both houses of the legislature blocked Ducey’s promise when it came time, late in the week, to pass a budget.  The Huffington Post‘s labor writer, Dave Jamieson reports: “The budget bill gives teachers a 9 percent pay raise next year, which, combined with a 1 percent raise already given, gets them halfway to the 20 percent hike they have called for. Ducey has promised that the second installment will come by 2020, though that is not guaranteed by the package he signed. The plan steers bulk money to districts and gives them the discretion to dole out the raises as they see fit, meaning not all teachers will receive the same percentage pay bump.  An analysis done by the Arizona Republic found that a minority of districts under the plan will not receive enough money to give all their teachers 20 percent raises.  The bill also hikes state spending on schools by $200 million per year more than Ducey originally proposed at the start of the year.  Still, it comes up well short of the walkout organizers’ demand that funding be restored to 2008 levels, adjusted for inflation.” There is not enough to make classes smaller, add enrichments, or relieve outrageous case loads for counselors.

And the way Arizona legislators raised enough funds in the budget bill to pay for investing in the public schools reflects the anti-tax bias of Arizona’s legislative majority.  The NY TimesDana Goldstein explains, “In Arizona, as in Oklahoma, legislators refused requests to raise income taxes on the wealthy, and instead turned to a hodgepodge of revenue sources that are likely to hit a wide range of voters… Mr. Ducey, a first-term Republican facing re-election, ran for governor promising never to raise taxes, and has said his budget keeps that commitment. In addition to an $18 car registration fee, a plan to shift the costs of several school desegregation plans to local property taxpayers from state government is expected to raise $18 million, in part by increasing property taxes in some low-income school districts.”

The attitude of many legislators is reflected in amendments offered as part of the budget bill. For Think Progress, Casey Quinlan reports: “Republicans offered amendments in response to the walkouts, which included an amendment from Rep. Kelly Townsend, which would have prohibited teachers in public schools from talking about or showing their political views during classroom time, and fined those who did.  She also proposed an amendment to make it illegal to close schools with the exception of natural disasters and other dangers…. Both amendments failed.”

While walkouts by school teachers this spring have not always yielded the level of victory in terms of salary increases and investments like enough money to lower ballooning class sizes, an analysis for The American Prospect concludes that teachers have been able to challenge the anti-tax, accountability-driven rhetoric of the past two decades: “Perhaps the most exciting aspect of this latest round of teacher labor power is that their demands are broad and inclusive. Even though the teachers who have gone on strike or are considering it are paid well below the average and have terrible benefits, they have put the focus of their demands on their students’ needs, on improving classroom quality and increasing classroom resources.  In doing so they made clear that winning a raise for themselves would be insufficient—they have demanded a significant investment in children as well as a win on the ‘bread and butter’ issues.”

This spring, school teachers walking out across a number of states have successfully highlighted a widespread, catastrophic shortage of financial inputs and challenged our society’s myopic focus on test-score outcomes