Rand Corp. Report Says Grading Teachers by Student Scores Doesn’t Work; Ohio Law Will Diminish Use of Student Scores for Evaluating Teachers

In 2009, the Bill & Melinda Gates Foundation launched a huge project to demonstrate that evaluating teachers by their students’ standardized test scores would improve education and especially the education of “low-income, minority” students. Now the Gates Foundation has paid for a huge Rand Corporation study that showed its original experiment didn’t work. Although the Gates Foundation can move on to testing another hypothesis, its prescription for grading teachers has done immeasurable damage by injecting econometric teacher evaluation into the laws of many states. It will take a long time for the 50 state legislatures to clean up laws based on a mistake.

Chalkbeat‘s Matt Barnum describes the original plan: “Barack Obama’s 2012 State of the Union address reflected the heady moment in education. ‘We know a good teacher can increase the lifetime income of a classroom by over $250,000,’ he said. ‘A great teacher can offer an escape from poverty to the child who dreams beyond his circumstance.’ Bad teachers were the problem; good teachers were the solution. It was a simplified binary, but the idea and the research it drew on had spurred policy changes across the country, including a spate of laws establishing new evaluation systems designed to reward top teachers and help weed out low performers.  Behind that effort was the Bill and Melinda Gates Foundation… Now, new research commissioned by the Gates Foundation finds scant evidence that those changes accomplished what they were meant to: improve teacher quality or boost student learning.  The 500-plus page report by the Rand Corporation… details the political and technical challenges of putting complex new systems in place…”

The Gates Foundation not only launched a giant experiment without an adequate research base, but it also leveraged the investment of public dollars and used its own lobbying might to influence public policy. The Obama administration conditioned qualification for Race to the Top grants on the use of students’ standardized test scores in teachers’ evaluations and later made the same requirement for states to qualify for No Child Left Behind waivers.

The Washington Post‘s Valerie Strauss details the history: “Put this in the ‘they-were-warned-but-didn’t-listen’ category.”  She describes the project launched in Hillsborough County (Greater Tampa), Florida, Memphis, and Pittsburgh along with four charter management organizations: “The Bill & Melinda Gates Foundation pumped nearly $215 million into the project while the partnering school organizations supplied their own money, for a total cost of $575 million.”  Federal policy makers jumped into the mix: “The Obama administration, through its Race to the Top initiative, dangled federal funds in front of states that agreed to establish teacher evaluation systems using test scores to varying extents.  And Gates funded his ‘Empowering Effective Teachers’ project with the aim of finding proof that such systems could improve student achievement…  (M)ost states adopted test-based teacher evaluation systems.  In a desperate attempt to evaluate all teachers on tested subjects—reading and math—some of the systems would up evaluating teachers on subjects they didn’t teach or on students they didn’t have. Some major organizations questioned them, including the American Statistical Association…. And so did the Board on Testing and Assessment of the National Research Council.”

Strauss quotes the conclusion of the Rand Corporation’s huge new assessment of the experiment: “Overall, the initiative did not achieve its stated goals for students, particularly LIM (low-income minority) students. By the end of 2014-2015, student outcomes were not dramatically better than outcomes in similar sites that did not participate in the IP (Intensive Partnerships) initiative. Furthermore, in the sites where these analyses could be conducted, we did not find improvement in the effectiveness of newly hired teachers relative to experienced teachers; we found very few instances of improvement in the effectiveness of the teaching force overall; we found no evidence that LIM students had greater access than non-LIM students to effective teaching; and we found no increase in the retention of effective teachers, although we did find declines in the retention of ineffective teachers in most sites.”

What the Rand Report fails to calculate is the collateral damage. It is well known that, in Hillsborough County, Florida, the Gates Foundation suspended its study before it had been completed—leaving the school district itself to cover a significant part of the cost. But beyond Hillsborough County, the consequences were long lasting as state legislatures, lured by Race to the Top funding and the need to qualify for No Child Left Behind waivers, passed laws basing teachers’ evaluations on students’ standardized test scores. When, in December of 2015, Congress replaced No Child Left Behind with the Every Student Succeeds Act, it removed the requirement that states use  students’ test scores in teachers’ evaluations, but the laws the states had put in place to meet federal requirements remained.

For example, only last week did the Ohio Legislature act to reduce the role of students’ test scores in the state teacher evaluation system. Finally—before going on a 2018 summer recess, the Ohio lawmakers passed a new statute reducing the weight of students’ standardized tests in the formal evaluation of teachers. The law passed with bipartisan support, and it is hoped that Governor John Kasich will sign it.

Last Sunday, the Columbus Dispatch‘s Jim Siegel reported that Ohio has been basing 50 percent of teachers’ ratings on students’ standardized  test scores .  Keep in mind that it is now 2018, and Ohio, like many other states, has still been using a plan that the Rand Corporation has now declared ineffective for measuring the quality of teachers.

Siegel quotes Jonathan Juravich, the 2018 Ohio Teacher of the Year, describing the new system: “No longer… (will) student growth measures be used as a disconnected evaluation factor linked to an arbitrary weighted percentage.”

Ohio is also finally doing away with “shared attribution,” according to Siegel: “Changes include doing away with shared attribution—growth measures attributed to a group of teachers that, critics say, does not accurately measure individual performance….”

State Superintendent Paolo DeMaria is quoted describing the new law: “Most importantly, we want our teachers on a path of continuous improvement, and with these changes the system places a greater focus on improvement in teacher practices that lead to better outcomes for students.”

The Bill & Melinda Gates Foundation and Obama administration’s collaborative scheme to evaluate teachers econometricaly has undermined the morale of school teachers and contributed to a climate in which teachers have been blamed unfairly when test scores don’t rise. Contrast the Gates theory, now rejected by the Rand Corporation report, with the research of Harvard’s Daniel Koretz, who explains how the test scores—so central to the school accountability movement—don’t really measure the quality of the schools or specific teachers, but instead primarily reflect the aggregate economic level of a school’s families and neighborhood:

“One aspect of the great inequity of the American educational system is that disadvantaged kids tend to be clustered in the same schools. The causes are complex, but the result is simple: some schools have far lower average scores…. Therefore, if one requires that all students must hit the proficient target by a certain date, these low-scoring schools will face far more demanding targets for gains than other schools do. This was not an accidental byproduct of the notion that ‘all children can learn to a high level.’ It was a deliberate and prominent part of many of the test-based accountability reforms…. Unfortunately… it seems that no one asked for evidence that these ambitious targets for gains were realistic. The specific targets were often an automatic consequence of where the Proficient standard was placed and the length of time schools were given to bring all students to that standard, which are both arbitrary.” (The Testing Charade; Pretending to Make Schools Better, pp. 129-130)

Ohio is now joining other states trying to undo the damage. Writing for the Stamford Advocate, Wendy Lecker, a columnist and attorney for the Education Law Center, explains: “Technology writer Eugene Morozov coined the term ‘solutionism’: a pathology that recognizes a problem based on one criterion only… solvable with a simple, preferably technological, solution. Solutionists operate with a myopic hubris, believing that if they get their simple fix right, as the chair of Google once claimed, ‘we can fix all the world’s problems.'”

The story of America’s nine year experiment with rating teachers by their students’ test scores ought to teach us to beware solutionists with gobs of money and the power to seduce policy makers.

(This blog has tracked education philanthropy from the Bill & Melinda Gates Foundation here.)

NY Times’ Motoko Rich Fails to Examine the Deeper Issues in No Child Left Behind Waivers

At the beginning of her NY Times article yesterday,  Washington State, Political Stand Puts Schools in a Bind, Motoko Rich, the NY Times‘ education reporter, describes a Washington state public school that perfectly epitomizes Arne Duncan’s test-and-punish school turnaround narrative: replace the principal and half the staff, lengthen the school day and year, and use a $3 million federal grant to retrain teachers.  The school clearly won a School Improvement Grant to be turned around, and it seems to have miraculously improved its math scores, though we are not told, really, exactly how the transformation occurred and how the children’s experience at school changed.

Rich tells the story of Lakeridge Elementary School to make her point about Washington state’s foolish choice to lose its No Child Left Behind Waiver by refusing to comply with one of the conditions set by Education Secretary Arne Duncan’s Department of Education, that to qualify for a waiver, a state must use scores from a statewide standardized test as part of its plan to evaluate teachers.  She quotes Michael Petrilli, of the far-right Fordham Foundation, “We’re punishing schools and educators and arguably kids, because state policy makers don’t want to do what the Education Department demands.”

Rich explains that the No Child Left Behind Law is now known to be deeply flawed.  She describes the public schools in Washington state that have lost their waiver as now forced to be “held to an outdated benchmark that is all but impossible to achieve,” and she describes the consequences.  Washington’s schools that were unable to prove all their children proficient by 2014 are to be labeled failures.  Because the law embodied a utopian aspiration that all American children were to be proficient by 2014, virtually all of the state’s schools and all schools across America are—according to No Child Left Behind—now “failing” schools.  And under the old rules of No Child Left Behind—rules that have been eliminated for states winning waivers—Washington must set aside Title I funding for children who wish to transfer out of their “failing” school and for privatized tutoring for parents who want their children in failing schools tutored.  Rich writes that the loss of the waiver and the naming of all schools as “failures” has demoralized teachers, confused parents, and perplexed educators and lawmakers.

What Rich does not do in her article is examine the U.S. Department’s conditions for states to qualify for waivers.  Rich does not explore questions about the use of state standardized test scores for evaluating teachers, despite the presence of a healthy debate about these subjects in both research literature, including a warning just last spring from the American Statistical Association, and the press.  The policy of using students’ scores to rate teachers has been widely questioned because standardized test scores were not designed to evaluate teachers.  Neither have students’ standardized test scores shown themselves to be reliable or stable from year to year as a representation of the performance of particular teachers.  Rich’s article does not acknowledge that such widespread concern may have motivated some legislators in Washington state to vote against the use of standardized test scores for evaluation of teachers.  She merely treats the loss of the waiver as a catastrophe that should have been avoided.

Although Motoko Rich is the NY Times education reporter, she is not an educator.  Her biography on the newspaper’s website says: “Motoko Rich writes about national K-12 education for the New York Times.  Prior to that, she covered the national economy, writing about work force training, unemployment, housing and retirement.  She also covered the book publishing industry for four years… Before joining The Times, Motoko worked as a staff reporter at The Wall Street Journal for six years, in Atlanta and New York.  She started her career as a reporter at The Financial Times in London.”  One wonders if Rich might have covered the story of the Department of Education’s denial of Washington state’s waiver differently if she had a background in education.

In the context of Rich’s article in yesterday’s NY Times, it is worth remembering the letter sent out to all parents in Vermont just two months ago, on August 6, 2014.  In this letter, Rebecca Holcombe, the state’s Secretary of Education writes:

“Under the No Child Left Behind Act (NCLB), as of 2014, if only one child in your school does not score as ‘proficient’ on state tests, then your school must be ‘identified’ as ‘low performing’ under federal law.  This year, every school whose students took the NECAP tests last year is now considered a ‘low performing’ school by the US Department of Education… The Vermont Agency of Education does not agree with this federal policy, nor do we agree that all of our schools are low performing.” (emphasis Holcombe’s)

Holcombe describes positive accomplishments in the public schools across the state of Vermont, but then reminds parents why she is sending this strange letter telling them their school is a failure: “Nevertheless, if we fail to announce that each Vermont school is ‘low performing,” we jeopardize federal funding for elementary and secondary education… This policy does not serve the interest of Vermont schools, nor does it advance our economic or social well-being.  Further, it takes our focus away from other measures that give us more meaningful and useful data on school effectiveness.”

Holcombe explains why Vermont has chosen never to apply for a waiver: “Most other states have received a waiver to get out from under the broken NCLB policy.  They did this by agreeing to evaluate their teachers and principals based on the standardized test scores of their students… We chose not to agree to a waiver for a lot of reasons, including that the research we have read on evaluating teachers based on test scores suggests these methods are unreliable in classes with 15 or fewer students, and this represents about 40-50% of our classes.  It would be unfair to our students to automatically fire their educators based on technically inadequate tools.  Also, there is evidence suggesting that over-relying on test-based evaluation might fail to credit educators for doing things we actually want them to do, such as teach a rich curriculum across all important subject areas, and not just math and English language arts.”

Finally, Holcombe tells parents how she thinks they themselves ought to investigate the quality of their children’s schools.  She suggests parents talk with their children about school, and she suggests questions to guide these conversations, questions guaranteed to engage the parents in what’s happening with their children each day at school:

  • “What evidence does your school provide of your child’s growing proficiency?
  • Is your child developing the skills and understanding she needs to thrive in school and
    the community?
  • Are graduates of your school system prepared to succeed in college and/or careers?
  • Is your child happy to go to school and engaged in learning?
  • Can your child explain what he is learning and why? Can your child give examples of
    skills he has mastered?
  • Is your child developing good work habits? Does she understand that practice leads to
    better performance?
  • Does your child feel his work in school is related to his college and career goals?
  • Does your child have one adult at the school whom she trusts and who is committed to
    her success?
  • If you have concerns, have you reached out to your child’s teacher to share your

While I am not particularly fond of the cliche about people who can’t think outside the box, I think in today’s article about Washington state’s loss of its No Child Left Behind Waiver, Motoko Rich fails to think outside the box by uncritically accepting all the assumptions behind Arne Duncan’s No Child Left Behind Waivers and the entire test-and-punish accountability narrative about school reform that has been the foundation of federal education policy under Presidents George Bush and Barack Obama.

Rebecca Holcombe, Vermont’s Secretary of Education is far more creative in the letter she sent to all of Vermont’s parents at the beginning of the school year.  Her sending this letter is a bureaucratic requirement of the flawed and very damaging No Child Left Behind Law, but she seized the requirement as an opportunity to educate parents about the serious consequences of rating teachers by students’ scores and the ludicrous “failing” label being attached to Vermont’s schools.   And her letter categorically affirms her confidence in Vermont’s teachers and her expectations for Vermont’s parents to engage actively in their children’s education.

I would prefer to see education coverage in the NY Times explore far more deeply the implications of federal policy, including the federal testing law No Child Left Behind, the competitive grant programs of President Obama’s Department of Education like  Race to the Top and School Improvement Grants, and the No Child Left Behind Waivers.  Vermont’s Secretary of Education is clearly less concerned about her state’s lack of a waiver than Motoko Rich is about the loss of Washington state’s waiver.

Congressional Dithering and No Child Left Behind Waivers Subvert Democratic Checks and Balances

In this morning’s New York Times, Robert Reich describes a serious consequence of gridlock in Congress: the loss of democracy.  “Political decision making has moved to peripheral public entities, where power is exercised less transparently and accountability to voters is less direct.”  Reich explains the consequences for the management of the economy, for the balance of power with the Supreme Court, for our nation’s capacity to slow climate change, and for the balance among state and federal responsibilities.

Democracy is also being abrogated in another policy area less reported these days: public education.  And while this month the most obvious examples of threats to democratic governance of the public schools are at the local level—the power of emergency managers appointed by Michigan’s governor to privatize whole school districts, to override labor contracts, and even to close school districts while elected school boards stand by to watch—there are also very serious issues of federal overreach by the U.S. Department of Education at this time when Congress cannot agree to act.

Although Congress is supposed to reauthorize the federal Elementary and Secondary Education Act (ESEA) every five years, the 2002 version, called No Child Left Behind, is now six years overdue to be rewritten.  Still Congress dithers.  No Child Left Behind (NCLB), has now shaped federal public education policy for eleven years.

U.S. Secretary of Education Arne Duncan and the U.S. Department of Education have dealt with Congress’s inability to build consensus around an ESEA reauthorization by managing federal education policy administratively, granting waivers to states from onerous consequences of NCLB if the states submit proposals that conform to priorities set by the Department.  The waivers are being granted without any oversight by Congress.

No Child Left Behind waivers permit states to escape the unworkable and utopian Adequate Yearly Progress requirement that schools post ever higher annual test scores or be declared “failing.”  The waivers permit states to stop setting aside federal Title I money for transportation to support school choice by which students in so-called “failing” schools can move to other schools.  And the waivers permit states to get rid of the controversial Supplemental Education Services, a tutoring program that has diverted federal funds away from in-school Title I programming to outside, after-school tutoring programs that have too often profited unscrupulous providers.

While there is now widespread agreement that Adequate Yearly Progress, school transfers, and Supplemental Education Services harmed public schools, there is growing concern about the administrative—not democratic—way the Department of Education has sought to provide relief.  To qualify for waivers, states’ applications had to include adoption of one of two versions of the Common Core Standards, including new and controversial standardized tests like those whose scores New York reported earlier this week.  To qualify for waivers states have also had to agree to incorporate students’ standardized test scores in the evaluation of teachers.  These are Departmental—not Congressional—priorities.

The latest flap about the side-stepping of democracy through the NCLB waivers happened just this month.  After the Department of Education rejected California’s application for a waiver last year because California’s proposal did not meet the Department’s requirements for evaluating teachers, eight local California school districts including very large districts like Los Angeles, Fresno, and Long Beach, did an end run around their state department of education to apply for their own waiver as a consortium of school districts.  Writing for Education Week, Michele McNeil reported that on August 6,  the consortium of districts received a one-year NCLB waiver, which can be renewed on the condition that they fully implement the school-rating system and teacher-evaluation plans the eight districts proposed.

How does the federal granting of a NCLB waiver to a group of school districts interfere with democracy?  The issue is constitutional.  State constitutions, not the U.S. Constitution, grant the authority for the establishment and regulation of local school districts.  According to  McNeil, “…this waiver is unprecedented for its scope, and for how it changes the dynamic between districts, the state, and the federal government.”

In a follow-up commentary, McNeil wonders, for example, about a new oversight panel the eight California districts created to police themselves: “Will the new ‘oversight’ panel provide enough oversight… as a state would provide in the traditional accountability relationship?  …the oversight panel’s authority (such as it is) is derived from a really squishy place.  This new panel’s power is not rooted in law, or state board regulations, but in a waiver agreement between the feds and these districts.”

You may wonder: Isn’t this too arcane and too far “out in the policy weeds” for most of us to care about it?  That’s just the problem.  While as individuals we are unlikely to be able to track and impact policy at this level, our democracy was set up with three branches of government to balance and check each other. When Congress (the legislative branch) fails to make the laws (what our civics classes taught us Congress is supposed to do), when it cannot reach any sort of compromise to reauthorize the Elementary and Secondary Education Act, there is no check on what is today the overreach of the executive branch, in this case represented by the U.S. Department of Education.

To its credit, the Department of Education has made an effort to alleviate the damage of the No Child Left Behind Act.  The problem is that the Department’s method of trying to fix NCLB without adequate Congressional oversight increasingly interferes with the democratic process.  Democratic governance of public education is essential for ensuring that  families can secure the education to which their children have a right.