The Problems of Outcomes-Based School Accountablity

I am so tired of the narrative of “failing” schools—a story which is always accompanied by the story of “failing” teachers and their “failing” students. I find myself trapped in arguments about this subject in places where I don’t want to be talking about it—with good friends and relatives around dinner tables, at parties, during intermissions at concerts.  And even though I know a lot about the topic, I can never really win the argument, because the people with whom I am discussing it have always read about it in the newspapers where the test score comparisons are published.  This narrative has no reference whatsoever to what is happening in particular classrooms or particular schools or school districts. Many people with strong opinions have not been in a public school for decades.

The real subject here, of course, is what education is.  But the conversation instead is always a comparison of test scores as a proxy for the quality of a community and its schools.  One wants to get at the the real meaning and purpose of outcomes-based, test-measured school accountability, but that is hard to do in a casual conversation.  And underneath any conversation about “failing” schools are lots of realities about segregation—by class and also by race.

Research has documented growing economic inequality and segregation by family income. Sean Reardon, a Stanford University sociologist, used a massive data set to document the consequences of widening economic inequality for children’s outcomes at school. Reardon showed that while in 1970, only 15 percent of families lived in neighborhoods classified as affluent or poor, by 2007, 31 percent of families lived in such neighborhoods. By 2007, fewer families across America lived in mixed income communities. Reardon also demonstrated that along with growing residential inequality is a simultaneous jump in an income-inequality school achievement gap. The achievement gap between the children with income in the top ten percent and the children with income in the bottom ten percent, was 30-40 percent wider among children born in 2001 than those born in 1975, and twice as large as the black-white achievement gap.

Then there is segregation by race.  Recently I had occasion to revisit a 2014 article by Richard Rothstein on the long-term effects of racism in our caste society: “Even for low-income families, other groups’ disadvantages—though serious—are not similar to those faced by African Americans. Although the number of high-poverty white communities is growing (many are rural)… poor whites are less likely to live in high poverty neighborhoods than poor blacks.  Nationwide, 7 percent of poor whites live in high-poverty neighborhoods, while 23 percent of poor blacks do so. Patrick Sharkey’s Stuck in Place showed that multigenerational concentrated poverty remains an almost uniquely black phenomenon; white children in poor neighborhoods are likely to live in middle-class neighborhoods as adults, whereas black children in poor neighborhoods are likely to remain in such surroundings as adults.  In other words, poor whites are more likely to be temporarily poor, while poor blacks are more likely to be permanently so…. Certainly, Hispanics suffer discrimination, some of it severe… but the undeniable hardship faced by recent, non-English speaking, unskilled, low-wage immigrants is not equivalent to blacks’ centuries of lower-caste status. The problems are different, and the remedies must also be different….”

Our public schools across America are situated in very different communities—small towns of all sorts, small cities, big cities, poor neighborhoods, rich neighborhoods—schools whose children speak English and other schools where for many children, English is not the primary language. Within all this diversity, however is the reality of segregation by race, and according to Reardon, growing segregation by family income.  In more and more places across America, children live in pockets of extreme poverty or pockets of extreme affluence.   While teachers can work with all the outside-of-school variables the children bring to their classrooms—including intensifying segregation by income, there is much of the experience of each child that schoolteachers cannot control. Children are neither blank slates nor empty vessels into which knowledge can be poured.

On Sunday morning, the subject of “failing” schools and “failing” teachers and “failing” students arrived on my doorstep in Patrick O’Donnell’s Plain Dealer article about what key Ohio legislators believe is dangerous: that too many students graduated from high school this year because of “soft” alternative pathways to graduation.  These alternative pathways were only for the 2018 school year— because educators successfully lobbied that the new graduation tests were so hard that all sorts of young people would be denied graduation.  O’Donnell tells us the educators’ fears were well grounded: “More than a third of this spring’s high school graduates from some urban areas would never have received their diplomas under Ohio’s new graduation requirements, were it not for some temporary and easier ‘pathways’ added to avert a statewide graduation ‘crisis.’ In Akron and Columbus, new test-based requirements would have prevented more than a third of this year’s graduates from marching at ceremonies in caps and gowns. In Cleveland, the impact of the controversial new standards would have been even stronger. The higher expectations would have wiped out diplomas for nearly half of the seniors who received them. Those students instead graduated using special one-time alternate pathways created just for this year to ease the transition to the new standards.”

This is the “failing” schools narrative at work.  If you can find a way to read this without noticing legislators’ indictment of those “failing” schools in Akron and Columbus and especially in Cleveland, Rep. Andy Brenner, Chair of the Ohio House Education Committee, will correct you: “What’s going on that they’re not able to get kids up to being college and career-ready?”

Contrast the understanding of education by outcomes-based education accountability hawks like Andy Brenner with the understanding of learning depicted in the new documentary film about Fred Rogers, Won’t You Be My Neighbor?  Mr. Rogers—influenced by prominent experts in child development like Barry Brazelton and Margaret McFarland—defined education as relating to children, listening to children, and responding to children’s questions and needs and concerns.  For Mr. Rogers, education was not teacher- or school-driven but instead happened in relationship—building a child’s understanding from the foundation within the child. A teacher guides instead of lecturing; a teacher responds instead of driving material into a child’s brain.  A teacher starts where the child is.

Contrast such a developmental understanding of teaching and learning with the model framed by an outcomes-driven reformer intent on pouring in enough testable material to get enough adolescents to pass the tests and produce a career-ready cohort from each high school. The outcomes-based reformer worries about the so-called quality of the diploma; the educator in Mr. Rogers’ mold considers beginning where the child is and helping that child realize her or his promise.

In this year’s very best book on education, Harvard’s Daniel Koretz describes the flaws in outcomes-based school accountability. The title explains the book’s importance for our times: The Testing Charade: Pretending to Make Schools Better.

Koretz is a psychometrician.  While he is neither a child psychologist nor a specialist in child development, Koretz describes the omission of all sorts of essential parts of education, including the kind of teaching Fred Rogers believed was important: “A… critical failure of the reforms is that they left almost no room for human judgment. Teachers are not trusted to evaluate students or each other, principals are not trusted to evaluate teachers, and the judgment of professionals from outside the school has only a limited role. What the reformers trust is ‘objective’ standardized measures…. (T)he focus of reform in the United States has been to rely as much as possible on standardized measures and to minimize human judgment, even though the result was to leave a great deal of what is most important unmeasured—and therefore to give educators no incentive to focus on it.  This is one of the most fundamental flaws of test-based accountability and one of the most significant reasons for its failures.” (The Testing Charade, pp. 34-35)

Koretz explains how outcomes-based education is undermining our very understanding of education—and undermining teaching: “Not only is bad test prep pervasive. It has begun to undermine the very notion of good instruction… One of the rationales given to new teachers for focusing on score gains is that high-stakes tests serve a gatekeeping function, and therefore training kids to do well on tests opens doors for them… Whether raising scores will improve students’ later success… depends on how one raises scores.  Increasing scores by teaching well can increase students’ later success… In the early days of test-based accountability, some observers worried that educators were coming to confuse the test with the curriculum… Some of today’s teacher educators, however, make a virtue of this mistake. They often tell new teachers that tests, rather than standards or a curriculum should define what they teach… Why does this matter so much? To start, it encourages reallocation—that is, focusing instruction on the tested sample rather than the domain or the curriculum that it is supposed to represent… What we want is for students to gain the ability to apply knowledge and skills to problems they actually encounter—not to ensure their proficiency in applying them only to test items that look exactly like the ones they will confront in the main test at the end of the year.”  (The Testing Charade, pp. 112-116)

Finally, Koretz speaks directly to the problem in Ohio, where alternative pathways to high school graduation have been needed to ensure high school graduation for large percentages of students in the state’s poorest cities but where students in affluent suburbs with schools to which the state awards “A+” grades merely sail through the new graduation requirements. Outcomes-based education accountability hawks set benchmarks more easily reached by the privileged, but we blame the schools and teachers in poorer communities—and with high school graduation benchmarks, we penalize the students themselves.

Koretz explains: “One aspect of the great inequity of the American educational system is that disadvantaged kids tend to be clustered in the same schools. The causes are complex, but the result is simple: some schools have far lower average scores…. Therefore, if one requires that all students must hit the proficient target by a certain date, these low-scoring schools will face far more demanding targets for gains than other schools do. This was not an accidental byproduct of the notion that ‘all children can learn to a high level.’ It was a deliberate and prominent part of many of the test-based accountability reforms…. Unfortunately… it seems that no one asked for evidence that these ambitious targets for gains were realistic. The specific targets were often an automatic consequence of where the Proficient standard was placed and the length of time schools were given to bring all students to that standard, which are both arbitrary.” (The Testing Charade, pp. 129-130)

Sometimes I think I ought to carry a copy of Koretz’s book in my purse, though I’d be written off as such a bore if I were to pull it out and read from it when somebody at a party begins bragging about their school—rated “A+” by the state of Ohio—while the school across town gets an “F.”  Everybody ought to take Daniel Koretz’s book to read at the beach this summer.

7 thoughts on “The Problems of Outcomes-Based School Accountablity

  1. Old story

    We are living on the tail end of an old story… a story that extolled the virtues of data, metrics, analytics as tools for assessing the value of our work, our schools, and, distressingly, our students.

    Such stories become stories as they are repeated and gradually accepted as truth.

    Old stories die hard because of how deeply ingrained they have become through repetition and our tendency toward confirmation bias – i.e., our tendency to assign greater validity to information that confirms our beliefs.

    Inaccurate tales become stories because they are unchallenged and may have some resonance with our experiences.They grow in acceptance due to laziness and/or ineffective challenges.

    The “old story”, the story of our people, extols the virtue of hard work, doing well in school, getting into a good college, obtaining a college degree, getting a good job with the accompanying secure future. That story included the myth that such opportunity was equally open to all Americans as well as a healthy portion of blame aimed at those whose experiences contradicted the validity of that story.

    Now, not only the poor and people of color are challenged to find the validity of that story.Far more of us are confronted on a regular basis with a challenge to it.Many have watched their children work hard, do well in school, be successful in good colleges and find no jobs.They have watched the security of pensions disappear. They have watched the promise of progress and development rape the land and threaten our continued existence.Many have recognized the death throes of the old story.

    The story of accountability has been told and retold so frequently that it has become a part of the fabric of the old story.A chapter in that story must be devoted to human arrogance. This arrogance is filled with irony… an irony that names the flagship legislation No Child Left Behind, while designed to leave millions of poor children behind and labeled, along with their schools, as failing.c

  2. Pingback: 2018 Medley #18 – Live Long and Prosper

  3. It is tragic that there continues to be so much misunderstanding about the nature of standardized tests, the scoring, and the ranking/labeling of students, teachers, and schools. I’ve just been reading Gerald Bracey’s book Education Hell: Rhetoric vs. Reality: Transforming the Fire Consuming America’s Schools. This book was published in 2009, before Common Core and associated testing. Bracey meticulously explains the fallacies in comparing supposed quality of teaching/learning on the basis of this kind of test. Sadly, public school bashing is not a modern phenomenon. He traces it back several hundred years, and focuses a chapter on the post-Sputnik test mania. I was about 10 years old at that time. I don’t remember much in the way of standardized testing, but I do remember the pressure put on teachers and students toward rigor and acceleration. Bracey describes how there was no evidence to connect the public schools as a reason for (the prevalent fear that) America was losing out in competitiveness. We’re still hearing this hysterical rhetoric today. No one is saying that the schools are fine and no change is necessary. Depressingly, the “remedies” the ed reformers hawk are counter-productive. Those who would deny high school students a diploma because they score poorly on ill-conceived tests based on ill-conceived standards are not willing to look at the facts, and too easily manipulated by corporate interests.

  4. in my city I confronted a person at a meeting of our sub-committee that for 2 years he has been going around saying “The Haverhill Schools are lousy”… I ask him, do you have any kids in the schools? No… It is this focus on “test and punish” and the people who use invalid test scores (that are not reliable) and bring fury to the parents instead of working on sensible programs or solutions. Meanwhile, our Fair Share amendment in MA is knocked out by the Courts and the legislature (and governor) fail to pass the Foundation Budget — through bureaucratic stalling — so that the outdated 1993 formula exists and it penalizes the inner city schools which are in “gateway” cities all the time these characters go around saying “the schools are lousy”. There were 5 witnesses to my vehement proclamation but everyone backs away….. I wish they would realize what we do to students in the schools is similar to what we do to immigrants and what they do to the schools they also do to the hospitals and that is YOUR HEALTH CARE…. same cruel and sadistic policies.

  5. This report, “How High Is the Bar?” (see link below) is an analysis of what most states including Ohio are using to develop standards and test items: NAEP and Common Core. Despite “revisions,” Ohio’s learning standards are the Common Core–I was on one of the ODE revision committees. The report concludes that in looking at international test results, most students even in high-scoring countries would have difficulty passing our tests. This is an important contribution to the discussions and continued false narratives about failing schools and failing students.

  6. Hi Jan,

    FYI, the League is having a forum related to this topic August 29 at the Heights Lee Road Library. I’ve attached the flyer.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s