EXTRA: A Teacher’s Summary of What Proficiency Might Mean

Here, in The Insufficiency of Proficiency, Oklahoma teacher, Rob Miller speculates on what proficiency might mean. His essay specifically explores any of a number of kinds of complex (or simple) understanding that we might be aiming for—but really cannot measure—when we test “reading proficiency.”

His post is far deeper, however—about the meaning for all of us of more than 15 years of nationally mandated standardized testing. This is a fascinating essay about making educational policy based on a reductive theory of human learning.

Miller begins with a seasonal theme:

Twas the week before Christmas, when all thro’ the state
All the children were stirring, eager to learn their fate;
Their test scores from April would soon be delivered,
I hope I’m proficient the children all quivered;
The wait’s been soooo long…my hands are all sweaty
I need to know now … am I college and career ready?

His piece is also a seasonal reflection for the new year. How many more years will it take us to recognize the limitations of test scores for measuring what we really want children to know?

Advertisements

Test Scores Poor Indicator of Students’ Life Outcomes and School Quality: New Consensus?

According to Education Next, “Jay P. Greene is endowed chair and head of the Department of Education Reform at the University of Arkansas and a senior fellow at the Manhattan Institute.” This Arkansas “Department of Education Reform” epitomizes the far-right think tank posing as a university department. For years, Greene has been at the center of education “reform” orthodoxy, and yet today he is questioning one of its primary tenets—the use of short term test score gains as the primary measure of school improvement.  While federal policy in education as prescribed in the new Every Student Succeeds Act continues to prescribe annual testing and the requirement that states develop plans to turn around schools that can’t quickly raise scores, there seems to be growing consensus in the academic world about problems with accountability that is purely test-based.

Here is what Professor Greene posted on his blog on Tuesday:  “I’ve written several times recently about how short term gains in test scores are not associated with improved later life outcomes for students. Schools and programs that increase test scores quite often do not yield higher high school graduation or college attendance rates. Conversely, schools and programs that fail to produce greater gains in test scores sometimes produce impressive improvements in high school graduation and college attendance rates, college completion rates, and even higher employment and earnings. I’ve described at least 8 studies that show a disconnect between raising test scores and stronger later life outcomes.” Greene devotes the rest of his post to describing a new study that replicates these findings.

Greene concludes: “It’s time that people start paying a lot more attention to this pattern of a disconnect between short term test score gains and long term life outcomes. We can’t just dismiss this pattern as a fluke… If we think we can know which schools of choice are good and ought to be expanded and which are bad and ought to be closed based primarily on annual test scores gains, we are sadly mistaken.  Various portfolio management and ‘accountability’ regimes depend almost entirely on this false belief that test scores reveal which are the good and bad schools. The evidence is growing quite strong that these strategies cannot properly distinguish good from bad schools and may be inflicting great harm on students.”

Paul Tough’s new book, Helping Children Succeed, describes educational research that confirms Greene’s concerns about test-based accountability.  Tough explains the research of Kirabo Jackson: “What he found was that some teachers were reliably able to raise their students’ standardized-test scores year after year… But Jackson also found that there was another distinct cohort of teachers who were reliably able to raise their students’ performance on his non-cognitive measure.  If you were assigned to the class of a teacher in this cohort, you were more likely to show up to school, more likely to avoid suspension, more likely to move on to the next grade… Jackson found that these two groups of successful teachers did not necessarily overlap much.” (pp. 69-70)  Tough is describing teachers and schools that build intrinsic motivation and that attend to challenging students, connecting them with other students, and building autonomy. And Tough describes schools that are supportive, not punitive: “When kids feel a sense of belonging at school, when they receive the right kind of messages from an adult who believes they can succeed and who is attending to them with some degree of compassion and respect, they are then more likely to show up to class, to persevere longer at difficult tasks, and to deal more resiliently with the countless small scale setbacks and frustrations that make up the typical student’s school day.” (p. 73)

This week  the National Education Policy Center at the University of Colorado released a policy brief that cautions about condemning schools that continue to struggle rapidly to raise test scores even as they are in the midst of concerted reform and turnaround.  NEPC’s brief warns:

“Attempts to dramatically turn around schools to show quick improvements in student outcomes are often counterproductive, resulting instead in school conditions associated with persistently low performance. Many quick school turnarounds, like those initiated via the federal School Improvement Grant program, were associated with unintended, negative outcomes such as high teacher turnover, large numbers of inexperienced teachers, administrative instability, poor school and classroom climate, and socioeconomic segregation.”

NEPC instead endorses comprehensive reforms that address the multifaceted issues that are known to affect test scores: “Part of the challenge in turning around schools is that outside-of-school factors likely account for twice as much of the variance in student outcomes as do inside-of-school factors. Accordingly, the community schools approach—one of the most prominent and research-based approaches to sustained reform—addresses the academic, social-emotional, and health needs of children as well as the capacity to systemically meet these needs in communities of concentrated poverty.”

Academic reforms—beyond the social and health reforms that surround children in a wraparound Community School—also take time: “Research offers strong caution against claims of miraculous school change.  Instead, changing a school’s culture and practices in sustainable ways that improve student learning takes years of commitment by all the stakeholders in the school… Effective schools have stable leaders who support teachers…. Effective schools have teacher leadership that’s distributed through the school and that facilitates a continuous improvement cycle…. Effective schools meaningfully engage families and the community.”

The NEPC brief concludes: “Policies that demand rapid school turnaround largely ignore the complexity of reforming schools for sustainable improvement and also ignore out-of-school factors such as poverty, race, and systemic funding disparities. These mistakes arise, in part, from an imbalanced focus on test scores that can be gamed to show temporary and shallow improvements.  Instead, policies should look to a broad range of appropriate interim indicators to assess whether a school is improving.”

All this research points to the need for a radical shift in America’s domestic policy agenda. Our society will be required to reverse tax policy that has slashed public education budgets across many states and also to invest in the institutions, including public schools, that serve America’s poorest children living in concentrated and sometimes extreme poverty across our urban centers.

NY Times Misses the Point in “Counterfeit Diploma” Editorial

On New Year’s Eve, the NY Times published a lead editorial lamenting our public schools, quoting accountability hawk organizations—Achieve and the Education Trust, and blaming teachers’ unions (as usual) for what it calls The Counterfeit High School Diploma.  The editorial explains that if we were to weaken standardized testing (I guess the editorial is celebrating that Congress just chose not to weaken testing in the new federal education law it passed in December.), it would be “impossible for the country to know what if anything children were learning from year to year,” and the Times once again endorses the Common Core standards as a way to ensure that students are “qualified to compete for higher-skilled jobs at companies like Boeing, Volvo and BMW.”  The editorial follows a December 26th critique by reporter Motoko Rich on the quality of the high school diploma, an article that examines ACT scores of students at Berea High School in Greenville, South Carolina. All of this echoes the bemoaning of the quality of American public education that followed the 1983 report, A Nation at Risk, and that set in motion the drive for more testing, more rigorous standards, and more accountability.

Diane Ravitch, the education historian who served for seven years on the National Assessment Governing Board and devoted the fifth chapter of her 2013 book, Reign of Error to explaining the role of the National Assessment of Education Progress for helping us to know what children are learning from year to year, responded promptly on her blog to the NY Times‘ recent editorial: “There is a federal testing program called the National Assessment of Educational Progress (NAEP) that reports on what U.S. students are learning every other year.  NAEP has been testing students since the 1970s and reporting on states and individual districts since 1992.  The scores on NAEP have steadily increased until the adoption of NCLB in 2002, when progress slowed.  Test score gains came to a crashing halt in 2015, as NCLB, Race to the Top, and Common Core converged….”  Ravitch suggests that the recent flattening of NAEP scores reflects the failure of the very policies endorsed by the NY Times in its editorial last week.

In recent years, the federal government has added graduation rates to test scores as part of the accountability yardstick.  There has been intense pressure on schools to raise test scores and on high schools to increase their rates of graduation.  Lyndsey Layton, writing last month for the Washington Post, announced that High School Graduation Rate Hits All-Time High; 82 Percent Finish on Time:  “The data show that every category of student—broken down by race, income, learning disabilities and whether they are English-language learners—has posted annual progress in graduation rates since 2010, when states adopted a uniform method of calculating those rates… But disparities persist.  In 2013-2014, 87.2 percent of white students graduated on time, compared with 72.5 percent of African Americans and 76.3 percent of Hispanics.  Asian Americans had the highest graduation rate, at 89.4 percent.  The rate was 74.6 percent for low-income students, 62.6 percent for English-language learners and 63.1 percent for students with disabilities.”

We want students to be well educated, and we definitely want to ensure that more students graduate from high school.  Layton quotes Arne Duncan, who explained the new graduation statistics this way: “School districts that made the greatest progress closely tracked student academic performance early, not waiting for a student to reach 11th or 12th grade before realizing there was a problem, Duncan said. ‘Many schools are focusing on the freshmen, on the first six weeks, with a laser like focus on making sure students are staying on track,’ he said.”  To the degree that educators are focusing on students in the transition from middle school to high school, Duncan is correct that this must be a key strategy.  Here is what researchers Lisa Abrams and Walt Haney reported in 2004: “The first major finding from our cohort progression analyses is that the rate at which students disappear between grades 9 and 10 has tripled over the last 30 years.”  “These analyses have allowed us to identify grade 9 as a key valve in the education pipeline, one that is closing for many students.” (Gary Orfield, editor, Dropouts in America, chapter 8, Lisa Abrams and Walt Haney, “Accountability and the Grade 9-10 Transition: The Impact on Attrition and Retention Rates.”)

The NY Times is correct to call our attention to indicators that something is not right in our schools, even if the newspaper’s editors incorrectly diagnose the problem as low standards and not enough testing and accountability. We definitely need to be paying attention to the atrophy of NAEP scores this year.  Perhaps NAEP scores reflect that we are spending less on education across America than prior to the Great Recession in 2008.  Or perhaps the 2015 NAEP scores reflect that school districts, under intense federal pressure to raise test scores and graduation rates at all costs—trying to lift the floor if not the ceiling—have been drilling the most vulnerable students on the very basic reading and math skills tested under No Child Left Behind with a narrowed curriculum.  Or maybe the problem is the so-called dropout-recovery programs in which students try to make up lost work through computerized instruction.

Consider some of the additional investments our society could make if we were to set out seriously to improve high school graduation rates, enrich the experience of schooling and close opportunity gaps.

We could institute enriched pre-Kindergarten to close the achievement gaps that already exist when young children begin their schooling.

We could reduce retention in grade.  Holding kids back is among the most counter productive of the education reforms that have swept the nation in the past decade. David Berliner and Gene Glass explain: “Researchers have estimated that students who have repeated a grade once are 20-30% more likely to drop out of school than students of equal ability who were promoted along with their age mates.  There is almost a 100% chance that students retained twice will drop out before completing high school.”  (Fifty Myths and Lies about How to Make Our Schools Better, p. 97)

We could make sure all students have access to advanced courses and science labs.

Finally we could personalize learning, and I do not mean the Orwellian definition of “personalized learning”: on-line, virtual schooling in which students are said to be free to explore their own interests and make up for their deficits, but in which computers substitute for teachers.  We could authentically personalize learning by investing in small classes where it would be much easier for teachers to be able to know and support every student; we could provide enough counselors and social workers to ensure personal attention for all students.  We could ensure that all high schools could afford enrichments like school newspapers and debate teams and a full range of competitive sports. We could prioritize school music programs with orchestras and jazz ensembles.

In a profound article, School Reform Fails the Test: How Can Our Schools Get Better When We’ve Made Our Teachers the Problem and Not the Solution?, Mike Rose the writer and UCLA professor of education wonders: “What if reform had begun with the assumption that at least some of the answers for improvement were in the public schools themselves, that significant unrealized capacity exists in the teaching force, that even poorly performing schools employ teachers who work to the point of exhaustion to benefit their students?  Imagine, then, what could happen if the astronomical amount of money and human resources that went into the past decade’s vast machinery of high states testing… had gone into a high-quality, widely distributed program of professional development.  I don’t mean the quick-hit, half-day events that teachers endure, but serious, extended engagement of the kind offered by the National Science Foundation and the National Writing Project…. Imagine as well that school reform acknowledged poverty as a formidable barrier to academic success.  All low-income schools would be staffed with a nurse and a social worker and have a direct link to local health and service agencies… Extra tutoring would be provided… Schools would be funded to stay open late, providing academic and recreational activities for their students.”

Worrying about the quality of every student’s experience at school is far more important than worrying about the quality of the diploma.  It is also stunningly clear in the disparities in high school graduation rates reported by Lyndsey Layton that our society must address the disparities in opportunity across racial and ethnic groups and between wealthy and poor students.

Another Former Supporter of Test-Based Accountability Confesses His Error

After more than a decade of federal test-and-punish education policy, true believers in the schemes spun by corporate education reformers are reevaluating how it has all worked out since the passage of the No Child Left Behind Act.  One at a time they are changing their minds.  Most notable for recanting her original support is the education historian Diane Ravitch, who has written two books and conducts a daily blog to demonstrate all the ways she was mistaken.

Now Harold Kwalwasser, the former general counsel for the Los Angeles Unified School District, the man who was responsible for handling the dismissal of weak teachers, confesses his error: “One major problem was that we lacked objective measures of teacher effectiveness.  So when the 2001 No Child Left Behind Act brought the nation annual standardized testing for math and reading, I applauded… But 14 years on, I think that’s a mistake.  I believe our exam system is deeply flawed, especially when it comes to to teacher evaluation.”

Kwalwasser makes his case simply and logically: “First, the results are too variable.  Teachers may one year be rated ‘highly effective’ while the next year they are merely ‘effective’ or worse, even though there are no observable changes in their teaching skills or strategies… Second, there is reason to doubt the relationship between test scores and an individual teacher’s competence… Third, we have the vagaries of student class assignment… None of the above even takes into consideration the segregation by race or class of school populations because of the continued (indeed increasing) segregation of housing patterns…. Fourth, the tests are too narrow in scope.  They largely focus on math and reading…. Finally, there is the little matter of the ‘cut score.'”  Because cut scores are usually set artificially high to motivate teachers and students alike to try harder, there is noting objective or scientific about a cut score. “So teacher evaluations are at times as much a statement about politics as teaching ability.”

Kwalwasser is not convinced that standardized tests are necessary at all for the evaluation of teachers.  “Before standardized tests, some districts had great evaluation and professional development programs that weeded out low performers.  Others did not.  Adding test data can’t turn weak programs into effective ones…”

In an endorsement of grade span testing as an alternative to the grind of annual standardized tests, Kwalwasser concludes: “Civil rights advocates worry that without standardized tests, the troubling disparities in our public education system will sink back into the mists…. I concur… Testing at the end of fourth and eighth grade can meet that need.”

Kwalwasser’s critique is sensible and principled, without the rhetoric that clouds today’s usual conversations about education policy.  His standard is the impact of public policy on the people in the schools—the students and their teachers: “Holding teachers and schools accountable is important, but the means should be accurate and fair.  The current standardized test program doesn’t pass muster.”  I urge you to read Kwalwasser’s piece carefully.

John Oliver Examines NCLB, Race to the Top, and Testing in Comedy Monologue

Surely it must be significant that John Oliver, the HBO comedian, did an 18 minute segment last week on what’s gone haywire in American public education.  Oliver traces the history of test-and-punish since the federal testing law No Child Left Behind Act was enacted in January of 2002.  I urge you to watch this segment of Last Week Tonight with John Oliver.

You will see a clip of President George W. Bush defining (Definition probably wasn’t his strong point.) school accountability and a video of candidate Barack Obama, in a speech to the National Education Association, disdaining an education philosophy centered around children coloring in bubbles on standardized tests.  Oliver puzzles about the inconsistency—a candidate Obama who said he hated testing and a President Obama whose administration has vastly expanded the amount of testing through programs like Race to the Top that led to the Common Core and stretched the uses of testing not only for rating and punishing schools but also for evaluating teachers with algorithms based on their students’ scores.

As if that isn’t enough, Oliver looks into the corporate testing industry—its reach and power in the lives of children and teachers, it’s secrecy, and the public’s inability to do anything about it even when test questions are poorly written.  We hear from the people who work as test graders—people who responded to ads on Craig’s List, people who are themselves held accountable for coming up with a bell curve in scores among the essays they read—not too many high scores.

The situation in our schools has been at the same time absurd and deeply troubling for almost fifteen years now, but none of this seems to have seeped inside the Beltway, where Congress is considering legislation that leaves annual testing in place, continues to blame teachers, and fails to address serious problems in the struggling schools of our nation’s impoverished communities.

Is nobody paying attention to what is happening with our children?  I have wondered if, as a culture, we have adopted an education philosophy of “out of sight, out of mind”—if we have accepted a focus on test scores as a proxy for caring about our society’s children.  I am delighted to see John Oliver raising this issue as though it is something people watching television ought to be thinking and talking about.

Our children and our more than 3 million school teachers across America ought to matter to us.  Watch this video.  Talk about it.  Get some other people to watch it and talk about it.

Advocates and National Organizations Are Questioning Test-and-Punish School Accountability

Suddenly for the first time in years, there is considerable talk about reforming federal policy in education.  Yesterday this blog reviewed the way federal education policy has become stuck and discussed an academic paper that seems to have stimulated new thinking by a number of education advocacy and civil rights organizations.  Today, the blog will share two new policy statements from prominent civil rights and education policy organizations and review growing protests against the standardized testing that has—due to growing federal and state accountability requirements—come to dominate our public schools.

As this blog described yesterday, in an academic paper published in August, Accountability for College and Career Readiness: Developing a New Paradigm, Linda Darling-Hammond and Gene Wilhoit propose that federal law stop merely blaming teachers and punishing the public schools in the poorest communities when, as we all surely know, there is massive inequity of investment by states and wide variance across school districts in their capacity to raise revenue locally.   A just society, Darling-Hammond and Wilhoit suggest, should be reciprocally accountable for investing significantly in the public schools that serve our society’s most vulnerable children– addressing gaps in opportunity as a primary way to address gaps in school achievement.

Darling-Hammond and Wilhoit’s paper seems to have united commitment across national advocacy organizations around the concept of reciprocal accountability.  First eleven of the nation’s most prominent civil rights organizations sent a joint letter to President Obama, Secretary Duncan and Congressional leaders, a letter that echoes the proposals in the paper published by Darling-Hammond and Wilhoit.  Last week this blog covered the new civil rights letter here. The civil rights organizations are Advancement Project, Lawyers’ Committee for Civil Rights Under Law, League of United Latin American Citizens, Mexican American Legal Defense and Educational Fund, National Association for the Advancement of Colored People, National Opportunity to Learn Campaign, National Urban League, NAACP Legal Defense and Educational Fund, National Council on Educating Black Children, National Indian Education Association, and Southeast Asia Resource Action Center.  Their statement disdains “overly punitive accountability systems that do not take into account the resources, geography, student population, and needs of specific schools.  In particular, the No Child Left Behind law has not accomplished its intended goals of substantially expanding educational equity or significantly improving educational outcomes.  Racial achievement and opportunity gaps remain large.”  These organizations advocate that accountability should measure resource inputs and support the academic, social, emotional, physical health, and cultural well-being of students.

Then seventeen national organizations—including some of the same civil rights groups along with a number of national educational organizations released New Accountability: A New Social Compact for American Education, a document that supports the idea of reciprocal accountability.  Sponsors are American Association of School Administrators, American Federation of Teachers, American Youth Policy Forum, Albert Shanker Institute, Alliance for Quality Education, Committee for Economic Development, Center for Teaching Quality, Education Law Center, Institute for Educational Leadership/Coalition for Community Schools, League of United Latin American Citizens, National Association for Bilingual Education, National Association of Secondary School Principals, National Education Association, National Opportunity to Learn Campaign, National School Boards Association, Partnership for 21st Century Skills, and Southeast Asia Resource Action Center.  The new “social compact” declares: “Accountability in American education must focus both on gathering complete information on the performance of students, educators, schools and districts, and on providing the feedback, resources and supports necessary for their improvement.  A fundamental paradigm shift in our accountability regime will be required, as the failed approach of ‘test and punish‘ is replaced with a strategy of ‘support and improve.'” “Genuine accountability rests on shared responsibility for educational outcomes.  All of the institutions participating in American education—from the federal government, state governments and higher education to school boards, school districts and schools—must be accountable for the contributions each must make to ensure high-quality learning opportunities for every child. Government must be accountable for equitably allocating adequate resources—dollars, curriculum and learning tools, well-qualified educators, and safe healthy environments for learning—to meet student needs and support meaningful learning.”  You are invited to join the authors of the Social Compact for American Education by signing on.

Finally, there is growing conversation about the tests themselves.  Darling-Hammond and Wilhoit’s academic paper also addresses this issue at length and declares: “If meaningful learning for all students is the focus of an accountability system, the system should use a range of measures that encourage and reflect such learning, and it should use those measures in ways that improve, rather than limit, educational opportunities for students.  This means we need both much better assessments of learning—representing much more authentically the skills and abilities we want students to develop—and multiple measures of how students, educators, schools, districts, and states are performing.”

The problem is not merely the quality of the tests, however.  An enormous concern is the amount and frequency of testing.  Sixteen superintendents of large, county-wide school districts recommend that the U.S. Department of Education, even in the waivers it is offering from NCLB’s failed policies, reduce the time and energy being devoted in America’s classrooms to testing by substituting grade-span testing instead of annual testing.  They are suggesting that federally required standardized tests be reduced from seven (grades 3-8 and once in high school) to three times (once in elementary school, once in middle school, and once in high school).

Last week Monty Neill, executive director of the National Center for Fair & Open Testing was featured as a guest columnist by Valerie Strauss in the Washington Post: “Across the nation, resistance to test overuse and misuse reached unprecedented heights in the spring of 2014.  The rapidly growing movement built on significant test opposition unleashed in 2013.  This year, resistance erupted in more states with far more participants, and it won notable victories such as ending, lessening or postponing graduation exams in at least eight states and easing or ending grade promotion tests.”  He describes a growing opt-out movement among parents and adds, “School boards are also resisting test overkill.  In New York, about 20 districts refused to administer tests used for the sole purpose of trying out items for next year’s state exams.”

Neill remains sober about the amount of work still needed to grow such actions, however.  “The ultimate goals of the movement are to dramatically reduce the amount of testing, end high stakes uses, and implement educationally sound assessments.  Progress has been made, but much more must be done.  To succeed, the movement must keep rapidly expanding while uniting across lines of race, class and where possible, politically ideology.  And it must turn its growing strength into greater victories.”