Big Data on Learning Loss Is Not the Point: Teachers Know How to Use Formative Assessments to Guide Their Work with Each Child

Recently on the news, I heard an education researcher discussing the importance of using standardized tests to measure something called “learning loss” across racial, ethnic, and regional groups of children during the pandemic.

Like so many others, this so-called expert made the for case for reinstating—this spring during the pandemic—the standardized testing prescribed by the federal government in No Child Left Behind (NCLB) and the Every Student Succeeds Act (ESSA). Ignoring the impossibility of collecting valid and reliable data and the impracticality of even of trying to administer the tests when some students are learning online and others attending in-person classrooms, this person pretended Miguel Cardona, the new education secretary, can merely require the tests and they will happen.  She implied that the mandated tests in two subjects, basic language arts and math, will inform teaching once students come back to school even though we know that teachers do not receive scores for months and the data they receive will not contain information about the particular questions students get right or wrong. And she didn’t mention that even if a teacher did know exactly how a student marked any particular multiple choice answer, it would tell the teacher nothing about that student’s thinking.

Then this specialist who wanted to measure “learning loss” reinforced the notion that the collection of national data would enable the federal government and the states to invest in the schools where children are farthest behind. Advocates for standardized testing this spring often justify the need for testing during the pandemic as a way to drive investment in schools, as though investing in schools where children are farther behind has ever been the result of our regime of standardized testing. Anybody who has been paying attention over the past two decades since NCLB mandated annual standardized testing knows that aggregate test scores have not once—federally or at the state level—driven added funding to the schools where students’ scores are low.

The whole regime has been correctly called “test-and-punish” because NCLB prescribed sanctions for so-called failing schools and the states have adopted the same responses: add charter school choice in so called “failing” school districts; make students in so-called “failing” schools eligible for vouchers for private school tuition; close so-called “failing” schools and relocate the students; take over the so-called “failing” school districts in Detroit or Newark or Philadelphia or Chester Upland or Lorain or Youngstown or East Cleveland and put a state-appointed overseer in charge.

Title I is an important national program providing federal compensatory funding for school districts serving concentrations of poor children, but if he can bring Congress along, President Biden has already promised to triple funding for this federal program which has long been underfunded despite two decades of standardized testing which have clearly identified Title I schools as places desperately in need of greater investment.  During the presidential campaign last year, Biden identified the need for more money in the same school districts where families have been devastated by COVID-19: “There’s an estimated $23 billion annual funding gap between white and non-white school districts today, and gaps persist between high- and low-income districts as well.”

Seventy-four education advocacy groups and over 10,000 individuals wrote a letter to the incoming education secretary to demand that he grant states waivers to cancel the NCLB/ESSA mandated testing.  One sentence in that letter stood out for me: “To believe that it is impossible for teachers to identify and address learning gaps without a standardized test is to have a breathtaking lack of faith in our nation’s teachers.”

I worry that many people do lack faith in schoolteachers, because I believe that most people lack any understanding of what teachers do.

In a profound article, the executive director of the New Jersey Principals and Supervisors Association, Patricia Wright explains the work of teachers in words that will perhaps help the public grasp why Miguel Cardona should readily grant states waivers to skip standardized testing this spring.

Wright begins: “There has been a lot of talk lately about ‘learning loss.’ How will students catch up? What will educators do when schools are finally able to return to full in-person instruction?… Students do not need to feel like they are now susceptible to failure or that their future is in jeopardy because they may not have fully grasped certain skills and knowledge. Educators know that students need to see themselves, not just making up what they may have lost, but moving forward and accelerating their learning. Educators know how to do this work. They do it every day. Schools across the state are already collecting student data, examining and revising their curriculum and making plans to continually use assessment information throughout the next school year to inform their instruction. This will allow them to provide the necessary interventions and supports to ensure students can continue to accelerate their learning. This is the professional practice of education, something we do very well in New Jersey.”

Wrignt continues: “Schools will need to depend on formative assessment, which is assessment for learning.  It is currently used by educators to identify where students are in relation to the academic standards that are required in their current grade level… Formatively assessing students throughout the year will allow educators to bridge the learning gaps as students continue to move forward, focused simultaneously on remediation and acceleration.”

For a better understanding of formative testing, we can turn to The National Center for Fair and Open Testing’s Monty Neill, who defines the kind of formative testing teachers use all the time: “(T)eachers must gather information. Teachers must keep track of student learning, check up on what students have learned, and find out what’s going on with them. Keeping track means observing and documenting what students do. Checking up involves various kinds of testing and quizzing. Finding out is the heart of classroom assessment: What does the child mean? What did the child get from the experience? Why did the child do what he or she did? To find out, teachers must ask questions for which they do not already know the answers. To gather all this information, teachers can rely on a range of assessment activities. These include structured and spur-of-the-moment observations that are recorded and filed; formal and informal interviews; collections of work samples; use of extended projects, performances, and exhibitions; performance exams; and various forms of short-answer testing.”

Monty Neill of FairTest Comments on ESSA Rules: You Can, Too

The rule-making process for the implementation of the new, federal education law, the Every Student Succeeds Act (ESSA), is a little deep into the policy weeds for most readers of this blog.  But until August 1, 2016, the U.S. Department of Education is accepting public comments on the rules it has proposed, and some organizations with the expertise carefully to analyze the law itself and the Department’s proposed rules have been drafting and submitting comments.  FairTest, the National Center for Fair and Open Testing, has submitted a formal comment, and yesterday Valerie Strauss published in the Washington Post an explanation from Monty Neill, the organization’s executive director, about concerns at FairTest about ESSA.

Monty Neill is very familiar with the federal education law’s accountability philosophy and provisions.  In his piece yesterday in the Washington Post, he shares concerns about the rules now drafted by Secretary John King’s Department of Education, rules that will govern enforcement of school accountability. Overall Neill believes that the Obama administration’s Department of Education has proposed rules that yield too much power back to the federal government and undermine the ways in which Congress tried to return power to states for school accountability in the new law.  And test-and-punish accountability is far too prominent in the proposed rules.

Neill writes, “While the accountability provisions in the Every Student Succeeds Act (ESSA) are superior to those of No Child Left Behind, the Department of Education’s draft regulations intensify ESSA’s worst aspects and will have the effect of perpetuating some of NCLB’s most damaging practices. The draft regulations over-emphasize testing, mandate punishments not required in law, and continue federal micromanagement. When the department makes decisions that should be set at the state and local level in partnership with local educators, parents, and students, it takes away local voices that ESSA restores.”  In his Washington Post column, Neill names five provisions in the draft rules that he believes need to be eliminated or amended.  I agree with Neill’s concerns, but I am reordering a bit as a way to indicate my own concerns about their importance.

  1. The proposed rules require (according to Neill’s third concern) that states come up with performance indicators for their schools and… combine the multiple indicators each state selects into a “single ‘summative’ score for each school.”  Neill writes: “As Rep. John Kline, chair of the House Education Committee pointed out, no such requirement is included in ESSA.  Summative scores are simplistically reductive and opaque. They encourage the sort of flawed school grading schemes promoted by diehard NCLB defenders.”  I will add my own concern here.  Congress, under much pressure from teachers and other supporters of public school improvement, left room for states to include some indicators of unequal opportunity-to-learn—disparities in resource inputs, not just test score outcomes. The requirement in the proposed rules demands that while states can choose indicators by which they judge school quality, they must, before they submit the rating, combine all the indicators into school-by-school “grades.”  Those overall “grades” will very likely mirror the current school grades (based merely on test scores) being awarded by too many states today.
  2. The law requires that states continue to identify the lowest scoring 5 percent of schools and the schools whose subgroups of students continue to test poorly.  The proposed rules (and here is Neill’s second concern) require states also to identify three or more levels of performance to differentiate the performance of schools across the state.  FairTest proposes that the regulations should scrap any reference to performance levels and allow states themselves to decide how to identify low-performing schools.
  3. For Neill, the top concern is the the proposed rules’ limitation of parents’ right to protest by opting their children out of testing. (I have moved it to third place.)  Here is how Neill describes his concern: “Most egregiously, the department would require states to lower the ranking of any school that does not test 95 percent of its students or to identify it as needing ‘targeted support.’  No such mandate exists in ESSA.”  Neill recommends that, “The department should simply restate ESSA’s language allowing the right to opt out as well as its requirements that states test 95 percent of students in identified grades and factor low participation rates into their accountability systems.”
  4. In the new rules, the Department of Education requires that a state’s academic indicators for accountability carry “much greater” weight than its “school quality” indicators (Neill’s fourth concern).  Neill recommends that the department leave the language of the law in place—that academic indicators count for more than (not much more than) 50 percent of a state’s school quality indicators.  Personally, I would like to see school quality indicators given perhaps additional weight, though that would depend on what school quality indicators are being used.  Ideally the federal government would be pressuring the states to invest in the poorest schools for equity and thereby improve the quality indicators in the poorest schools, but Congress itself, as it wrote the new ESSA, neglected to use the law as an opportunity to create incentives to promote states’ investments for equity.
  5. Finally  the implementation time line for accountability seems short (Neill’s fifth concern): “The department would require states to use 2016-17 data to select schools for ‘support and improvement’ in 2017-2018.  This leaves states barely a year for implementation, too little time to overhaul accountability systems.” Remember that the new law assigns to the states the responsibility for creating improvement plans for schools that struggle.

Although this all may seem overly technical, reading Neill’s concerns helps clarify at least some of the ways that ESSA remains, on the whole, a test-and-punish accountability law. While ideally states would find ways to invest in, improve, and support the schools in the poorest communities, the places where test scores—the primary measurement states will continue to use—are lowest, the danger is that states will continue to punish teachers and close and charterize so-called “failing” schools across America’s big city neighborhoods where poverty is concentrated.

You can submit your own comments to the U.S. Department of Education until August 1. Neill provides the instructions here: “The regulations are at https://www.regulations.gov/#!documentDetail;D=ED-2016-OESE-0032-0001.  To submit comments on the regulations, go to https://www.regulations.gov/#!docketDetail;D=ED-2016-OESE-0032.  FairTest even has a sample action alert response that you may copy and paste in to the Department’s comment form if you want to simply support FairTest’s comments.

Advocates and National Organizations Are Questioning Test-and-Punish School Accountability

Suddenly for the first time in years, there is considerable talk about reforming federal policy in education.  Yesterday this blog reviewed the way federal education policy has become stuck and discussed an academic paper that seems to have stimulated new thinking by a number of education advocacy and civil rights organizations.  Today, the blog will share two new policy statements from prominent civil rights and education policy organizations and review growing protests against the standardized testing that has—due to growing federal and state accountability requirements—come to dominate our public schools.

As this blog described yesterday, in an academic paper published in August, Accountability for College and Career Readiness: Developing a New Paradigm, Linda Darling-Hammond and Gene Wilhoit propose that federal law stop merely blaming teachers and punishing the public schools in the poorest communities when, as we all surely know, there is massive inequity of investment by states and wide variance across school districts in their capacity to raise revenue locally.   A just society, Darling-Hammond and Wilhoit suggest, should be reciprocally accountable for investing significantly in the public schools that serve our society’s most vulnerable children– addressing gaps in opportunity as a primary way to address gaps in school achievement.

Darling-Hammond and Wilhoit’s paper seems to have united commitment across national advocacy organizations around the concept of reciprocal accountability.  First eleven of the nation’s most prominent civil rights organizations sent a joint letter to President Obama, Secretary Duncan and Congressional leaders, a letter that echoes the proposals in the paper published by Darling-Hammond and Wilhoit.  Last week this blog covered the new civil rights letter here. The civil rights organizations are Advancement Project, Lawyers’ Committee for Civil Rights Under Law, League of United Latin American Citizens, Mexican American Legal Defense and Educational Fund, National Association for the Advancement of Colored People, National Opportunity to Learn Campaign, National Urban League, NAACP Legal Defense and Educational Fund, National Council on Educating Black Children, National Indian Education Association, and Southeast Asia Resource Action Center.  Their statement disdains “overly punitive accountability systems that do not take into account the resources, geography, student population, and needs of specific schools.  In particular, the No Child Left Behind law has not accomplished its intended goals of substantially expanding educational equity or significantly improving educational outcomes.  Racial achievement and opportunity gaps remain large.”  These organizations advocate that accountability should measure resource inputs and support the academic, social, emotional, physical health, and cultural well-being of students.

Then seventeen national organizations—including some of the same civil rights groups along with a number of national educational organizations released New Accountability: A New Social Compact for American Education, a document that supports the idea of reciprocal accountability.  Sponsors are American Association of School Administrators, American Federation of Teachers, American Youth Policy Forum, Albert Shanker Institute, Alliance for Quality Education, Committee for Economic Development, Center for Teaching Quality, Education Law Center, Institute for Educational Leadership/Coalition for Community Schools, League of United Latin American Citizens, National Association for Bilingual Education, National Association of Secondary School Principals, National Education Association, National Opportunity to Learn Campaign, National School Boards Association, Partnership for 21st Century Skills, and Southeast Asia Resource Action Center.  The new “social compact” declares: “Accountability in American education must focus both on gathering complete information on the performance of students, educators, schools and districts, and on providing the feedback, resources and supports necessary for their improvement.  A fundamental paradigm shift in our accountability regime will be required, as the failed approach of ‘test and punish‘ is replaced with a strategy of ‘support and improve.'” “Genuine accountability rests on shared responsibility for educational outcomes.  All of the institutions participating in American education—from the federal government, state governments and higher education to school boards, school districts and schools—must be accountable for the contributions each must make to ensure high-quality learning opportunities for every child. Government must be accountable for equitably allocating adequate resources—dollars, curriculum and learning tools, well-qualified educators, and safe healthy environments for learning—to meet student needs and support meaningful learning.”  You are invited to join the authors of the Social Compact for American Education by signing on.

Finally, there is growing conversation about the tests themselves.  Darling-Hammond and Wilhoit’s academic paper also addresses this issue at length and declares: “If meaningful learning for all students is the focus of an accountability system, the system should use a range of measures that encourage and reflect such learning, and it should use those measures in ways that improve, rather than limit, educational opportunities for students.  This means we need both much better assessments of learning—representing much more authentically the skills and abilities we want students to develop—and multiple measures of how students, educators, schools, districts, and states are performing.”

The problem is not merely the quality of the tests, however.  An enormous concern is the amount and frequency of testing.  Sixteen superintendents of large, county-wide school districts recommend that the U.S. Department of Education, even in the waivers it is offering from NCLB’s failed policies, reduce the time and energy being devoted in America’s classrooms to testing by substituting grade-span testing instead of annual testing.  They are suggesting that federally required standardized tests be reduced from seven (grades 3-8 and once in high school) to three times (once in elementary school, once in middle school, and once in high school).

Last week Monty Neill, executive director of the National Center for Fair & Open Testing was featured as a guest columnist by Valerie Strauss in the Washington Post: “Across the nation, resistance to test overuse and misuse reached unprecedented heights in the spring of 2014.  The rapidly growing movement built on significant test opposition unleashed in 2013.  This year, resistance erupted in more states with far more participants, and it won notable victories such as ending, lessening or postponing graduation exams in at least eight states and easing or ending grade promotion tests.”  He describes a growing opt-out movement among parents and adds, “School boards are also resisting test overkill.  In New York, about 20 districts refused to administer tests used for the sole purpose of trying out items for next year’s state exams.”

Neill remains sober about the amount of work still needed to grow such actions, however.  “The ultimate goals of the movement are to dramatically reduce the amount of testing, end high stakes uses, and implement educationally sound assessments.  Progress has been made, but much more must be done.  To succeed, the movement must keep rapidly expanding while uniting across lines of race, class and where possible, politically ideology.  And it must turn its growing strength into greater victories.”