Standardized Tests: Truth and Consequences

What responsibility do standardized test advocates have to the tested?

In recent posts I have been addressing issues related to student testing on the PARCC, SBAC and DIBELS. My research for these posts has led me down many dark alleys and more than a few rabbit holes, but one piece of information I came across created a real "Eureka!" moment in my standardized test addled brain. The discovery, which was roaming fuzzily around in my mind for a long time, finally came into focus when I discovered the concept of consequential validity.

Most of you who are certified teachers probably remember something about test validity from that ed. psch. course you took as an undergraduate. Simply stated a standardized test is said to be valid (at least statistically) if it measures what it claims to measure. So, a test of reading comprehension would be required to demonstrate that it indeed measured reading comprehension and not something else, like say, the relative wealth of the people taking the test. But in 1989 a psychologist named Samuel Messick posited that tests had a higher calling to answer to than just a statistical validity. A test also needed to be valid in the way it was used and interpreted. Messick called this new take on testing consequential validity.

Consequential validity requires test makers and test givers and test interpreters to ask, "What are the risks if the tests are invalid or incorrectly interpreted?" and "Is the test worthwhile given the risks?"

The recent history of the test and punish movement in America would suggest that we are coming up very short in the consequential validity department. A recent report from Fair Test chronicles the failure of the No Child Left Behind (NCLB) law of 2002 to meet any of its stated goals. NCLB, of course, brought on yearly standardized testing in grades 3-8 with the promise of narrowing the achievement gap in America's schools. Fair Test found that NCLB has been notably unsuccessful in narrowing gaps and that in many cases (such as English Language Learners, students with disabilities) the gaps are wider than they were in 1998.

Clearly, the NCLB testing regime has failed to narrow achievement gaps, but that is not the worst news. Just what have been the consequences for children of this move to more standardized testing? According to Fair Test the consequences include widespread evidence of curriculum narrowing, extensive teaching to the test, pushing low-scorers out of school and widespread cheating scandals.

I could add a few more consequences to this list. Since these tests were used to label schools erroneously as "failing", the tests have undermined the morale of teaching staffs and demonized schools in urban areas struggling with myriad issues ranging from student poverty to lack of textbooks to crumbling infrastructure. Many elementary schools have done away with recess to cram in more test prep. New elementary schools were built without a playground, because the test trumped student active play. Less time was allotted for arts instruction, so that students could focus on tested subjects. Could future researchers point back to NCLB and find other consequences like increased obesity and a declining participation in the arts?

In 2009, the Obama administration doubled down on NCLB with their program entitled Race to the Top (RTTT). RTTT called for new tests tied to the Common Core State Standards and for using the scores from those tests, not only to rate schools and children, but also to evaluate teachers. What are the likely consequences of rating teachers based on these tests? According to the Education Policy Institute we can expect the following:

Tying teacher evaluation and sanctions to test score results can discourage teachers from wanting to work in schools with the neediest students, while the large, unpredictable variation in the results and their perceived unfairness can undermine teacher morale. Surveys have found that teacher attrition and demoralization have been associated with test-based accountability efforts, particularly in high-need schools.

So, one consequence of the new testing regime is likely to be to make it even harder for urban schools to recruit the best, brightest, most dedicated teachers. In New York where the new tests have already been instituted, State Department of Education officials predicted that as a consequence of the new test only 30% of children would be found proficient. Low and behold this prophesy came true, perhaps because those same officials were responsible for determining the "cut scores" after the test results were in. 

What were the consequences? Further humiliation of children, teachers and schools and a general outcry from concerned parents. The parental concern led federal Secretary of Education to declare that these "white suburban moms" were surprised to find their kids were not as smart as they thought. No officials seemed to consider that the tests were not as smart as they might be.

As the new tests spread across the country we can predict that students’ scores will fall. Testing advocates will cheer and say the new tougher standards have been validated and they will use the scores to push for more school choice, more charter schools, more teacher union bashing and more tests.

These are the consequences we can look forward to as the push for more standardized testing continues. These tests have already proven that they have no validity as a tool for narrowing achievement gaps or for improving the lives of the vast majority of the 25% of American children living in poverty. 

When we look at the consequences of standardized testing, low student and teacher moral, narrowed curriculum, cheating scandals, test prep parading as learning, it is also clear that this level of high stakes standardized testing in schools fails the test of consequential validity.

To return to Messick again I would ask, "Are these tests worth the risks?" The clear answer is absolutely not.

As Lily Tomlin's wise little girl character, Edith Ann would say, "And that's the truth!"