The Move Away From Standardized Tests

Beginning in Fall 2020, Marquette University will become the latest Wisconsin university to drop the requirement that prospective students take either the ACT or SAT exam to be considered for entrance into their university. Wisconsin Public Radio reported that other Wisconsin colleges/universities have dropped the test requirement, including Beloit, Carthage, Colleges of Menominee Nation, Lawrence, Marian, Milwaukee Institute of Art Design, Northland and Ripon. Nationally more than a thousand higher education institutions have done the same, and the list of schools dropping the requirement continues to grow.

These colleges/universities claim that there is little correlation between test scores and how well students do once they enter their hallowed halls. Rather one should look at the rigor of the high school classes students took, their performance in those classes and extracurricular activities; in other words, what they did in high school.

They also point out there is an inherent bias against students of other ethnic backgrounds, who grew up in poverty, who did not have the benefit of test prep programs.

A chorus of educational advocates are calling for the elimination of such entrance tests at all colleges/universities. SAT has responded by creating an “Adversity Score” to take into account 15 factors including the quality of the high school, the crime rate and poverty level in the neighborhood that the student lives with.

While that discussion is taking place at the higher education level, the same questions are being raised as to what degree we should rely on standardized test scores in the primary and secondary levels to evaluate our students, teachers, schools and school districts.

When Ted Kennedy and George Bush hammered out their agreement to improve primary and secondary education in the No Child Left Behind Act of 2001 (NCLB), they had to determine some method to measure student success. Policy makers turned to standardized test scores using a system called the Value-Added Model (VAM).

With VAM, students would be given a pre-test at the beginning of the year and a post-test at the end. The larger the gain, the more successful a teacher or even a whole school would be measured.

VAM has been under attack for some time. Researchers note that test scores can be easily manipulated and often do not take into account numerous variables. When it came time to reauthorize the federal role in primary and secondary education, the 2015 law, Every Student Succeeds Act (ESSA), narrowed the requirements in using standardized tests. As researchers like Audrey Amrein-Beardsley have written: “ESSA has since curbed the extent to which states are adopting and implementing VAMs.”

Wisconsin, along with most other states, still uses VAM or some other testing measurement system as a key component in their valuation of schools and teaching. This is, in part, because they don’t know what else to use, and the public still wants to see test scores.

So upset was Professor Howard Wainer with VAM that he devoted an entire chapter on VAM in his 2011 book, Uneducated Guesses. Wainer was the principal researcher at the Educational Testing Service and now serves as an adjunct professor of statistics at the University of Pennsylvania. What Wainer and others point out is that schools can discourage “good” students from showing up for the pre-test and “bad” students from the post-test. Wisconsin tries to minimize the problem by only counting students who take both the pre- and post-tests and looking at the percentage of a school’s population who take the test. That minimizes the problem but does not do away with it completely.

Wainer shows that a teacher who had excellent student test scores one year can have terrible scores the next simply because of the makeup of a class, a room change next to the cafeteria and so on. A single day test can be impacted by a major fight in the school the day before, the death of a popular teacher, or winning a sport’s championship.

“… the more you know about VAM the less faith you have in the validity of inferences drawn from it,” writes Wainer.

That opinion is supported by the position statement of the American Statistical Association.

Yet reporters and opinion writers keep on making headlines reporting VAM scores and comparing schools when many educational researchers no longer use them as a major benchmark.

While some researchers say we should stop using standardized tests, others recommend we minimize their importance. We should not use VAM scores alone to compare teachers or schools as to their relative success or failures.

Researchers are looking to other measurements of educational achievement and are turning to a general category of “character.”

“A growing research base shows that non-cognitive (or socio-emotional) skills like adaptability, motivation, and self-restraint are key to determinants of adult outcomes,” writes C. Kirabo Jackson, an educational researcher at Northwestern University. “… the impact of teachers on behavior is about 10 times more predictive of whether they increase students’ high school completion than their impact on test scores.”

One might think that there would be a correlation between good character and good test scores. Surprisingly, that correlation is fairly weak, says Jackson and other researchers. Teachers who fixate on improving test scores through drill and practice often destroy creativity, curiosity and a joy of learning in their students.

Angela Duckworth of the University of Pennsylvania started to examine some of these same characteristics which she called “grit.” Her research showed that those with the greater determination or grit were more likely to graduate from West Point than those who had higher incoming test scores but had less grit. She has developed methods to measure grit and advocates that grit can be taught. But not everyone agrees with Duckworth’s findings, notes David Denby in The New Yorker, believing she oversimplifies these issues and overlooks other major factors.

Jackson advocates measuring behavior value-added, and he has developed a system measuring such behaviors through his research. Jackson created a “behavior index” which measures factors such as the number of absences, suspensions, grade point averages, and on-time progression to 10th grade. Concludes Jackson, “The behavior index is also a better predictor than 9th-grade test scores of high-school GPA and the likelihood that a student takes the SAT and plans to attend college.”

Researchers are still working on methods to measure behavior, grit or character, and getting reliable measurement models will take time. “The standardized-testing agencies that administer the National Assessment of Educational Progress (NAEP) and the Program for International Student Assessment (PISA) are moving toward the inclusion of character assessment as a measure of student performance,” writes Denby. Wisconsin’s school report card has been modified in the last few years to downplay the role of standardized test scores and highlight some of these behavioral measurements such as school attendance and suspension rates.

Primary and secondary schools will likely look more to character development than just raising test scores.

If colleges/universities are dropping the ACT/SAT requirement, just how are they evaluating who gets into their programs? Are they using some standardized measurement of academic rigor, character or grit? The answer appears to be that they are just going with their gut, reading essays, looking at class rigor, but not using any quantifiable measurement system. Whether these systems produce better outcomes is still unknown.

A footnote here is needed. Twelve years ago, when I was first elected to the Milwaukee school board, I pushed to make taking the ACT a graduation requirement. That did happen, and within a few years, Wisconsin changed how schools and districts would be evaluated by the use of the ACT and other tests from the same organization.

I still stand by that decision for these reasons:

• With our previous state required tests, high school students showed little interest in doing well on such standardized tests because they believed they were of little benefit to them. But once that ACT was used to evaluate their schools, and students also understood that these tests could be used to gain entrance into colleges, reports back from high school teachers indicated that students took this test much more seriously.
• Even though there is a growing list of colleges/universities dropping the test requirement, the majority still require one of these tests. Not taking the test eliminates a vast number of schools that students can consider.
• We know anecdotally there are Milwaukee students who never considered going to college, who did well on the ACT and then received letters of interest from prospective colleges/universities. Ultimately, they attended college. We have a fair number of college graduates from Milwaukee today because we required the ACT.

For the foreseeable future, we must continue using the ACT and other academic tests, but it cannot be the major way we evaluate students, teachers and schools.