How Did MPS Do During the Pandemic?

There was something for everyone when the Milwaukee school administration presented the results of state and national test scores to the school board on Thursday. Supporters of Milwaukee Public School (MPS) could point to how well the school system had bounced back on the state tests coming out of the pandemic. Critics of the districts could counter that MPS still has some of the lowest scores of urban cities on this national test. Dr. Melanie Stewart, director of research and assessment for MPS, did her best to make sense of the numbers for the board and public.

The state assessment numbers compare data from Spring 2021 to Spring 2022. “MPS improved in all subjects and all tests in that time frame,” said Stewart. “We are very excited… the positive move we are making coming out of the pandemic.” 27 schools actually had higher scores than they had pre-pandemic. One of the concerns nationally is that the learning loss that took place during the pandemic might take years to make up if at all. The results from MPS suggests that isn’t a problem.

But when Stewart turned to the national test, the results were somewhat darker and more complex.

The National Assessment of Educational Progress (NAEP) is conducted by the U.S. Department of Education. NAEP tests a representative group of students from each state and 26 urban cities in various subject areas. A profile of each student is compiled including socioeconomic status, race, native language, technology usage and other factors. Unlike individual state assessments, the NAEP data can be used to gauge a national educational assessment and compare individual states and communities.

These test results were in reading and mathematics in grade 4 and 8. The results showed that Milwaukee has some of the lowest test scores among the 26 urban districts in the nation, ranking around 3rd or 4th from the bottom. The bottom half dozen are Detroit, Cleveland, Baltimore city, Milwaukee, Shelby County (Memphis) and Philadelphia.

During the pandemic, every one of the 26 districts lost ground except Los Angles. But Stewart pointed out that Milwaukee lost less ground than many other districts despite its low ranking. Its learning loss in the 8th grade math was much lower than 18 other city school districts.

Wisconsin is reported to have the largest achievement gap between blacks and whites in the nation, but Stewart was able to show that was not true for Milwaukee on this test. Four other districts had greater black/white gaps on NAEP than Milwaukee, though the gap here was still quite large.

District 7 school board director, Henry Leonard, found the differences in the two set of data perplexing. “If you look at the previous presentation [on state scores] and growth, it is exciting to see, and then we look at this [NAEP scores], and I can’t deny it, I want to see those scores far exceed what we are looking at.”

Part of the problem is that the two sets of data were taken at different times and comparing different sets of data. The state scores were comparing data from Spring 2021 with Spring 2022. During 2021, school systems were in the mist of the pandemic and during Spring 2022 schools had completed almost all of entire school year with fairly normal instruction. But NAEP was comparing pre-pandemic data with that for winter 2022 when schools had been back with regular instruction for only a few months.

In-person vs. virtual instruction

Perhaps no Wisconsin school district stuck with virtual education during the pandemic longer than Milwaukee Public Schools.

The Wall Street Journal published an October 27 article entitled, “School COVID lockdowns worsened learning losses.” that highlighted several examples of school district learning loss, and  concluded that in-person education was superior to virtual education.

However, Peggy Carr, commissioner of the National Center for Education Statistics, was quoted in CNN saying, “There’s nothing in this data that tells us that there is a measurable difference in the performance between states and districts based solely on how long schools were closed.”

Sara Shaw, senior education policy researcher for Wisconsin Policy Forum (WPF), notes that Carr is essentially saying the learning loss between virtual and in-person education during the pandemic was about the same. Although WPF has not conducted its own analysis, Shaw says that what she has heard from other researchers is similar. “They do see [a difference], but it is not as large as they expected it to be.”

The impact of poverty

Yet we still see that Milwaukee ranks near the bottom of the NAEP scores. It is also true that the half dozen of the school districts at the bottom ranking have some of the highest percentage of students in poverty of the 26 districts listed.

We cannot automatically point to a direct cause/effect relationship of poverty and poor performance. School districts with high percentage of impoverished students are also often districts that have less funds to spend on their children, higher class sizes, and less experienced teachers just to name some of the disadvantages, writes Michelle A. Bassetti for PATIMES, a publication of the American Society for Public Administration.

“What we have known for a very long time is that there is a very strong negative correlation between test scores and poverty,” says Bradley Carl, co-director of the Wisconsin Evaluation Collaborative at the Wisconsin Center for Education Research, UW-Madison.

Next steps

Stewart outlined what work needed to be done by MPS:

• The district has to hold true to its direction on “Ambitious Instruction.”
• The district needs to concentrate on lifting up educational leaders and teachers. The district has been offering inservice training Saturdays in order not to interfere with student instruction. Stewart reported that the Saturday sessions have been well attended.
• More must be done to increase student attendance. Students can’t learn if they are not in school.
• More resources must go out to the community beyond the building during school hours. That means learning resources, technology, tutoring and partnerships with other organizations. Stewart pointed to another pot of federal dollars (besides pandemic funding) specifically offered for efforts to improve academic progress, which could be a funding source to provide some of these resources.