Can MPS Learn From the Data?
Its rejection of the Opportunity Schools Partnership is merely one sign the answer is no.
When Demond Means resigned as commissioner of the Opportunity Schools Partnership Program (OSPP), groups like the Working Families Party and the Milwaukee Teachers’ Education Association celebrated. Is it really cause for celebration?
Milwaukee Public Schools surely needs help. Compared to the average urban school districts, MPS performance has been disappointing, particularly for low income students (generally defined as those who qualify for free lunch) and African Americans and Hispanics. And it has been slow in improving. This first graph compares the performance of 4th grade students who qualify for free lunch on the NAEP (National Assessment of Educational Progress) reading test in Milwaukee, Boston, Chicago, and Atlanta. Milwaukee starts lower and has not seen major improvement.
Here are the NAEP 8th grade mathematics scores for African American students from the same four districts. A similar pattern appears on all the NAEP tests. Generally MPS is near the bottom with a handful of other districts, such as Detroit and Fresno. Earlier this year, NAEP released its scores for 2015, but Milwaukee was not among them.
The 2015 test results, the most recent, are not included in these charts because then-Superintendent Gregory Thornton pulled MPS out of NAEP. So far as I can tell, Thornton didn’t share this decision with the school board, the press, or the public. The first Journal Sentinel article mentioning this decision appeared this spring when reporting the 2015 NAEP test scores. The MPS spokesman attributed this decision to Thornton’s belief that the shift to “state tests aligned to the Common Core State Standards would provide better and more timely measures.”
Since then, under new leadership, MPS has applied and been accepted for the next cycle of NAEP testing, scheduled for 2017. This is good news for those who believe that the first step in improving is good measurement. The NAEP test is considered the gold standard of tests. It allows the comparison of a school district with similar districts and the tracking of results over time. Since it is based on a sample of students and results for individual schools are not released, there is little incentive for schools to cheat.
Despite Thornton’s faith in the state tests, the combined efforts of the governor, legislature, and Department of Public Instruction have succeeded in making a thorough botch of the Wisconsin testing program. For the 2014-15 school year, the existing test was replaced with a new “Badger Exam,” supposedly aligned with the Common Core and able to be taken online. As the result of technical glitches and ideological opposition to the Common Core, the legislature proceeded to kill the Badger Exam, replacing it with a third exam this year.
The result is that Wisconsin will have three different exams in three years, making it hard to gauge progress and impossible to calculate growth scores. DPI expanded this information desert by embargoing the public release of the Badger exam results, even while providing them to schools and districts. The legislature decided to skip school and district report cards for the 2014-15 school year. Thus there is little current information on such measures as proficiency rates, attendance, truancy, and dropout rates. Since the 2014-15 report cards were meant to be the first to include private schools, there is still no reliable way to compare the performance of public and private schools.
As if this isn’t messy enough, the legislature opened the door to each district choosing its own assessments. This will make it difficult if not impossible to compare schools and districts.
The single theme underlying Data Wonk articles is that good measurement should underlie decision making. To improve schools we need to understand which schools are serving students well and which are falling short. At best, research in education is inherently difficult. A school’s success may come down to difficult-to-quantify factors such as the commitment of the people involved or the skills of school leaders.
In addition, many people enter the fray with their own agenda, such as a commitment to a particular curriculum or the desire to avoid making uncomfortable adjustments. There is also the persistent belief that there is one model that will serve the needs of all students. Educational decisions, particularly in Milwaukee, are often dominated by people who regard data as their enemy.
Under Obama’s program, schools identified as in need of improvement could receive up to $2 million annually for three years if they adopted one of four strategies: replace the principal and at least 50 percent of the staff (called the “Turnaround Model”), transform the school through new instructional strategies and other techniques (“Transformation Model”), close the school and enroll students in another, better-performing school (“Closure Model”), or close the school and reopen it as a charter school (“Restart Model”).
The Council of the Great City Schools, whose members include large urban school districts including MPS, studied the outcomes of SIG schools. Its report found that both the transformation and the turnaround models led to improvement and that there were no statistically significant differences in the rates of improvement between these two models.
The authors also report that “we were not able to say anything about the relative effects of the restart or closure models because they were used so infrequently.” This is not surprising. Closing a school invariably generates intense opposition, no matter how bad the school is. This opposition is intensified when a charter school is proposed as its replacement.
The law establishing the OSPP closely follows the Obama administration’s Restart Model. The major difference is that the transition would be made by a commissioner appointed by the County Executive rather than the school district.
From public reports of discussions about the OSPP it appears that Abele and Means were trying to make the program more palatable to MPS by proposing something more along the lines of a Turnaround or Transformation Model. If so, it didn’t work.
In MPS, upwards of 50 schools are classified as “failed to meet expectations.” These are the schools eligible for the Opportunity Schools program. However, as already noted, the data this determination is based on is already two years out of date.
In the plot below, the vertical scale shows the 2013-14 DPI performance scores for Milwaukee schools (including charter schools but not choice schools). The horizontal scale shows the school’s poverty level (as measured by the percentage of students qualifying for free lunch). There is a clear relationship between the two. Schools with more middle class students are far more likely to be rated “meets expectations” or better.
However, this relationship is not completely rigid. Notice, in particular, that schools deemed as “meeting expectations” (those in green) serve a wide range of economic populations. Which raises the question: what are these schools doing that the others are not doing and what can be learned from them?
The message of this chart is two-fold: one is that poverty adds greatly to educational challenge. But the second is that this challenge is being overcome, today in Milwaukee, at many schools. Unfortunately In Milwaukee, there is a strong resistance to learn from these schools and use that knowledge to improve student outcomes.
Now that Means has resigned, what happens next? One scenario is that both Abele and the state legislators decide to drop the OSPP concept. In that case present trends will likely to continue. Enrollment in traditional MPS schools will continue to represent a declining portion of Milwaukee enrollment, as students switch to charter schools, private schools under the choice program, and suburban schools under open enrollment. As a result, membership in the Milwaukee Teachers’ Education Association will continue to decline. For example, a recent Public Policy Forum report on Milwaukee choice schools found their enrollment has grown from 15,435 in 2005-06 to 26,639 in 2014-15.
Another scenario is that the legislator transfers the OSPP’s powers to a new or existing charter school authorizer, whom they deem as more willing than Abele to proceed in the face of MPS opposition.
In any case, failure of the OSPP is likely to reinforce the opinion in the legislature that Milwaukee is incompetent to manage its own affairs. This perception has arguably been damaging in areas beyond education, as the legislature killed the city employee residency requirement, killed the Milwaukee inspection program for rental properties, threatened to gut Milwaukee’s historical protection powers, and repeatedly attempted to sabotage the Milwaukee streetcar. An inability to make hard choices at MPS may not only hurt Milwaukee educationally, but can lead to other negative results that hurt the city’s prosperity.