The Charter School Advantage

The recently issued Urban Charter School Study from Stanford’s Center for Research on Education Outcomes (CREDO) has good news and bad news for Milwaukee education. The good: as a group Milwaukee charter schools are outperforming traditional public schools. The bad: this differential is not great enough to bring those students up to the level of average students statewide.

A word about the study’s methodology. Most studies of charter school outcomes depend on finding schools with more applicants than available seats. Under the law, these schools are required to choose their students using a random lottery. The studies track the two groups of students—those randomly accepted and those rejected—and compare their subsequent progress.

This traditional approach has two limitations. First, it limits the number of charter schools that can be studied to those with a surplus of applicants. Yet most charter schools are able to accommodate all applicants. The second—and more serious—is that it may not tell much about other charter schools. If parents pick charter schools that are most effective (the evidence for this is weak) the sample of charter schools itself may be biased, with the most effective charter schools.

CREDO gets around this limitation by matching a database of a million charter school student records with a comparison group of traditional public school students. As the CREDO report explains, its “matching process uses the Virtual Control Record (VCR) protocol, matching each charter student with up to seven traditional public school students based on prior test scores and demographic characteristics.” The report says it is thereby able to match over 80 percent of the charter school students in the analysis.

Charter School Effect vs Traditional
Public Schools

CREDO applied this methodology to 41 urban regions, including Milwaukee. There was considerable variation in the results from one region to another. In a few regions, charter school students did more poorly than those in traditional schools, but on average the charter school students gained more than the public school peers. Milwaukee charter schools followed the national pattern.

The chart to the right shows the charter school advantage for all regions and for Milwaukee in mathematics and reading. On average, Milwaukee charter school students annually gained the equivalent of about 65 extra days of learning in math compared to students in traditional public schools. In reading they gained an extra 29 days.

The report also breaks out the charter school effect for various sub-groups. On the whole students in groups considered disadvantaged had a greater charter school effect, but there was considerable variation.

Charter vs Traditional Schools–Math

Nationally, the extra days of learning increased the longer a student was in a charter school. The report did not break out this effect by region.

When individual schools were compared to their traditional school peers, Milwaukee charter schools stacked up well. Only 3 percent of Milwaukee charter schools did worse in math, while 60 percent did better, as shown in the graph to the right.

There were similar school results for reading, although the percentage of schools rated better drops to about 40 percent.

The bad news is that Milwaukee students have a very large deficit compared to the rest of the states and the charter school advantage is not sufficient to overcome this. As the report says:

…students enrolled in charter schools in Cleveland, Miami, and Milwaukee can expect to see higher levels of academic growth than expected in their region’s local TPS, but this charter lift is not enough for the average charter student to offset the achievement deficit of the region relative to the rest of the state in both math and reading.

Charter Effect vs Region Achievement

By contrast the report identifies regions faced with large region-wide achievement deficits relative to their state’s average, but whose “charter sectors appear to provide their students with strong enough annual growth in both math and reading that continuous enrollment in an average charter school can erase the typical deficit seen among students in their region.”

The CREDO report does not identify the individual charter schools in which an individual student is likely to do better — or worse. The Wisconsin Department of Public Instruction does publish school report cards that rate schools on various measures. Unlike the CREDO methodology these do not attempt to untangle the school effect from the student effect. In an earlier report, I listed all the Milwaukee schools—traditional and charters and how they did on two scores from the Department of Public Instruction. There was a strong correlation between the DPI Achievement score and the prosperity of the families a school serves.

DPI Achievement Scores

There are three active charter authorizers in Milwaukee: Milwaukee Public Schools, the Milwaukee Common Council, and UW-Milwaukee (a fourth potential authorizer, Milwaukee Area Technical College, has shown no interest in authorizing charter schools). In addition, schools authorized by MPS fall into two groups: instrumentality charter schools in which teachers and other staff are employed by MPS and non-instrumentality charters which hire their own employees.

The chart to the right shows the average DPI Achievement score for each of these groups of schools. As a group, charters authorized by UWM outshine all the others on this measure. Recently the charter program run by the Milwaukee Common Council has come under attack by anti-charter groups because several of its schools fared poorly on the Achievement Scores. Still, on average, its schools outscored MPS regular schools and non-instrumentality schools.

DPI Growth Scores

Probably the DPI scores that comes closest to the CREDO methodology are its Growth scores in reading and math. These are based on how well individual students’ test scores improve from one year to the next. These also have limitations; for instance, high schools are excluded since they only test 10th grade students. The Growth scores shows City of Milwaukee charters with the highest average scores, likely reflecting the emphasis placed on student growth by the city’s consultants. As the chart to the right shows, all segments of charter schools outperform MPS regular schools when it comes to student math and reading growth.

The MPS instrumentality charter schools exist in a kind of limbo between regular traditional public schools and true charter schools. (CREDO tells me they treated them as charter schools.) So far as I know, none of the instrumentality schools has a board with legal responsibility for governance of the school. Applicants for instrumentality charter school status have mainly come from two groups. The first are existing MPS schools looking for more autonomy and hoping to win chart development grants. The second is MPS teachers with a particular vision for a school or a desire to serve a particular population.

Instrumentality Charter Schools

Over the years, instrumentality charters have suffered considerable attrition. Of 43 that opened, only seven remain, three converted traditional schools and four small teacher-led schools. As the chart to the right shows, the others were closed or reverted back to traditional schools.

One likely factor in the high attrition rate, particularly with the teacher-led schools, is that the union contract made it hard to assure that new staff shared the vision of the founders, particularly during layoffs when senior teachers had bumping rights.

It is unclear whether charter status had a positive effect on student achievement at instrumentality charters, as might be implied by their higher average Achievement and Growth scores compared to traditional MPS schools. The higher score is likely to reflect selection bias—that low-achieving instrumentality schools were switched back to traditional schools.

MPS Non-instrumentality Charter
Schools

Attrition among the MPS non-instrumentality charter schools has been much lower, as shown in the graph to the right. A majority of the schools that opened are still running.

It is unclear whether MPS will continue to be a major player in the world of charters. The tone on the Milwaukee School Board (on which I once served) has become increasingly hostile to instrumentality charters, particularly if the proposal comes from someone with past associations with any of the national education reform groups, such as Teach for America, or from one of the national charter organizations. The fact that this past year MPS received only two applications for charter schools (compared to an average of about 9 per year since 200) suggests the word is out that MPS is not welcoming of charters.

The results from the CREDO study and previous research, as well as the DPI report cards, suggest that charter schools can contribute to better outcomes for Milwaukee’s children. However, governance is no magic elixir that in itself allows a school to master the challenges facing urban education. Yet it seems clear that the freedom allowed by the charter school approach can lead to interesting results if allowed to flourish.