imageHow do you measure the value of a charter school?Reuters/Lee Celano

Imagine a police officer pulls you over and tickets you for speeding. She tells you she measured you going 50.5 MPH in a 50 MPH zone. No, you reply, my speedometer shows that I was going exactly 49.5. The entire discussion would be absurd, since neither your speedometer nor the officer’s radar gun is sufficiently accurate to support the opposing claims, and a 0.5 MPH difference is not practically meaningful.

Welcome to the world of debates about the effects of American charter schools on children’s test scores. This was all underscored last year when advocates of charter schools – which are publicly funded but independent – reacted to a high-profile study released by CREDO, a charter-friendly research unit connected to Stanford University. The study showed great variation between different charter schools and between states.

The CREDO study also offered an overall national estimate showing students in charter schools scoring approximately 0.01 standard deviations higher on reading tests and 0.005 standard deviations lower on math tests than their peers in conventional public schools (the former being statistically significant; the latter not). These findings were, in fact, highly consistent with an overall body of researchconcluding that the test-score outcomes of charter schools and public schools are almost identical.

Notwithstanding these pedestrian results as well as some limitations and problems with the study itself, the results were seized upon by advocates and resulted in headlines such as, Stanford Study Says Charter School Children Outperform and Study: Minority, poor students gain from charters.

Bangs for the buck

Even more recently, the staunchly pro-charter Department of Education Reform at the University of Arkansas attempted to shift the discussion to issues of productivity, arguing that charter outcomes look more impressive when we take inputs such as resources into account. In a report called The Productivity of Public Charter Schools they put forward the contention that charter funding per child is relatively low compared to regular public schools, so the charter sector is more productive.

This report generated headlines such as: A bigger bang for school bucks and Charter schools: More bang for the buck. But huge problems undermined the Arkansas report’s figures for both financial inputs into schools and outputs such as test scores.

This brings us right back to the speeding ticket. The lack of controlled laboratory conditions for this kind of research overwhelms attempts to isolate small differences that might be inherent in charter versus non-charter status. Since many school districts, for example, are called upon to provide services such as transportation to charter schools, any attempt to measure the resources put into the charter system in those jurisdictions should assign a cost to the school districts and a benefit to the charter schools. Yet the Arkansas report does the opposite.

Sensible estimates on school outputs, such as analyses of test scores, would attempt to compare like with like – comparing schools with similar demographics like poverty (measured by free lunch eligibility) or special-needs status – but the Arkansas report fails to do that too.

Even analyses that do try to control for such variables come up short, given that part of the variance is undoubtedly due to non-random but unmeasured factors – such as parental differences linked to motivation for seeking out a charter school.

More variation between charters

Some researchers have attempted to get around these bias issues by studying charter schools that are over-enrolled and that use lotteries to select which students to admit. But these studies suffer from at least two limitations.

First, they only allow for studies of over-enrolled, popular charter schools, thus undermining the ability of the research findings to be generalized to the larger population that includes less popular charter schools.

Second, parents tend to wreak havoc on the control groups in such studies by continuing to seek out other options, whether they be other charters, selective public schools, or even private schools. In other words, it’s difficult to get a clear picture.

In any case, while researchers aspire to make sweeping conclusions that compare the charter sector to the traditional public school sector, the variation among schools of the same type swamps any differences between charter and public schools that we might be able to tease out.

This shouldn’t come as a surprise. Students learn when they have opportunities to learn, and the quality and quantity of those opportunities vary greatly among different charter schools and among different traditional public schools. Our own research at the National Education Policy Center suggests very large resource differences among charter schools. This is in accord with longstanding research showing such differences among traditional public schools.

Feeding research into policy

None of this means research cannot help us understand charter schools. The fact that the studies that do exist find little overall difference between charters and non-charters is itself important. Even with the limitations of any given study, this finding has consistently held up. The entirely predictable finding of variations within the charter sector also helpfully points us to more useful areas of inquiry. For example, how do well-resourced charters use those extra resources to extend or enrich learning time?

Advocates use research as a tool, and charter school research is inextricably tied to charter school policy. For this reason and others, we can expect that the heated debate will continue even while more research continues. In the meantime, those in charge of education policy need to make decisions about whether to encourage the expansion (or contraction) of charter schools, whether they should be funded at higher (or lower) levels, and whether they should be subjected to regulation (or further deregulation). Keep a close eye on the speedometer.


The National Education Policy Center (NEPC), which Kevin Welner directs, has received a grant from The Atlantic Philanthropies to study issues of access to charter schools. NEPC funding from the Ford Foundation and the Great Lakes Center for Education Research and Practice has also at times been used to commission work related to charter schools.

Read more

3 Tips for Improving Your Trading Game

No matter in which profession you are in, you must need to improve yourself day by day. Even if you are a businessman then you must need to expand your...

Google and Facebook shouldn't subsidise journalism, but the government could

Jeff Chiu/APYou might have missed it – what with the biggest recession since the 1930s and a pandemic going on – but there may be big, and bad, changes happening to a media landscape near ...

The Conversation - avatar The Conversation

Testamentary trusts are one of the last truly outrageous means of avoiding tax

Scott Graham/UpsplashIt’s one thing to bring forward tax cuts, as the government is thinking of doing in the October budget.It’s another to leave wide open an arrangement that allows subst...

The Conversation - avatar The Conversation

Homes are flooding outside FEMA's 100-year flood zones, and racial inequality is showing through

Hurricane Harvey showed the racial disparities in flood damage outside Houston's 100-year flood zones.Joe Raedle/Getty ImagesWhen hurricanes and other extreme storms unleash downpours like Tropical St...

Kevin T. Smiley, Assistant Professor of Sociology, Louisiana State University - avatar Kevin T. Smiley, Assistant Professor of Sociology, Louisiana State University

In death, as in life, Ruth Bader Ginsburg balanced being American and Jewish

No lengthy viewing of the body, but no quick burial either.Andrew Harnik-Pool/Getty ImagesAs news of the passing of Justice Ruth Bader Ginsburg spread on the eve of Rosh Hashana, the Jewish New Year, ...

Jonathan D. Sarna, University Professor and Joseph H. & Belle R. Braun Professor of American Jewish History, Brandeis University - avatar Jonathan D. Sarna, University Professor and Joseph H. & Belle R. Braun Professor of American Jewish History, Brandeis University

Los trolls políticos se adaptan: crean nuevo material para engañar y confundir más a la audiencia

Los trolls se ponen creativos con su decepción electoral.Planet Flem/DigitalVision Vectors via Getty ImagesLos trolls de Twitter patrocinados por Rusia, que explotaron tan agresivamente las red...

Gianluca Stringhini, Assistant Professor of Electrical and Computer Engineering, Boston University - avatar Gianluca Stringhini, Assistant Professor of Electrical and Computer Engineering, Boston University

Writers Wanted