September 3, 2015
What do effective charter schools have in common?
Research tells us to look beyond traditional metrics like class size
A volunteer tutor works with students in Washington, D.C. Frequent tutoring in small groups, termed "high-dosage tutoring" by the authors, was more common in high-achieving New York City charter schools than low-achieving ones.
U.S. Department of Education/Flickr
In the contentious debate surrounding charter schools, a shaky consensus has emerged: it is not clear if charter schools are decisively better or worse than public schools at raising achievement on average, but they tend to vary widely in quality, with many standouts and laggards across the country. Such a result isn't surprising because charter schools are given more latitude to change priorities and focus resources in novel ways. But this divergence between the best and worst charter schools prompts a new question: what precisely are the good schools doing right? A 2013 article from the American Economic Journal: Applied Economics analyzes a set of charter schools in New York City to see what makes effective schools tick.
In Getting beneath the Veil of Effective Schools: Evidence from New York City (PDF), authors Will Dobbie and Roland Fryer assemble a dataset that describes the inner workings of 39 charter schools in New York City. The authors analyzed interviews with principals, teachers, and students and videotaped classroom observations to expand their dataset beyond what is typically available in administrative records.
These sources allowed them to construct highly specific variables of interest, ranging from how often teachers receive written feedback from superiors to what fraction of students are off-task or physically slouching during class. Such measures capture aspects of school culture and day-to-day life that are not readily quantified but which might be key to understanding the differences between effective schools and ineffective ones. The authors combined this data with information on traditional measures like student-to-teacher ratios and per-pupil expenditures.
Measures of teacher development, data driven instruction, school culture, and student expectations have never been collected systematically, despite decades of qualitative research suggesting their importance.
Dobbie and Fryer (2013)
With this rich dataset in hand, Dobbie and Fryer split their sample into two groups: "high-achieving schools" that have an above-average impact on math and language arts scores, and "low-achieving schools" with a below-average impact (the authors measure test score impact by comparing students who gained entrance to the schools through lotteries with students who did not win spots).
The high-achieving schools actually rank slightly worse by traditional measures of school quality: they tend to have larger class sizes and fewer teachers with advanced degrees. But as the figure below shows, the high-performers are much more likely to implement practices that some advocates think are important for success, like frequent teacher feedback, increased instructional time, and intensive tutoring regimens. All told, an index of five such measures is much more predictive of school quality than an index of four traditional measures.
What practices are more common at the best schools?
A comparison of policies at 26 charter elementary schools in New York City that were identified as high-achieving or low-achieving by the authors.
These results can’t be interpreted as causal because the attributes were not assigned to schools as in a randomized experiment; instead, they were chosen by the schools themselves. If floundering schools tend to reduce class sizes in an effort to improve, for example, that would create a correlation between poor performance and small class sizes, even if reducing class size is actually beneficial (the evidence on that point is quite mixed). Meanwhile, practices like intensive tutoring or data-driven instruction might only be implemented if a school already has highly motivated students and teachers.
While the findings do not imply that low-achieving schools will necessarily improve overnight if they simply adopt these practices, they do suggest a possible path for failing schools to take. The results of this study helped convince officials in Houston and Denver to partner with one of the authors to inject these "best practices" into their public school systems.
Because charter schools differ from public schools in several important ways—teachers are typically not unionized, for example—it was not clear if these findings would translate to public schools. Nevertheless, Fryer found that adopting the practices at Houston schools significantly raised student achievement on math exams (he discussed how this paper fits into a broader research agenda at the annual ASSA meeting in January). Given their success in Houston, these practices will probably get a closer look in school systems across the country. ♦
"Getting Beneath the Veil of Effective Schools: Evidence from New York City" appears in the October 2013 issue of the American Economic Journal: Applied Economics.