Abstract
Many interventions in education occur in settings where treatments are applied to groups. For example, a reading intervention may be implemented for all students in some schools and withheld from students in other schools. When such treatments are nonrandomly allocated, outcomes across the treated and control groups may differ due to the treatment or due to baseline differences between groups. When this is the case, researchers can use statistical adjustment to make treated and control groups similar in terms of observed characteristics. Recent work in statistics has developed matching methods designed for contexts where treatments are clustered. This form of matching, known as multilevel matching, may be well suited to many education applications where treatments are assigned to schools. In this article, we provide an extensive evaluation of multilevel matching and compare it to multilevel regression modeling. We evaluate multilevel matching methods in two ways. First, we use these matching methods in a within-study comparison design and attempt to recover treatment effect estimates from three clustered randomized trials. Second, we conduct a simulation study. We find evidence that generally favors an analytic approach to statistical adjustment that combines multilevel matching with regression adjustment. We conclude with an empirical application.
Matthew Lenard
PhD candidate in Education Policy & Program Evaluation
I am a Ph.D. candidate at the Harvard Graduate School of Education (HGSE) in its Education Policy and Program Evaluation concentration. My research centers on using causal inference research designs to measure the impacts of education programs and policies. I typically study topics at the intersection of education and economics, but also draw from the traditions of sociology and political science. My current projects focus on career and technical education, peer effects, school choice, and teacher labor markets.