Abstract
Clustered observational studies (COSs) are a critical analytic tool for educational effectiveness research. We present a design framework for the development and critique of COSs. The framework is built on the counterfactual model for causal inference and promotes the concept of designing COSs that emulate the targeted randomized trial that would have been conducted were it feasible. We emphasize the key role of understanding the assignment mechanism to study design. We review methods for statistical adjustment and highlight a recently developed form of matching designed specifically for COSs. We review how regression models can be profitably combined with matching and note best practice for estimates of statistical uncertainty. Finally, we review how sensitivity analyses can determine whether conclusions are sensitive to bias from potential unobserved confounders. We demonstrate concepts with an evaluation of a summer school reading intervention in Wake County, North Carolina.
Matthew Lenard
PhD candidate in Education Policy & Program Evaluation
I am a Ph.D. candidate at the Harvard Graduate School of Education (HGSE) in its Education Policy and Program Evaluation concentration. My research centers on using causal inference research designs to measure the impacts of education programs and policies. I typically study topics at the intersection of education and economics, but also draw from the traditions of sociology and political science. My current projects focus on career and technical education, peer effects, school choice, and teacher labor markets.