The Annals of Applied Statistics

Evaluating the causal effect of university grants on student dropout: Evidence from a regression discontinuity design using principal stratification

Fan Li, Alessandra Mattei, and Fabrizia Mealli

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

Regression discontinuity (RD) designs are often interpreted as locally randomized experiments for units with a realized value of a pretreatment variable falling around a threshold. Motivated by the evaluation of Italian university grants, we consider a fuzzy RD design where the treatment status is based on both eligibility criteria and a voluntary application status. Resting on the fact that grant application and grant receipt statuses are post-assignment (post-eligibility) intermediate variables, we use the principal stratification framework to define causal estimands within the Rubin Causal Model. We propose a probabilistic formulation of the assignment mechanism underlying RD designs, by reformulating the Stable Unit Treatment Value Assumption (SUTVA) and making an explicit local overlap assumption for a subpopulation around the threshold. We invoke a local randomization assumption instead of the more standard continuity assumptions. We also develop a Bayesian approach to select the target subpopulation(s) with adjustment for multiple comparisons, and to draw inference for the target causal estimands within this framework. Applying the method to the data from two Italian universities, we find evidence that university grants are effective in preventing students from low-income families from dropping out of higher education.

Article information

Source
Ann. Appl. Stat., Volume 9, Number 4 (2015), 1906-1931.

Dates
Received: December 2014
Revised: September 2015
First available in Project Euclid: 28 January 2016

Permanent link to this document
https://projecteuclid.org/euclid.aoas/1453993098

Digital Object Identifier
doi:10.1214/15-AOAS881

Mathematical Reviews number (MathSciNet)
MR3456358

Zentralblatt MATH identifier
06560814

Keywords
Bayesian causal effects intermediate variables principal stratification randomization regression discontinuity university grants

Citation

Li, Fan; Mattei, Alessandra; Mealli, Fabrizia. Evaluating the causal effect of university grants on student dropout: Evidence from a regression discontinuity design using principal stratification. Ann. Appl. Stat. 9 (2015), no. 4, 1906--1931. doi:10.1214/15-AOAS881. https://projecteuclid.org/euclid.aoas/1453993098


Export citation

References

  • Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Assoc. 88 669–679.
  • Angrist, J. D., Imbens, G. W. and Rubin, D. B. (1996). Identification of causal effects using instrumental variables (with discussion). J. Amer. Statist. Assoc. 91 444–472.
  • Battistin, E. and Rettore, E. (2008). Ineligibles and eligible non-participants as a double comparison group in regression discontinuity designs. J. Econometrics 142 715–730.
  • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300.
  • Berk, R. A. and de Leuuw, J. (1999). An evaluation of California’s inmate classification system using a generalized regression discontinuity design. J. Amer. Statist. Assoc. 94 1045–1052.
  • Berry, S. M. and Berry, D. A. (2004). Accounting for multiplicities in assessing drug safety: A three-level hierarchical mixture model. Biometrics 60 418–426.
  • Cattaneo, M. D., Frandsen, B. and Titiunik, R. (2015). Randomization inference in the regression discontinuity design: An application to party advantages in the U.S. Senate. Journal of Causal Inference 3 1–24.
  • Cellini, S. R., Ferreira, F. and Rothstein, J. (2010). The value of school facility investments: Evidence from a dynamic regression discontinuity design. Q. J. Econ. 125 215–261.
  • Chib, S. and Greenberg, E. (2014). Nonparametric Bayes analysis of the sharp and fuzzy regression discontinuity designs. Technical report, Washington Univ. St Louis, Olin School of Business.
  • Chib, S. and Jacobi, L. (2015). Bayesian fuzzy regression discontinuity analysis and returns to compulsory schooling. J. Appl. Econometrics. Published online in Wiley Online Library (wileyonlinelibrary.com), DOI:10.1002/jae.2481.
  • Cook, T. D. (2008). “Waiting for life to arrive”: A history of the regression-discontinuity design in psychology, statistics and economics. J. Econometrics 142 636–654.
  • de Finetti, B. (1937). La prévision: ses lois logiques, ses sources subjectives. Ann. Inst. Henri Poincaré 7 1–68.
  • Dinardo, J. and Lee, D. S. (2011). Program evaluation and research designs. In Handbook of Labor Economics 4A 463–536. Elsevier, Philadelphia, PA.
  • Elliott, M. R., Raghunathan, T. E. and Li, Y. (2010). Bayesian inference for causal mediation effects using principal stratification with dichotomous mediators and outcomes. Biostatistics 11 353–372.
  • Frangakis, C. E. and Rubin, D. B. (2002). Principal stratification in causal inference. Biometrics 58 21–29.
  • Frumento, P., Mealli, F., Pacini, B. and Rubin, D. B. (2012). Evaluating the effect of training on wages in the presence of noncompliance, nonemployment, and missing outcome data. J. Amer. Statist. Assoc. 107 450–466.
  • Garibaldi, P., Giavazzi, F., Ichino, A. and Rettore, E. (2012). College cost and time to complete a degree: Evidence from tuition discontinuities. Rev. Econ. Stat. 94 699–711.
  • Gelman, A., Meng, X.-L. and Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statist. Sinica 6 733–807.
  • Gelman, A. E. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statist. Sci. 7 457–472.
  • Gosselin, F. (2011). A new calibrated Bayesian internal goodness-of-fit method: Sampled posterior $p$-values as simple and general $p$-values that allow double use of the data. PLoS ONE 6 1–10.
  • Hahn, J., Todd, P. E. and Van der Klaauw, W. (2001). Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica 69 201–209.
  • Imbens, G. W. (2004). Nonparametric estimation of average treatment effects under exogeneity: A review. The Review of Economics and Statistics 86 4–29.
  • Imbens, G. W. and Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica 62 467–476.
  • Imbens, G. and Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. Rev. Econ. Stud. 79 933–959.
  • Imbens, G. W. and Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. J. Econometrics 142 615–635.
  • Imbens, G. W. and Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with noncompliance. Ann. Statist. 25 305–327.
  • Imbens, G. W. and Rubin, D. B. (2015). Causal Inference—for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge Univ. Press, New York.
  • Imbens, G. W. and Zajonc, T. (2011). Regression discontinuity design with multiple forcing variables. Technical report, Harvard Univ., Dept. Economics.
  • Johnson, V. E. (2007). Bayesian model assessment using pivotal quantities. Bayesian Anal. 2 719–733.
  • Lee, D. S. (2008). Randomized experiments from non-random selection in U.S. House elections. J. Econometrics 142 675–697.
  • Lee, D. S. and Lemieux, T. (2010). Regression discontinuity designs in economics. J. Econ. Lit. 485 281–355.
  • Li, F., Mattei, A. and Mealli, F. (2015). Supplement to “Evaluating the causal effect of university grants on student dropout: Evidence from a regression discontinuity design using principal stratification.” DOI:10.1214/15-AOAS881SUPP.
  • Ludwig, J. and Miller, D. L. (2007). Does head start improve children’s life chances? Evidence from a regression discontinuity design. Q. J. Econ. 122 15981–208.
  • Mattei, A., Li, F. and Mealli, F. (2013). Exploiting multiple outcomes in Bayesian principal stratification analysis with application to the evaluation of a job training program. Ann. Appl. Stat. 7 2336–2360.
  • Mealli, F. and Pacini, B. (2013). Using secondary outcomes to sharpen inference in randomized experiments with noncompliance. J. Amer. Statist. Assoc. 108 1120–1131.
  • Mealli, F. and Rampichini, C. (2012). Evaluating the effects of university grants by using regression discontinuity designs. J. Roy. Statist. Soc. Ser. A 175 775–798.
  • Mealli, F. and Rubin, B. D. (2002). Assumptions when analyzing randomized experiments with noncompliance and missing outcomes. Health Serv. Outcomes Res. Methodol. 3 225–232.
  • Mercatanti, A. (2013). A likelihood-based analysis for relaxing the exclusion restriction in randomized experiments with noncompliance. Aust. N. Z. J. Stat. 55 129–153.
  • Mercatanti, A., Li, F. and Mealli, F. (2015). Improving inference of Gaussian mixtures using auxiliary variables. Stat. Anal. Data Min. 8 34–48.
  • Murphy, S. A. (2003). Optimal dynamic treatment regimes. J. R. Stat. Soc. Ser. B Stat. Methodol. 65 331–366.
  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psycology 66 688–701.
  • Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Ann. Statist. 6 34–58.
  • Rubin, D. B. (1980). Discussion of “Randomization analysis of experimental data: The Fisher randomization test” by D. Basu. J. Amer. Statist. Assoc. 75 591–593.
  • Sales, A. and Hansen, B. (2014). Limitless regression discontinuity: Causal inference for a population surrounding a threshold. Available at arXiv:1403.5478.
  • Schwartz, S. L., Li, F. and Mealli, F. (2011). A Bayesian semiparametric approach to intermediate variables in causal inference. J. Amer. Statist. Assoc. 31 949–962.
  • Scott, J. G. and Berger, J. O. (2006). An exploration of aspects of Bayesian multiple testing. J. Statist. Plann. Inference 136 2144–2162.
  • Thistlethwaite, D. and Campbell, D. (1960). Regression-discontinuity analysis: An alternative to the ex-post facto experiment. J. Educ. Psychol. 51 309–317.
  • van der Klaauw, W. (2002). Estimating the effect of financial aid offers on college enrollment: A regression-discontinuity approach. Internat. Econom. Rev. 43 1249–1287.
  • van der Klaauw, W. (2008). Regression-discontinuity analysis: A survey of recent development in economics. Labour 22 219–245.
  • Zajonc, T. (2012). Bayesian inference for dynamic treatment regimes: Mobility, equity, and efficiency in student tracking. J. Amer. Statist. Assoc. 107 80–92.

Supplemental materials

  • Supplement to “Evaluating the causal effect of university grants on student dropout: Evidence from a regression discontinuity design using principal stratification”. We describe in detail the Bayesian approach we used to select the subpopulations, the Markov Chain Monte Carlo (MCMC) methods used to simulate the posterior distributions of the parameters of the models, the posterior predictive checks, and the sensitivity analysis regarding local randomization described in Section 6.