Bayesian Analysis

Regularization and Confounding in Linear Regression for Treatment Effect Estimation

P. Richard Hahn, Carlos M. Carvalho, David Puelz, and Jingyu He

Full-text: Open access


This paper investigates the use of regularization priors in the context of treatment effect estimation using observational data where the number of control variables is large relative to the number of observations. First, the phenomenon of “regularization-induced confounding” is introduced, which refers to the tendency of regularization priors to adversely bias treatment effect estimates by over-shrinking control variable regression coefficients. Then, a simultaneous regression model is presented which permits regularization priors to be specified in a way that avoids this unintentional “re-confounding”. The new model is illustrated on synthetic and empirical data.

Article information

Bayesian Anal., Volume 13, Number 1 (2018), 163-182.

First available in Project Euclid: 11 January 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

causal inference observational data shrinkage estimation

Creative Commons Attribution 4.0 International License.


Hahn, P. Richard; Carvalho, Carlos M.; Puelz, David; He, Jingyu. Regularization and Confounding in Linear Regression for Treatment Effect Estimation. Bayesian Anal. 13 (2018), no. 1, 163--182. doi:10.1214/16-BA1044.

Export citation


  • An, W. (2010). “Bayesian propensity score estimators: incorporating uncertainties in propensity scores into causal inference.” Sociological Methodology, 40(1): 151–189.
  • Belloni, A., Chernozhukov, V., and Hansen, C. (2014). “Inference on treatment effects after selection among high-dimensional controls.” The Review of Economic Studies, 81(2): 608–650.
  • Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). “The horseshoe estimator for sparse signals.” Biometrika, 97(2): 465–480.
  • Donohue III, J. J. and Levitt, S. D. (2001). “The impact of legalized abortion on crime.” Quarterly Journal of Economics, 379–420.
  • Ertefaie, A., Asgharian, M., and Stephens, D. (2015). “Variable selection in causal inference using a simultaneous penalization method.” arXiv:1511.08501.
  • Ghosh, D., Zhu, Y., and Coffman, D. L. (2015). “Penalized regression procedures for variable selection in the potential outcomes framework.” Statistics in Medicine, 34(10): 1645–1658.
  • Hahn, P. R., He, J., and Lopes, H. (2016). “Elliptical slice sampling for Bayesian shrinkage regression with applications to causal inference.” Technical report, University of Chicago Booth School of Business.
  • Hahn, P. R., Carvalho, C. M., He, J., and Puelz, D. (2016). “Supplement to “Regularization and confounding in linear regression for treatment effect estimation”.” Bayesian Analysis.
  • Heckman, J. J., Lopes, H. F., and Piatek, R. (2014). “Treatment effects: A Bayesian perspective.” Econometric reviews, 33(1–4): 36–67.
  • Imbens, G. W. and Rubin, D. B. (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.
  • Jacobi, L., Wagner, H., and Frühwirth-Schnatter, S. (2016). “Bayesian treatment effects models with variable selection for panel outcomes with an application to earnings effects of maternity leave.” Journal of Econometrics, 193(1): 234–250.
  • Leamer, E. E. (1978). Specification searches: Ad hoc inference with nonexperimental data, volume 53. John Wiley & Sons Incorporated.
  • Leamer, E. E. (1983). “Let’s take the con out of econometrics.” The American Economic Review, 73(1): 31–43.
  • Lefebvre, G., Atherton, J., and Talbot, D. (2014). “The effect of the prior distribution in the Bayesian Adjustment for Confounding algorithm.” Computational Statistics & Data Analysis, 70: 227–240.
  • Levitt, S. D. and Dubner, S. J. (2010). Freakonomics, volume 61. Sperling & Kupfer editori.
  • Li, M. and Tobias, J. L. (2014). “Bayesian analysis of treatment effect models.” Bayesian Inference in the Social Sciences, 63–90.
  • Liang, F., Paulo, R., Molina, G., Clyde, M. A., and Berger, J. O. (2008). “Mixtures of $g$ priors for Bayesian variable selection.” Journal of the American Statistical Association, 103(481): 410–423.
  • McCandless, L. C., Gustafson, P., and Austin, P. C. (2009). “Bayesian propensity score analysis for observational data.” Statistics in Medicine, 28(1): 94–112.
  • Morgan, S. L. and Winship, C. (2014). Counterfactuals and causal inference. Cambridge University Press.
  • Robins, J. M., Mark, S. D., and Newey, W. K. (1992). “Estimating exposure effects by modelling the expectation of exposure conditional on confounders.” Biometrics, 479–495.
  • Rosenbaum, P. R. and Rubin, D. B. (1983). “The central role of the propensity score in observational studies for causal effects.” Biometrika, 70(1): 41–55.
  • Talbot, D., Lefebvre, G., and Atherton, J. (2015). “The Bayesian causal effect estimation algorithm.” Journal of Causal Inference, 3(2): 207–236.
  • Wang, C., Dominici, F., Parmigiani, G., and Zigler, C. M. (2015). “Accounting for uncertainty in confounder and effect modifier selection when estimating average causal effects in generalized linear models.” Biometrics, 71(3): 654–665.
  • Wang, C., Parmigiani, G., and Dominici, F. (2012). “Bayesian effect estimation accounting for adjustment uncertainty.” Biometrics, 68(3): 661–671.
  • Wilson, A. and Reich, B. J. (2014). “Confounder selection via penalized credible regions.” Biometrics, 70(4): 852–861.
  • Zellner, A. (1986). “On assessing prior distributions and Bayesian regression analysis with g-prior distributions.” Bayesian Inference and Decision Techniques: Essays in Honor of Bruno De Finetti, 6: 233–243.
  • Zigler, C. M. and Dominici, F. (2014). “Uncertainty in propensity score estimation: Bayesian methods for variable selection and model-averaged causal effects.” Journal of the American Statistical Association, 109(505): 95–107.

Supplemental materials