Bayesian Analysis

A New Bayesian Single Index Model with or without Covariates Missing at Random

Kumaresh Dhara, Stuart Lipsitz, Debdeep Pati, and Debajyoti Sinha

Advance publication

This article is in its final form and can be cited using the date of online publication and the DOI.

Full-text: Open access

Abstract

For many biomedical, environmental, and economic studies, the single index model provides a practical dimension reaction as well as a good physical interpretation of the unknown nonlinear relationship between the response and its multiple predictors. However, widespread uses of existing Bayesian analysis for such models are lacking in practice due to some major impediments, including slow mixing of the Markov Chain Monte Carlo (MCMC), the inability to deal with missing covariates and a lack of theoretical justification of the rate of convergence of Bayesian estimates. We present a new Bayesian single index model with an associated MCMC algorithm that incorporates an efficient Metropolis–Hastings (MH) step for the conditional distribution of the index vector. Our method leads to a model with good interpretations and prediction, implementable Bayesian inference, fast convergence of the MCMC and a first-time extension to accommodate missing covariates. We also obtain, for the first time, the set of sufficient conditions for obtaining the optimal rate of posterior convergence of the overall regression function. We illustrate the practical advantages of our method and computational tool via reanalysis of an environmental study.

Article information

Source
Bayesian Anal., Advance publication (2018), 22 pages.

Dates
First available in Project Euclid: 6 August 2019

Permanent link to this document
https://projecteuclid.org/euclid.ba/1565056835

Digital Object Identifier
doi:10.1214/19-BA1170

Subjects
Primary: 62H12: Estimation
Secondary: 62G08: Nonparametric regression

Keywords
Markov Chain Monte Carlo missing covariates Gaussian process mode aligned proposal density importance sampling

Rights
Creative Commons Attribution 4.0 International License.

Citation

Dhara, Kumaresh; Lipsitz, Stuart; Pati, Debdeep; Sinha, Debajyoti. A New Bayesian Single Index Model with or without Covariates Missing at Random. Bayesian Anal., advance publication, 6 August 2019. doi:10.1214/19-BA1170. https://projecteuclid.org/euclid.ba/1565056835


Export citation

References

  • Antoniadis, A., Grégoire, G., and McKeague, I. W. (2004). “Bayesian estimation in single-index models.” Statistica Sinica, 14(4): 1147–1164.
  • Chambers, J. M. (1983). Graphical methods for data analysis.
  • Choi, T., Shi, J. Q., and Wang, B. (2011). “A Gaussian process regression approach to a single-index model.” Journal of Nonparametric Statistics, 23(1): 21–36.
  • Dhara, K., Lipsitz, S., Pati, D., and Sinha, D. (2019). “Supplementary materials for “A New Bayesian Single Index Model with or without Covariates Missing at Random”.” Bayesian Analysis.
  • Friedman, J., Hastie, T., and Tibshirani, R. (2001). The elements of statistical learning, volume 1. Springer series in statistics Springer, Berlin.
  • Gramacy, R. B. and Lian, H. (2012). “Gaussian process single-index models as emulators for computer experiments.” Technometrics, 54(1): 30–41.
  • Guo, X., Niu, C., Yang, Y., and Xu, W. (2015). “Empirical likelihood for single index model with missing covariates at random.” Statistics, 49(3): 588–601.
  • Guo, X., Wang, T., and Zhu, L. (2016). “Model checking for parametric single-index models: a dimension reduction model-adaptive approach.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78(5): 1013–1035.
  • Guo, X., Xu, W., and Zhu, L. (2014). “Multi-index regression models with missing covariates at random.” Journal of Multivariate Analysis, 123: 345–363.
  • Hardle, W., Hall, P., Ichimura, H., et al. (1993). “Optimal smoothing in single-index models.” The Annals of Statistics, 21(1): 157–178.
  • Hristache, M., Juditsky, A., and Spokoiny, V. (2001). “Direct estimation of the index coefficient in a single-index model.” The Annals of Statistics, 595–623.
  • Ichimura, H. (1993). “Semiparametric least squares (SLS) and weighted SLS estimation of single-index models.” Journal of Econometrics, 58(1): 71–120.
  • Li, K.-C. (1991). “Sliced inverse regression for dimension reduction.” Journal of the American Statistical Association, 86(414): 316–327.
  • Li, K.-C. and Duan, N. (1989). “Regression analysis under link violation.” The Annals of Statistics, 1009–1052.
  • Li, T. and Yang, H. (2016). “Inverse probability weighted estimators for single-index models with missing covariates.” Communications in Statistics-Theory and Methods, 45(5): 1199–1214.
  • Lin, W. and Kulasekera, K. (2007). “Identifiability of single-index models and additive-index models.” Biometrika, 94(2): 496–501.
  • Little, R. J. and Rubin, D. B. (2014). Statistical analysis with missing data. John Wiley & Sons.
  • Murray, I. and Adams, R. P. (2010). “Slice sampling covariance hyperparameters of latent Gaussian models.” In Advances in Neural Information Processing Systems, 1732–1740.
  • Niu, C., Zhu, L., et al. (2017). “An adaptive-to-model test for parametric single-index models with missing responses.” Electronic Journal of Statistics, 11(1): 1491–1526.
  • Park, C. G., Vannucci, M., and Hart, J. D. (2005). “Bayesian methods for wavelet series in single-index models.” Journal of Computational and Graphical Statistics, 14(4).
  • Powell, J. L., Stock, J. H., and Stoker, T. M. (1989). “Semiparametric estimation of index coefficients.” Econometrica: Journal of the Econometric Society, 1403–1430.
  • Stoker, T. M. (1986). “Consistent estimation of scaled coefficients.” Econometrica: Journal of the Econometric Society, 1461–1481.
  • Wang, H.-B. (2009). “Bayesian estimation and variable selection for single index models.” Computational Statistics & Data Analysis, 53(7): 2617–2627.
  • Xia, Y. (2006). “Asymptotic distributions for two estimators of the single-index model.” Econometric Theory, 22(6): 1112–1137.
  • Xia, Y., Tong, H., Li, W., and Zhu, L.-X. (2002). “An adaptive estimation of dimension reduction space.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(3): 363–410.
  • Xue, L. (2013). “Estimation and empirical likelihood for single-index models with missing data in the covariates.” Computational Statistics & Data Analysis, 68: 82–97.

Supplemental materials