Open Access
June 2017 Photo-$z$ estimation: An example of nonparametric conditional density estimation under selection bias
Rafael Izbicki, Ann B. Lee, Peter E. Freeman
Ann. Appl. Stat. 11(2): 698-724 (June 2017). DOI: 10.1214/16-AOAS1013


Redshift is a key quantity for inferring cosmological model parameters. In photometric redshift estimation, cosmologists use the coarse data collected from the vast majority of galaxies to predict the redshift of individual galaxies. To properly quantify the uncertainty in the predictions, however, one needs to go beyond standard regression and instead estimate the full conditional density $f(z|\mathbf{x})$ of a galaxy’s redshift $z$ given its photometric covariates $\mathbf{x}$. The problem is further complicated by selection bias: usually only the rarest and brightest galaxies have known redshifts, and these galaxies have characteristics and measured covariates that do not necessarily match those of more numerous and dimmer galaxies of unknown redshift. Unfortunately, there is not much research on how to best estimate complex multivariate densities in such settings.

Here we describe a general framework for properly constructing and assessing nonparametric conditional density estimators under selection bias, and for combining two or more estimators for optimal performance. We propose new improved photo-$z$ estimators and illustrate our methods on data from the Sloan Data Sky Survey and an application to galaxy–galaxy lensing. Although our main application is photo-$z$ estimation, our methods are relevant to any high-dimensional regression setting with complicated asymmetric and multimodal distributions in the response variable.


Download Citation

Rafael Izbicki. Ann B. Lee. Peter E. Freeman. "Photo-$z$ estimation: An example of nonparametric conditional density estimation under selection bias." Ann. Appl. Stat. 11 (2) 698 - 724, June 2017.


Received: 1 April 2016; Revised: 1 January 2017; Published: June 2017
First available in Project Euclid: 20 July 2017

zbMATH: 06775889
MathSciNet: MR3693543
Digital Object Identifier: 10.1214/16-AOAS1013

Keywords: Density estimation , nonparametric statistics , photometric redshift , selection bias

Rights: Copyright © 2017 Institute of Mathematical Statistics

Vol.11 • No. 2 • June 2017
Back to Top