Random forest estimation of conditional distribution functions and conditional quantiles

Abstract

We propose a theoretical study of two realistic estimators of conditional distribution functions using random forests. The estimation process uses the bootstrap samples generated from the original dataset when constructing the forest. Bootstrap samples are reused to define the first estimator, while the second uses the original sample, once the forest has been built. We prove that both proposed estimators of the conditional distribution functions are consistent uniformly a.s. To the best of our knowledge, it is the first proof of a.s. consistency (previous consistency results are in $L^2$ norm or in probability) and including the bootstrap part. The consistency result holds for a large class of functions, including additive models and products. The consistency of conditional quantiles estimators follows that of distribution functions estimators using standard arguments.