The Annals of Statistics
- Ann. Statist.
- Volume 46, Number 3 (2018), 1352-1382.
Distributed testing and estimation under sparse high dimensional models
This paper studies hypothesis testing and parameter estimation in the context of the divide-and-conquer algorithm. In a unified likelihood-based framework, we propose new test statistics and point estimators obtained by aggregating various statistics from $k$ subsamples of size $n/k$, where $n$ is the sample size. In both low dimensional and sparse high dimensional settings, we address the important question of how large $k$ can be, as $n$ grows large, such that the loss of efficiency due to the divide-and-conquer algorithm is negligible. In other words, the resulting estimators have the same inferential efficiencies and estimation rates as an oracle with access to the full sample. Thorough numerical results are provided to back up the theory.
Ann. Statist., Volume 46, Number 3 (2018), 1352-1382.
Received: September 2015
Revised: December 2016
First available in Project Euclid: 3 May 2018
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Battey, Heather; Fan, Jianqing; Liu, Han; Lu, Junwei; Zhu, Ziwei. Distributed testing and estimation under sparse high dimensional models. Ann. Statist. 46 (2018), no. 3, 1352--1382. doi:10.1214/17-AOS1587. https://projecteuclid.org/euclid.aos/1525313085
- Supplement to “Distributed testing and estimation under sparse high dimensional models”. We put all technical lemmas, proofs and low dimensional results in the supplementary materials for reference.