Let be a sequence of i.i.d. observations and be a parametric model. We introduce a new online algorithm for computing a sequence , which is shown to converge almost surely to at rate , with a user specified parameter. This convergence result is obtained under standard conditions on the statistical model and, most notably, we allow the mapping to be multi-modal. However, the computational cost to process each observation grows exponentially with the dimension of θ, which makes the proposed approach applicable to low or moderate dimensional problems only. We also derive a version of the estimator , which is well suited to student-t linear regression models that are popular tools for robust linear regression. As shown by experiments on simulated and real data, the corresponding estimator of the regression coefficients is, as expected, robust to the presence of outliers and thus, as a by-product, we obtain a new adaptive and robust online estimation procedure for linear regression models.
Mathieu Gerber. Kari Heine. "Online inference with multi-modal likelihood functions." Ann. Statist. 49 (6) 3103 - 3126, December 2021. https://doi.org/10.1214/21-AOS2076