## The Annals of Statistics

- Ann. Statist.
- Volume 13, Number 1 (1985), 236-245.

### A Newton-Raphson Version of the Multivariate Robbins-Monro Procedure

#### Abstract

Suppose that $f$ is a function from $\mathbb{R}^k$ to $\mathbb{R}^k$ and for some $\theta, f(\theta) = 0$. Initially $f$ is unknown, but for any $x$ in $\mathbb{R}^k$ we can observe a random vector $Y(x)$ with expectation $f(x)$. The unknown $\theta$ can be estimated recursively by Blum's (1954) multivariate version of the Robbins-Monro procedure. Blum's procedure requires the rather restrictive assumption that infimum of the inner product $(x - \theta)^tf(x)$ over any compact set not containing $\theta$ be positive. Thus at each $x, f(x)$ gives information about the direction towards $\theta$. Blum's recursion is $X_{n+1} = X_n - a_n Y_n$ where the conditional expectation of $Y_n$ given $X_1, \cdots, X_n$ is $f(X_n)$ and $a_n > 0$. Unlike Blum's method, the procedure introduced in this paper does not necessarily attempt to move in a direction that decreases $\|X_n - \theta\|$, at least not during the initial stage of the procedure. Rather, except for random fluctuations it moves in a direction which decreases $\|f\|^2$, and it may follow a circuitous route to $\theta$. Consequently, it does not require that $(x - \theta)^tf(x)$ have a constant signum. This new procedure is somewhat similar to the multivariate Kiefer-Wolfowitz procedure applied to $\|f\|^2$, but unlike the latter it converges to $\theta$ at rate $n^{-1/2}$. Deterministic root finding methods are briefly discussed. The method of this paper is a stochastic analog of the Newton-Raphson and Gauss-Newton techniques.

#### Article information

**Source**

Ann. Statist., Volume 13, Number 1 (1985), 236-245.

**Dates**

First available in Project Euclid: 12 April 2007

**Permanent link to this document**

https://projecteuclid.org/euclid.aos/1176346589

**Digital Object Identifier**

doi:10.1214/aos/1176346589

**Mathematical Reviews number (MathSciNet)**

MR773164

**Zentralblatt MATH identifier**

0571.62072

**JSTOR**

links.jstor.org

**Subjects**

Primary: 62L20: Stochastic approximation

**Keywords**

Root finding stochastic approximation asymptotic normality asymptotic efficiency Gauss-Newton algorithm

#### Citation

Ruppert, David. A Newton-Raphson Version of the Multivariate Robbins-Monro Procedure. Ann. Statist. 13 (1985), no. 1, 236--245. doi:10.1214/aos/1176346589. https://projecteuclid.org/euclid.aos/1176346589