Maximum Likelihood Estimation of Parameters of Autoregressive Processes with Moving Average Residuals and Other Covariance Matrices with Linear Structure

Abstract

The autoregressive process with moving average residuals is a stationary process $\{y_t\}$ satisfying $\sum^p_{s = 0} \beta_sy_{t - s} = \sum^q_{j = 0} \alpha_j\nu_{t - j}$, where the sequence $\{\nu_t\}$ consists of independently identically distributed (unobservable) random variables. The distribution of $y_1,\cdots, y_T$ can be approximated by the distribution of the $T$-component vector $\mathbf{y}$ satisfying $\sum^p_{s = 0} \beta_s\mathbf{K}_s\mathbf{y} = \sum^q_{j = 0} \alpha_j\mathbf{J}_j\mathbf{v}$, where $\mathbf{v}$ has covariance matrix $\sigma^2\mathbf{I}, \mathbf{K}_s = \mathbf{J}_s = \mathbf{L}^s$, and $\mathbf{L}$ is the $T \times T$ matrix with 1's immediately below the main diagonal and 0's elsewhere. Maximum likelihood estimates are obtained when $\mathbf{v}$ has a normal distribution. The method of scoring is used to find estimates defined by linear equations which are consistent, asymptotically normal, and asymptotically efficient (as $T\rightarrow \infty$). Several special cases are treated. It is shown how to calculate the estimates.