Asymptotic Distribution of the Likelihood Function in the Independent not Identically Distributed Case

Abstract

Let $\Theta$ be an open subset of $R^k$ and for each $\theta \in \Theta$, let $X_1, \cdots, X_n$ be independent rv's defined on the probability space $(\mathscr{X}, \mathscr{A}, P_\theta)$, and let $p_{j, \theta}$ be the distribution of the rv $X_j$. Let $f_j(\bullet; \theta)$ be a specified version of the Radon-Nikodym derivative of $p_{j, \theta}$ with respect to a $\sigma$-finite measure $\mu$ and set $f_j(\theta) = f_j(X_j; \theta)$. Furthermore, for $\theta, \theta^\ast \in \Theta$, set $\phi_j(\theta, \theta^\ast) = \lbrack f_j(\theta^\ast)/f_j(\theta)\rbrack^{\frac{1}{2}}$ and suppose that $\phi_j(\theta, \theta^\ast)$ is differentiable in quadratic mean (qm) with respect to $\theta^\ast$ at $(\theta, \theta)$, when the probability measure $P_\theta$ is employed, with qm derivative $\dot{\phi}_j(\theta)$. Set $\Delta_n(\theta) = 2n^{-\frac{1}{2}} \sum^n_{j=1} \dot{\phi}_j(\theta), \Gamma_j(\theta) = 4\mathscr{E}_\theta\lbrack\dot{\phi}_j(\theta)\dot{\phi}_j'(\theta)\rbrack, \bar{\Gamma}_n(\theta) = n^{-1} \sum^n_{j=1} \Gamma_j(\theta)$, and suppose that $\bar{\Gamma}_n(\theta) \rightarrow \bar{\Gamma}(\theta)$ and $\bar{\Gamma}(\theta)$ is positive definite on $\Theta$. Finally, for $h_n \rightarrow h \in R^k$, set $\theta_n = \theta + h_nn^{-\frac{1}{2}}$ and $\Lambda_n(\theta) = \log\lbrack dP_{n,\theta_n}/dP_{n,\theta}\rbrack$, where $P_{n,\theta}$ stands for the restriction of $P_\theta$ to $\mathscr{A}_n = \sigma(X_1, \cdots, X_n)$. Then, under suitable--and not too hard to verify--conditions, we obtain, the following results. The limits are taken as $n \rightarrow \infty$. THEOREM 1. $\Lambda_n(\theta) - h'\Delta_n(\theta) \rightarrow -\frac{1}{2} h'\bar{\Gamma}(\theta)h$ in $P_\theta$-probability, $\theta \in \Theta$. THEOREM 2. $\mathscr{L}\lbrack\Delta_n(\theta) \mid P_\theta\rbrack \Rightarrow N(0, \bar{\Gamma}(\theta)), \theta \in \Theta$. THEOREM 3. $\mathscr{L}\lbrack\Lambda_n(\theta) \mid P_\theta\rbrack \Rightarrow N(-\frac{1}{2} h'\bar{\Gamma}(\theta)h, h'\bar{\Gamma}(\theta)h), \theta \in \Theta$. THEOREM 4. $\Lambda_n(\theta) - h'\Delta_n(\theta) \rightarrow -\frac{1}{2}h'\bar{\Gamma}(\theta)h$ in $P_{\theta_n}$-probability, $\theta \in \Theta$. THEOREM 5. $\mathscr{L}\lbrack\Lambda_n(\theta) \mid P_{\theta_n}\rbrack \Rightarrow N(\frac{1}{2}h'\bar{\Gamma}(\theta)h, h'\bar{\Gamma}(theta)h), \theta \in \Theta$. THEOREM 6. $\mathscr{L}\lbrack\Delta_n(\theta) \mid P_{\theta_n}\rbrack \Rightarrow N(\bar{\Gamma}(\theta)h, \bar{\Gamma}(\theta)), \theta \in \Theta$.