Probability forecasts for a sequence of uncertain events may be compared with the outcomes of those events by means of a natural criterion of empirical validity, calibration. It is shown that any two sequences of forecasts which both meet this criterion must be in asymptotic agreement. These agreed values can then be considered as correct objective probability forecasts for the particular sequence of outcome results obtained. However, the objective forecasts vary with the extent of the information taken into account when they are formulated. We thus obtain a general theory of empirical probability, relative to an information base. This theory does not require that such probabilities be interpreted in terms of repeated trials of the same event. Some implications of this theory are discussed.

The likelihood is defined for a state space model with incompletely specified initial conditions by transforming the data to eliminate the dependence on the unspecified conditions. This approach is extended to obtain estimates of the state vectors and predictors and interpolators for missing observations. It is then shown that this method is equivalent to placing a diffuse prior distribution on the unspecified part of the initial state vector, and modified versions of the Kalman filter and smoothing algorithms are derived to give exact numerical procedures for diffuse initial conditions. The results are extended to continuous time models, including smoothing splines and continuous time autoregressive processes.

Self-consistent estimators for estimating distribution functions from incomplete data are presented. In many cases these estimators are also generalized maximum likelihood estimators. In this paper we discuss the theoretical properties of such estimators: existence, uniform consistency, law of the iterated logarithm, and weak convergence. Applications to the product limit estimator for right-censored data and to the estimator proposed by Turnbull (1974, 1976) for doubly (right- and left-) censored data are also given.

In a logistic regression model when covariates are subject to measurement error the naive estimator, obtained by regressing on the observed covariates, is asymptotically biased. We introduce a bias-adjusted estimator and two estimators appropriate for normally distributed measurement errors -a functional maximum likelihood estimator and an estimator which exploits the consequences of sufficiency. The four proposals are studied asymptotically under conditions which are appropriate when the measurement error is small. A small Monte Carlo study illustrates the superiority of the measurement-error estimators in certain situations.

This paper concerns the method of generalized cross validation (GCV), a promising way of choosing between linear estimates. Based on Stein estimates and the associated unbiased risk estimates (Stein, 1981), a new approach to GCV is developed. Many consistency results are obtained for the cross-validated (Steinized) estimates in the contexts of nearest-neighbor nonparametric regression, model selection, ridge regression, and smoothing splines. Moreover, the associated Stein's unbiased risk estimate is shown to be uniformly consistent in assessing the true loss (not the risk). Consistency properties are examined as well when the sampling error is unknown. Finally, we propose a variant of GCV to handle the case that the dimension of the raw data is known to be greater than that of their expected values.

The partially improper prior behind the smoothing spline model is used to obtain a generalization of the maximum likelihood (GML) estimate for the smoothing parameter. Then this estimate is compared with the generalized cross validation (GCV) estimate both analytically and by Monte Carlo methods. The comparison is based on a predictive mean square error criteria. It is shown that if the true, unknown function being estimated is smooth in a sense to be defined then the GML estimate undersmooths relative to the GCV estimate and the predictive mean square error using the GML estimate goes to zero at a slower rate than the mean square error using the GCV estimate. If the true function is "rough" then the GCV and GML estimates have asymptotically similar behavior. A Monte Carlo experiment was designed to see if the asymptotic results in the smooth case were evident in small sample sizes. Mixed results were obtained for $n = 32$, GCV was somewhat better than GML for $n = 64$, and GCV was decidedly superior for $n = 128$. In the $n = 32$ case GCV was better for smaller $\sigma^2$ and the comparison close for larger $\sigma^2$. The theoretical results are shown to extend to the generalized spline smoothing model, which includes the estimate of functions given noisy values of various integrals of them.

In a general linear model, $Y = X\beta + R$ with $Y$ and $R n$-dimensional, $X$ a $n \times p$ matrix, and $\beta p$-dimensional, let $\hat\beta$ be an $M$ estimator of $\beta$ satisfying $0 = \sum x_i\psi(y_i - x'_i\beta)$. Let $p \rightarrow \infty$ such that $(p \log n)^{3/2} /n \rightarrow 0$. Then $\max_i|x'_i(\hat{\beta} - \beta)| \rightarrow _P 0$, and it is possible to find a uniform normal approximation for the distribution of $\hat{\beta}$ under which arbitrary linear combinations $a'_n (\hat{\beta} - \beta)$ are asymptotically normal (when appropriately normalized) and $(\hat{\beta} - \beta)'(X'X)(\hat{\beta} - \beta)$ is approximately $\chi^2_p$.

This paper proposes an estimator for the mean of a positive random variable given a sample from that random variable. The positive mean estimation problem is common, but there are few results for this problem in the literature. This paper shows the proposed estimator to be less sensitive to outlying values than the sample mean and determines its asymptotic mean squared error.

Let $X_i, i = 1, \ldots, n$ be i.i.d. $\sim F_\theta$, where $F_\theta(x) = F(x - \theta)$ for some $F$ that has median equal to 0. $F$ is assumed unknown or only partially known, and the problem is to estimate $\theta$. Priors are put on the pair $(F, \theta)$. The priors on $F$ concentrate all their mass on c.d.f.s with median equal to 0. These priors include "Dirichlet-type" priors. The marginal posterior distribution of $\theta$ given $X_1, \ldots, X_n$ is computed. The mean of the posterior is taken as the estimate of $\theta$.

For data $\theta + \varepsilon_i, i = 1, \ldots, n$ where $\varepsilon_i$ are i.i.d. $\sim F$ with the median of $F$ equal to $0$ but $F$ otherwise unknown, it is desired to estimate $\theta$. In Doss (1985) priors are put on the pair $(F, \theta)$, the marginal posterior distribution of $\theta$ is computed, and the mean of the posterior is taken as the estimate of $\theta$. In the present paper a frequentist point of view is adopted. The consistency properties of the Bayes estimates computed in Doss (1985) are investigated when the prior on $F$ is of the "Dirichlet-type." Any $F$ whose median is 0 is in the support of these priors. It is shown that if the $\varepsilon_i$ are i.i.d. from a discrete distribution, then the Bayes estimates are consistent. However, if the distribution of the $\varepsilon_is$ is continuous, the Bayes estimates can be inconsistent.

Kernel estimators of an unknown multivariate regression function are investigated. A bandwidth-selection rule is considered, which can be formulated in terms of cross validation. Under mild assumptions on the kernel and the unknown regression function, it is seen that this rule is asymptotically optimal.

The well known pure error-lack of fit test, which can be used to assess the adequacy of a linear regression model, is generalized to accommodate the case of nonreplication. The asymptotic null distribution of the proposed test statistic is derived. Also, the proposed test statistic is shown to be asymptotically comparable under general alternatives to the test statistic obtained in the case of replication. Consistency properties associated with pseudo lack of fit and pure error mean squares are given which parallel those obtained in the case of replication. In addition, the test statistic is invariant with respect to location and scale changes made to the regression variables.

For the linear regression model, $y_i = \mathbf{x}_i\mathbf{\beta} + \varepsilon_i$ with fixed $\mathbf{x}_i s$, the asymptotic normality of $(\hat{\mathbf{\beta}}, \hat{\sigma})$ which minimizes the Huber-Dutter loss function, $\sum\sigma\rho\{(y_i - \mathbf{x}_i\mathbf{\beta})/\sigma\} + A_n\sigma$, is established under rather general conditions.

Strong consistency of least-squares estimates in stochastic regression models is established under the assumption that the underlying model can be reparametrized so that the new design vectors are weakly correlated. An application to fixed-width interval estimation in stochastic approximation schemes is also discussed.

In estimating a multivariate normal mean under quadratic loss, this paper looks into the existence of two-stage sequential estimators that are better both in risk (mean square error) and sample size than the usual estimator of a given fixed sample size. In other words, given any sample size $n$, we are looking for two-stage sequential estimators truncated at $n$, with a positive probability of stopping earlier and risk lower than that of the sample mean based on $n$ observations. Sequential versions of James-Stein estimators are used to produce two-stage sequential estimators better in risk and sample size than the usual estimator--the sample mean. A lower bound on the largest possible probability of stopping earlier without losing in the risk is also obtained.

Sequential selections are to be made from two stochastic processes, or "arms." At each stage the arm selected for observation depends on past observations. The objective is to maximize the expected sum of the first $n$ observations. For arm 1 the observations are identically distributed with probability measure $P$, and for arm 2 the observations have probability measure $Q; P$ is a Dirichlet process and $Q$ is known. An equivalent problem is deciding sequentially when to stop sampling from an unknown population. Optimal strategies are shown to continue sampling if the current observation is sufficiently large. A simple form of such a rule is expressed in terms of a degenerate Dirichlet process which is related to $P$.

The admissibility of inadmissibility of procedures for combining several one-sided tests of significance into one overall test when the individual tests are based on independent normal or noncentral $t$ variables is considered. Minimal complete classes are found, from which the following results (with some exceptions) are obtained. The likelihood ratio tests and Tippett's procedure are admissible in both cases, the inverse logistic and sum of significance levels procedures are inadmissible in both cases, and Fisher's and the inverse normal procedure are admissible in the normal case but inadmissible in the $t$ case.

In this paper moderate and large deviation theorems are presented for the likelihood ratio statistic and Pearson's chi squared statistic in multinomial distributions. Let $k$ be the number of parameters and $n$ the number of observations. Moderate and large deviation theorems are available in the literature only if $k$ is kept fixed when $n \rightarrow \infty$. Although here attention is focussed on $k = k(n) \rightarrow \infty$ as $n \rightarrow \infty$, explicit inequalities are obtained for both $k$ and $n$ fixed. These inequalities imply results for the whole scope of moderate and large deviations both for fixed $k$ and for $k(n) \rightarrow \infty$ as $n \rightarrow \infty$. It turns out that the $\chi^2$ approximation continues to hold in some sense, even if $k \rightarrow \infty$. The results are applied in studying the influence of the choice of the number of classes on the power in goodness-of-fit tests, including a comparison of Pearson's chi squared test and the likelihood ratio test. Also the question of combining cells in a contingency table is discussed.

Stein's general technique for improving upon the best invariant unbiased and minimax estimators of the normal covariance matrix is described. The technique is to obtain solutions to a certain differential inequality involving the eigenvalues of the sample covariance matrix. Several improved estimators are obtained by solving the differential inequality. These estimators shrink or expand the sample eigenvalues depending on their magnitude. A scale invariant, adaptive minimax estimator is also obtained.

For $x \in \mathbb{R}$ let $N_\alpha(x) := m\alpha, \operatorname{iff} x \in (\alpha m - \alpha/2, \alpha m + \alpha/2\rbrack$. For a sample $X_1,\ldots, X_n$ we mainly study the asymptotic properties of the estimators $\bar{N}_\alpha := 1/n\sum^n_{i = 1} N_\alpha(X_i)$ and $S^2_\alpha := 1/(n - 1)\sum^n_{i = 1}(N_\alpha(X_i) - \overline{N}_\alpha)^2$ for $\alpha = \alpha_n \rightarrow 0,$ as $n \rightarrow \infty.$ For example, if $E(X^2) < \infty, E(e^{itX}) = o(|t|^{-k}),(|r| \rightarrow\infty)$ for some $k \in \mathbb{N}$ and $\alpha_n = O(n^{-1/(2k + 2)})$ or $X \sim N(\theta, \sigma^2)$ and $\alpha_n \leq 2\pi\sigma(\log n)^{-1/2,}$ we prove that $\sqrt{n}(\overline{N}_{\alpha n} - EX)$ is asymptotically normal. Problems of truncation as well as general maximum likelihood estimation from discrete scale measurements are also considered.

The estimation problem of a function of normal parameters $\xi/\sigma^{2p}$ is considered. We prove that a "natural" estimator of this ratio is admissible for quadratic loss if and only if $p$ is nonnegative.

Boscovich's (1757) proposal to estimate the parameters of a linear model by minimizing the sum of absolute deviations subject to the constraint that the mean residual be zero is considered. The asymptotic theory of the estimator confirms a remark of Edgeworth who called it a "remarkable hybrid" between $\mathscr{l}_1$ and $\mathscr{l}_2$ methods.