Finite-state Markov chains with either a discrete or continuous time parameter, Markov renewal processes and Markov-additive processes are considered. We prove that their likelihood functions, in the nonsequential as well as in various sequential cases, belong to special $(n + k, n)$-curved exponential families in general, for which limit results are easily established. Subsequently, asymptotic normality of the corresponding nonsequential and sequential maximum likelihood estimators is established. Also in the case of Markov renewal and Markov-additive processes, stopping times are determined which reduce the corresponding curved exponential families in general to noncurved ones. The latter, together with results of Stefanov, are combined with results of Serfozo to imply explicit solutions in functional limit theorems for the considered processes. In particular, we derive explicit solutions for the important variance parameter in the functional central limit theorems and functional laws of iterated logarithm for those processes. Indeed, our explicit solutions cover more general cases than the known ones, even in the case of finite-state Markov chains. Moreover, we supply explicit solutions, not previously available, in functional limit theorems for Markov renewal processes and Markov-additive processes.

We use the method of maximum likelihood and regression splines to derive estimates of the parametric and nonparametric components of semiparametric generalized linear models. The resulting estimators of both components are shown to be consistent. Also, the asymptotic theory for the estimator of the parametric component is derived, indicating that the parametric component can be estimated efficiently without under-smoothing the nonparametric component.

Prediction of categorical responses in panel studies is considered. Prediction functions based on general conditional $\log$-linear models are investigated for statistical properties both from a population perspective and a sampling perspective. Problems such as existence and uniqueness of optimal prediction functions are addressed, and basic properties of measures of prediction quality are examined. Estimation, consistency and asymptotic normality are studied for the proposed parameter estimates and measures of prediction quality.

A geometric combination estimator is proposed for $d$-dimensional ordinal contingency tables. The proposed estimator is nonnegative. It is shown that, assuming sufficient smoothness and boundary conditions for the underlying probabilities, the rate of convergence of mean summed squared error (MSSE) of this estimator is $O(K^{-1}N^{-8/(d+8)})$ for $d$-dimensional tables $(d \leq 4)$ with $K$ cells and sample size $N$. This rate is optimal under the smoothness assumptions, and is faster than that attained by nonnegative kernel estimates. Boundary kernels for multidimensional tables are also developed for the proposed estimator to relax restrictive boundary conditions, resulting in summed squared error (SSE) being of order $O_p(K^{-1}N^{-8/(d+8)})$ for all $d \geq 1$. The behavior of the new estimator is investigated through simulations and applications to real data. It is shown that even for relatively small tables, these estimators are superior to nonnegative kernel estimators, in sharp contrast to the relatively unimpressive performance of such estimators for continuous data.

The iterative proportional fitting procedure (IPFP) was introduced in 1940 by Deming and Stephan to estimate cell probabilities in contingency tables subject to certain marginal constraints. Its convergence and statistical properties have been investigated since then by several authors and by several different methods. A natural extension of the IPFP to the case of bivariate densities has been introduced by Ireland and Kullback. It has been conjectured that also in the general case the IPFP converges to the minimum discrimination projection on the class of distributions with given marginals. We verify this conjecture under some regularity conditions.

A smoothing spline is a nonparametric curve estimate that is defined as the solution to a minimization problem. One problem with this representation is that it obscures the fact that a spline, like most other nonparametric estimates, is a local, weighted average of the observed data. This property has been used extensively to study the limiting properties of kernel estimates and it is advantageous to apply similar techniques to spline estimates. Although equivalent kernels have been identified for a smoothing spline, these functions are either not accurate enough for asymptotic approximations or are restricted to equally spaced points. This paper extends this previous work to understand a spline estimate's local properties. It is shown that the absolute value of the spline weight function decreases exponentially away from its center. This result is not asymptotic. The only requirement is that the empirical distribution of the observation points be sufficiently close to a continuous distribution with a strictly positive density function. These bounds are used to derive the asymptotic form for the bias and variance of a first order smoothing spline estimate. The arguments leading to this result can be easily extended to higher order splines.

An asymptotic expansion is provided for the transformation kernel density estimator introduced by Ruppert and Cline. Let $h_k$ be the band-width used in the $k$th iteration, $k = 1,2,\ldots, t$. If all bandwidths are of the same order, the leading bias term of the $l$th derivative of the $t$th iterate of the density estimator has the form $\bar{b}^{(l)}_t(x) \pi^t_{k=1} h^2_k$, where the bias factor $\bar{b}_t(x)$ depends on the second moment of the kernel $K$, as well as on all derivatives of the density $f$ up to order $2t$. In particular, the leading bias term is of the same order as when using an ordinary kernel density estimator with a kernel of order $2t$. The leading stochastic term involves a kernel of order $2t$ that depends on $K, h_1$ and $h_k/f(x), k = 2,\ldots, t$.

The definition of an orthogonal array imposes an important geometric property: the projection of an $\mathrm{OA}(\lambda 2^t, 2^k, t)$, a $\lambda 2^t$-run orthogonal array with $k$ two-level factors and strength $t$, onto any $t$ factors consists of $\lambda$ copies of the complete $2^t$ factorial. In this article, projections of an $\mathrm{OA}(N, 2^k, t)$ onto $t + 1$ and $t + 2$ factors are considered. The projection onto any $t + 1$ factors must be one of three types: one or more copies of the complete $2^{t+1}$ factorial, one or more copies of a half-replicate of $2^{t+1}$ or a combination of both. It is also shown that for $k \geq t + 2$, only when $N$ is a multiple of $2^{t+1}$ can the projection onto some $t + 1$ factors be copies of a half-replicate of $2^{t+1}$. Therefore, if $N$ is not a multiple of $2^{t+1}$, then the projection of an $\mathrm{OA}(N, 2^k, t)$ with $k \geq t + 2$ onto any $t + 1$ factors must contain at least one complete $2^{t+1}$ factorial. Some properties of projections onto $t + 2$ factors are established and are applied to show that if $N$ is not a multiple of 8, then for any $\mathrm{OA}(N, 2^k, 2)$ with $k \geq 4$, the projection onto any four factors has the property that all the main effects and two-factor interactions of these four factors are estimable when the higher-order interactions are negligible.

An optimum design of experiment for a class of estimates of the first derivative at 0 (used in stochastic approximation and density estimation) is shown to be equivalent to the problem of finding a point of minimum of the function $\Gamma$ defined by $\Gamma (x) = \det\lbrack 1, x^3,\ldots, x^{2m-1} \rbrack/\det\lbrack x, x^3,\ldots, x^{2m-1} \rbrack$ on the set of all $m$-dimensional vectors with components satisfying $0 < x_1 < -x_2 < \cdots < (-1)^{m-1} x_m$ and $\Pi|x_i| = 1$. (In the determinants, 1 is the column vector with all components 1, and $x^i$ has components of $x$ raised to the $i$-th power.) The minimum of $\Gamma$ is shown to be $m$, and the point at which the minimum is attained is characterized by Chebyshev polynomials of the second kind.

If an experimenter wants to determine the degree of a polynomial regression on the basis of a sample of observations, Anderson showed that the following method is optimal. Starting with the highest (specified) degree the procedure is to test in sequence whether the coefficients are 0. In this paper optimal designs for Anderson's procedure are determined explicitly. The optimal design maximizes the minimum power of a given set of alternatives.

Positron emission tomography (PET) is a radiologic tool offering a unique capability for measuring tissue metabolism in vivo. A number of biological and physical factors limit the resolution of PET so often the statistical aspects of image reconstruction have an appreciable effect on the quality of information obtained from a study. To a first approximation, reconstruction involves the solution of a linear inverse problem with a line-integral Radon-type transform. Standard filtered back-projection reconstruction is based on the method of least squares. Although computationally efficient, the method does not enforce positivity constraints leading to undesirable negative artifacts in the results. Maximum likelihood based approaches to reconstruction do not suffer from this problem, but their computational complexity has limited the ability to determine quantitatively the improvements in image quality. In this paper, asymptotic approximations and numerical simulations are used to examine the least squares and maximum likelihood methods in some detail. The studies are carried out for idealized representations of conventional and time-of-flight tomographs. The asymptotic analysis indicates that for a range of Sobolev norms the rates of estimation of least squares and maximum likelihood reconstructions are of the same order. This is borne out by numerical studies. However, in these studies maximum likelihood is found to be more efficient than least squares: on a conventional distance-angle tomograph, the root mean square error is on the order of 10-20% smaller for maximum likelihood reconstructions. The corresponding reduction in the root mean square error on a time-of-flight tomograph is in the 5% range. Similar results are found using more complex region of interest based measures of reconstruction accuracy. In addition it is found that much of the effect of maximum likelihood can apparently be realized by postprocessing least squares solutions in an ad hoc manner to reduce negativity artifacts. Since the postprocessing scheme is computationally fast, this suggests that in PET it may be possible to realize the benefits of maximum likelihood without the substantial computational overhead of the EM algorithm.

An iterated bootstrap confidence interval requires an additive correction to be made to the nominal coverage level of an uncorrected interval. Such correction is usually performed using a computationally intensive Monte Carlo simulation involving two nested levels of bootstrap sampling. Asymptotic expansions of the required correction and the iterated interval endpoints are used to provide two new computationally efficient methods for constructing an approximation to the iterated bootstrap confidence interval. The first asymptotic interval replaces the need for a second level of bootstrap sampling with a series of preliminary analytic calculations, which are readily automated, and from which an approximation to the coverage correction is easily obtained. The second interval directly approximates the endpoints of the iterated interval and yields, for the first time, the possibility of constructing an approximation to an iterated bootstrap confidence interval which does not require any resampling. The theoretical properties of the two intervals are considered. The computation required for their construction is detailed and has been coded in a fully automatic user-friendly Fortran program which may be obtained by anonymous ftp. A simulation study which illustrates their effectiveness on three examples is presented.

This paper studies the performance of the moving block bootstrap procedure for normalized sums of dependent random variables. Suppose that $X_1, X_2,\ldots$ are stationary $\rho$-mixing random variables with $\sum \rho (2^i) < \infty$. Let $T_n = (X_1 + \cdots + X_n - b_n)/a_n$, for some suitable constants $a_n$ and $b_n$, and let $T^\ast_{m,n}$ denote the moving block bootstrap version of $T_n$ based on a bootstrap sample of size $m$. Under certain regularity conditions, it is shown that, for $X_n$'s lying in the domain of partial attraction of certain infinitely divisible distributions, the conditional distribution $\hat{H}_{m,n}$ of $T^\ast_{m,n}$ provides a valid approximation to the distribution of $T_n$ along every weakly convergent subsequence, provided $m = o(n)$ as $n \rightarrow \infty$. On the other hand, for the usual choice of the resample size $m = n, \hat{H}_{n,n}(x)$ is shown to converge to a nondegenerate random limit as given by Athreya (1987) when $T_n$ has a stable limit of order $\alpha, 1 < \alpha < 2$.

We sequentially observe independent observations $X_1, X_2,\ldots$ such that initially they have distribution $G_0$; at some unknown time $\nu$ they become stochastically larger, having distribution $G_1$. Neither $G_0$ nor $G_1$ is fully specified. We wish to detect that a change has taken place as soon as possible after its occurrence, subject to a constraint on the rate of false alarms. We derive a family of nonparametric sequential procedures based on ranks, with noncontiguous alternatives in mind. Large-sample approximations to the operating characteristics are obtained analytically. The proposed procedures all possess robustness of validity, because they are based on ranks. Near-optimal sensitivity can be obtained for specific alternatives by choosing an appropriate procedure. For example, when observations are normally distributed with one standard deviation shift in mean postchange, an appropriate nonparametric surveillance scheme yields 97% asymptotic relative efficiency, as compared to the optimal procedure when all distributional parameters are known. Our procedures are computationally feasible. Monte Carlo experiments confirm the applicability of the asymptotic calculations, including high levels of efficiency, for sample sizes met in practice.

The problem is to estimate sequentially a nonparametric function known to belong to an $\alpha$-th-order Sobolev subspace $(\alpha > \frac{1}{2})$ with a minimax mean stopping time subject to an assigned maximum mean integrated squared error. For the case of a given $\alpha$ there exists a sharp estimator which has a minimal constant and a rate of minimax mean stopping time increasing as the assigned risk decreases. The situation changes drastically if $\alpha$ is unknown: a necessary and sufficient condition for sharp estimation is that $\gamma < \alpha \leq 2\gamma$ for some given $\gamma \geq \frac{1}{2}$.

We consider the composite hypothesis testing problem of classifying an unknown probability distribution based on a sequence of random samples drawn according to this distribution. Specifically, if $A$ is a subset of the space of all probability measures $\mathscr{M}_1(\Sigma)$ over some compact Polish space $\Sigma$, we want to decide whether or not the unknown distribution belongs to $A$ or its complement. We propose an algorithm which leads a.s. to a correct decision for any $A$ satisfying certain structural assumptions. A refined decision procedure is also presented which, given a countable collection $A_i \subset \mathscr{M}_1(\Sigma), i = 1,2,\ldots$, each satisfying the structural assumption, will eventually determine a.s. the membership of the distribution in any finite number of the $A_i$. Applications to density estimation are discussed.

Let $\mathbf{X}$ have a star unimodal distribution $P_0$ on $\mathbb{R}^p$. We describe a general method for constructing a star-shaped set $S$ with the property $P_0(\mathbf{X} \in S) \geq 1 - \alpha$, where $0 < \alpha < 1$ is fixed. This is done by using the Camp-Meidell inequality on the Minkowski functional of an arbitrary star-shaped set $S$ and then minimizing Lebesgue measure in order to obtain size-efficient sets. Conditions are obtained under which this method reproduces a level (high density) set. The general theory is then applied to two specific examples: set estimation of a multivariate normal mean using a multivariate $t$ prior and classical invariant estimation of a location vector $\mathbf{\theta}$ for a mixture model. In the Bayesian example, a number of shape properties of the posterior distribution are established in the process. These results are of independent interest as well. A computer code is available from the authors for automated application. The methods presented here permit construction of explicit confidence sets under very limited assumptions when the underlying distributions are calculationally too complex to obtain level sets.

Let $X_i = \theta + \sigma Z_i$, where $Z_i$ are i.i.d. from a distribution $F$, and $-\infty < \theta < \infty$ and $\sigma > 0$ are unknown parameters. If $F$ is $N(0, 1)$, a standard confidence interval for the unknown mean $\theta$ is the $t$-interval $\bar{X} \pm t_{\alpha/2} s/\sqrt n$. The question of conservatism of this interval under nonnormality is considered by evaluating the infimum of its coverage probability when $F$ belongs to a suitably chosen class of distributions $\mathscr{F}$. Some rather surprising phenomena show up. For $\mathscr{F} = \{\text{all symmetric unimodal distributions}\}$ it is found that, for high nominal coverage intervals, the minimum coverage is attained at $U\lbrack -1, 1 \rbrack$ distribution, and the $t$-interval is quite conservative. However, for intervals with low or moderate nominal coverages $(t_{\alpha/2} < 1)$, it is proved that the infimum coverage is zero, thus indicating drastic sensitivity to nonnormality. This phenomenon carries over to more general families of distributions. Our results also relate to robustness of the $P$-value corresponding to the $t$-statistic when the underlying distribution is nonnormal.