Strong consistency and asymptotic normality of the Gaussian pseudo-maximum likelihood estimate of the parameters in a wide class of ARCH(∞) processes are established. The conditions are shown to hold in case of exponential and hyperbolic decay in the ARCH weights, though in the latter case a faster decay rate is required for the central limit theorem than for the law of large numbers. Particular parameterizations are discussed.

In this paper the class of ARCH(∞) models is generalized to the nonstationary class of ARCH(∞) models with time-varying coefficients. For fixed time points, a stationary approximation is given leading to the notation “locally stationary ARCH(∞) process.” The asymptotic properties of weighted quasi-likelihood estimators of time-varying ARCH(p) processes (p<∞) are studied, including asymptotic normality. In particular, the extra bias due to nonstationarity of the process is investigated. Moreover, a Taylor expansion of the nonstationary ARCH process in terms of stationary processes is given and it is proved that the time-varying ARCH process can be written as a time-varying Volterra series.

Processes with almost periodic covariance functions have spectral mass on lines parallel to the diagonal in the two-dimensional spectral plane. Methods have been given for estimation of spectral mass on the lines of spectral concentration if the locations of the lines are known. Here methods for estimating the intercepts of the lines of spectral concentration in the Gaussian case are given under appropriate conditions. The methods determine rates of convergence sufficiently fast as the sample size n→∞ so that the spectral estimation on the estimated lines can then proceed effectively. This task involves bounding the maximum of an interesting class of non-Gaussian possibly nonstationary processes.

We develop a testing procedure for distinguishing between a long-range dependent time series and a weakly dependent time series with change-points in the mean. In the simplest case, under the null hypothesis the time series is weakly dependent with one change in mean at an unknown point, and under the alternative it is long-range dependent. We compute the CUSUM statistic T_n, which allows us to construct an estimator k̂ of a change-point. We then compute the statistic T_n,1 based on the observations up to time k̂ and the statistic T_n,2 based on the observations after time k̂. The statistic M_n=max [T_n,1,T_n,2] converges to a well-known distribution under the null, but diverges to infinity if the observations exhibit long-range dependence. The theory is illustrated by examples and an application to the returns of the Dow Jones index.

This paper deals with order identification for nested models in the i.i.d. framework. We study the asymptotic efficiency of two generalized likelihood ratio tests of the order. They are based on two estimators which are proved to be strongly consistent. A version of Stein’s lemma yields an optimal underestimation error exponent. The lemma also implies that the overestimation error exponent is necessarily trivial. Our tests admit nontrivial underestimation error exponents. The optimal underestimation error exponent is achieved in some situations. The overestimation error can decay exponentially with respect to a positive power of the number of observations.

These results are proved under mild assumptions by relating the underestimation (resp. overestimation) error to large (resp. moderate) deviations of the log-likelihood process. In particular, it is not necessary that the classical Cramér condition be satisfied; namely, the log-densities are not required to admit every exponential moment. Three benchmark examples with specific difficulties (location mixture of normal distributions, abrupt changes and various regressions) are detailed so as to illustrate the generality of our results.

Suppose that univariate data are drawn from a mixture of two distributions that are equal up to a shift parameter. Such a model is known to be nonidentifiable from a nonparametric viewpoint. However, if we assume that the unknown mixed distribution is symmetric, we obtain the identifiability of this model, which is then defined by four unknown parameters: the mixing proportion, two location parameters and the cumulative distribution function of the symmetric mixed distribution. We propose estimators for these four parameters when no training data is available. Our estimators are shown to be strongly consistent under mild regularity assumptions and their convergence rates are studied. Their finite-sample properties are illustrated by a Monte Carlo study and our method is applied to real data.

When do nonparametric Bayesian procedures “overfit”? To shed light on this question, we consider a binary regression problem in detail and establish frequentist consistency for a certain class of Bayes procedures based on hierarchical priors, called uniform mixture priors. These are defined as follows: let ν be any probability distribution on the nonnegative integers. To sample a function f from the prior π^ν, first sample m from ν and then sample f uniformly from the set of step functions from [0,1] into [0,1] that have exactly m jumps (i.e., sample all m jump locations and m+1 function values independently and uniformly). The main result states that if a data-stream is generated according to any fixed, measurable binary-regression function f₀≢1/2, then frequentist consistency obtains: that is, for any ν with infinite support, the posterior of π^ν concentrates on any L¹ neighborhood of f₀. Solution of an associated large-deviations problem is central to the consistency proof.

We introduce a natural conjugate prior for the transition matrix of a reversible Markov chain. This allows estimation and testing. The prior arises from random walk with reinforcement in the same way the Dirichlet prior arises from Pólya’s urn. We give closed form normalizing constants, a simple method of simulation from the posterior and a characterization along the lines of W. E. Johnson’s characterization of the Dirichlet prior.

Sample size criteria are often expressed in terms of the concentration of the posterior density, as controlled by some sort of error bound. Since this is done pre-experimentally, one can regard the posterior density as a function of the data. Thus, when a sample size criterion is formalized in terms of a functional of the posterior, its value is a random variable. Generally, such functionals have means under the true distribution.

We give asymptotic expressions for the expected value, under a fixed parameter, for certain types of functionals of the posterior density in a Bayesian analysis. The generality of our treatment permits us to choose functionals that encapsulate a variety of inference criteria and large ranges of error bounds. Consequently, we get simple inequalities which can be solved to give minimal sample sizes needed for various estimation goals. In several parametric examples, we verify that our asymptotic bounds give good approximations to the expected values of the functionals they approximate. Also, our numerical computations suggest our treatment gives reasonable results.

We investigate algorithms for reconstructing a convex body K in ℝⁿ from noisy measurements of its support function or its brightness function in k directions u₁,…,u_k. The key idea of these algorithms is to construct a convex polytope P_k whose support function (or brightness function) best approximates the given measurements in the directions u₁,…,u_k (in the least squares sense). The measurement errors are assumed to be stochastically independent and Gaussian.

It is shown that this procedure is (strongly) consistent, meaning that, almost surely, P_k tends to K in the Hausdorff metric as k→∞. Here some mild assumptions on the sequence (u_i) of directions are needed. Using results from the theory of empirical processes, estimates of rates of convergence are derived, which are first obtained in the L₂ metric and then transferred to the Hausdorff metric. Along the way, a new estimate is obtained for the metric entropy of the class of origin-symmetric zonoids contained in the unit ball.

Similar results are obtained for the convergence of an algorithm that reconstructs an approximating measure to the directional measure of a stationary fiber process from noisy measurements of its rose of intersections in k directions u₁,…,u_k. Here the Dudley and Prohorov metrics are used. The methods are linked to those employed for the support and brightness function algorithms via the fact that the rose of intersections is the support function of a projection body.

We consider the problem of estimating convex boundaries from blurred and noisy observations. In our model, the convolution of an intensity function f is observed with additive Gaussian white noise. The function f is assumed to have convex support G whose boundary is to be recovered. Rather than directly estimating the intensity function, we develop a procedure which is based on estimating the support function of the set G. This approach is closely related to the method of geometric hyperplane probing, a well-known technique in computer vision applications. We establish bounds that reveal how the estimation accuracy depends on the ill-posedness of the convolution operator and the behavior of the intensity function near the boundary.

Nonparametric methods have been very popular in the last couple of decades in time series and regression, but no such development has taken place for spatial models. A rather obvious reason for this is the curse of dimensionality. For spatial data on a grid evaluating the conditional mean given its closest neighbors requires a four-dimensional nonparametric regression. In this paper a semiparametric spatial regression approach is proposed to avoid this problem. An estimation procedure based on combining the so-called marginal integration technique with local linear kernel estimation is developed in the semiparametric spatial regression setting. Asymptotic distributions are established under some mild conditions. The same convergence rates as in the one-dimensional regression case are established. An application of the methodology to the classical Mercer and Hall wheat data set is given and indicates that one directional component appears to be nonlinear, which has gone unnoticed in earlier analyses.

The pattern of zero entries in the inverse covariance matrix of a multivariate normal distribution corresponds to conditional independence restrictions between variables. Covariance selection aims at estimating those structural zeros from data. We show that neighborhood selection with the Lasso is a computationally attractive alternative to standard covariance selection for sparse high-dimensional graphs. Neighborhood selection estimates the conditional independence restrictions separately for each node in the graph and is hence equivalent to variable selection for Gaussian linear models. We show that the proposed neighborhood selection scheme is consistent for sparse high-dimensional graphs. Consistency hinges on the choice of the penalty parameter. The oracle value for optimal prediction does not lead to a consistent neighborhood estimate. Controlling instead the probability of falsely joining some distinct connectivity components of the graph, consistent estimation for sparse graphs is achieved (with exponential rates), even when the number of variables grows as the number of observations raised to an arbitrary power.

We formulate necessary and sufficient conditions for an arbitrary discrete probability distribution to factor according to an undirected graphical model, or a log-linear model, or other more general exponential models. For decomposable graphical models these conditions are equivalent to a set of conditional independence statements similar to the Hammersley–Clifford theorem; however, we show that for nondecomposable graphical models they are not. We also show that nondecomposable models can have nonrational maximum likelihood estimates. These results are used to give several novel characterizations of decomposable graphical models.

The use of principal component methods to analyze functional data is appropriate in a wide range of different settings. In studies of “functional data analysis,” it has often been assumed that a sample of random functions is observed precisely, in the continuum and without noise. While this has been the traditional setting for functional data analysis, in the context of longitudinal data analysis a random function typically represents a patient, or subject, who is observed at only a small number of randomly distributed points, with nonnegligible measurement error. Nevertheless, essentially the same methods can be used in both these cases, as well as in the vast number of settings that lie between them. How is performance affected by the sampling plan? In this paper we answer that question. We show that if there is a sample of n functions, or subjects, then estimation of eigenvalues is a semiparametric problem, with root-n consistent estimators, even if only a few observations are made of each function, and if each observation is encumbered by noise. However, estimation of eigenfunctions becomes a nonparametric problem when observations are sparse. The optimal convergence rates in this case are those which pertain to more familiar function-estimation settings. We also describe the effects of sampling at regularly spaced points, as opposed to random points. In particular, it is shown that there are often advantages in sampling randomly. However, even in the case of noisy data there is a threshold sampling rate (depending on the number of functions treated) above which the rate of sampling (either randomly or regularly) has negligible impact on estimator performance, no matter whether eigenfunctions or eigenvectors are being estimated.

The difficulties of estimating and representing the distributions of functional data mean that principal component methods play a substantially greater role in functional data analysis than in more conventional finite-dimensional settings. Local maxima and minima in principal component functions are of direct importance; they indicate places in the domain of a random function where influence on the function value tends to be relatively strong but of opposite sign. We explore statistical properties of the relationship between extrema of empirical principal component functions, and their counterparts for the true principal component functions. It is shown that empirical principal component funcions have relatively little trouble capturing conventional extrema, but can experience difficulty distinguishing a “shoulder” in a curve from a small bump. For example, when the true principal component function has a shoulder, the probability that the empirical principal component function has instead a bump is approximately equal to ½. We suggest and describe the performance of bootstrap methods for assessing the strength of extrema. It is shown that the subsample bootstrap is more effective than the standard bootstrap in this regard. A “bootstrap likelihood” is proposed for measuring extremum strength. Exploratory numerical methods are suggested.

Many important problems in psychology and biomedical studies require testing for overdispersion, correlation and heterogeneity in mixed effects and latent variable models, and score tests are particularly useful for this purpose. But the existing testing procedures depend on restrictive assumptions. In this paper we propose a class of test statistics based on a general mixed effects model to test the homogeneity hypothesis that all of the variance components are zero. Under some mild conditions, not only do we derive asymptotic distributions of the test statistics, but also propose a resampling procedure for approximating their asymptotic distributions conditional on the observed data. To overcome the technical challenge, we establish an invariance principle for random quadratic forms indexed by a parameter. A simulation study is conducted to investigate the empirical performance of the test statistics. A real data set is analyzed to illustrate the application of our theoretical results.

The purpose of this note is to show that in a widely cited paper by Yakir [Ann. Statist. 25 (1997) 2117–2126], the proof that the so-called modified Shiryayev–Roberts procedure is exactly optimal is incorrect. We also clarify the issues involved by both mathematical arguments and a simulation study. The correctness of the theorem remains in doubt.

In his discussion of Davies and Gather [Ann. Statist. 33 (2005) 977–1035] Tyler pointed out that the theory developed there could not be applied to the case of directional data. He related the breakdown of directional functionals to the problem of definability. In this addendum we provide a concept of breakdown defined in terms of definability and not in terms of bias. If a group of finite order k acts on the sample space we show that the breakdown point can be bounded above by (k−1)/k. In the case of directional data there is a group of order k=2 giving an upper bound of 1/2.