While hidden class models of various types arise in many statistical applications, it is often difficult to establish the identifiability of their parameters. Focusing on models in which there is some structure of independence of some of the observed variables conditioned on hidden ones, we demonstrate a general approach for establishing identifiability utilizing algebraic arguments. A theorem of J. Kruskal for a simple latent-class model with finite state space lies at the core of our results, though we apply it to a diverse set of models. These include mixtures of both finite and nonparametric product distributions, hidden Markov models and random graph mixture models, and lead to a number of new results and improvements to old ones.

In the parametric setting, this approach indicates that for such models, the classical definition of identifiability is typically too strong. Instead generic identifiability holds, which implies that the set of nonidentifiable parameters has measure zero, so that parameter inference is still meaningful. In particular, this sheds light on the properties of finite mixtures of Bernoulli products, which have been used for decades despite being known to have nonidentifiable parameters. In the nonparametric setting, we again obtain identifiability only when certain restrictions are placed on the distributions that are mixed, but we explicitly describe the conditions.

We assume that we have observational data generated from an unknown underlying directed acyclic graph (DAG) model. A DAG is typically not identifiable from observational data, but it is possible to consistently estimate the equivalence class of a DAG. Moreover, for any given DAG, causal effects can be estimated using intervention calculus. In this paper, we combine these two parts. For each DAG in the estimated equivalence class, we use intervention calculus to estimate the causal effects of the covariates on the response. This yields a collection of estimated causal effects for each covariate. We show that the distinct values in this set can be consistently estimated by an algorithm that uses only local information of the graph. This local approach is computationally fast and feasible in high-dimensional problems. We propose to use summary measures of the set of possible causal effects to determine variable importance. In particular, we use the minimum absolute value of this set, since that is a lower bound on the size of the causal effect. We demonstrate the merits of our methods in a simulation study and on a data set about riboflavin production.

This paper discusses asymptotically distribution free tests for the classical goodness-of-fit hypothesis of an error distribution in nonparametric regression models. These tests are based on the same martingale transform of the residual empirical process as used in the one sample location model. This transformation eliminates extra randomization due to covariates but not due the errors, which is intrinsically present in the estimators of the regression function. Thus, tests based on the transformed process have, generally, better power. The results of this paper are applicable as soon as asymptotic uniform linearity of nonparametric residual empirical process is available. In particular they are applicable under the conditions stipulated in recent papers of Akritas and Van Keilegom and Müller, Schick and Wefelmeyer.

We suggest a robust nearest-neighbor approach to classifying high-dimensional data. The method enhances sensitivity by employing a threshold and truncates to a sequence of zeros and ones in order to reduce the deleterious impact of heavy-tailed data. Empirical rules are suggested for choosing the threshold. They require the bare minimum of data; only one data vector is needed from each population. Theoretical and numerical aspects of performance are explored, paying particular attention to the impacts of correlation and heterogeneity among data components. On the theoretical side, it is shown that our truncated, thresholded, nearest-neighbor classifier enjoys the same classification boundary as more conventional, nonrobust approaches, which require finite moments in order to achieve good performance. In particular, the greater robustness of our approach does not come at the price of reduced effectiveness. Moreover, when both training sample sizes equal 1, our new method can have performance equal to that of optimal classifiers that require independent and identically distributed data with known marginal distributions; yet, our classifier does not itself need conditions of this type.

Asymptotic equivalence theory developed in the literature so far are only for bounded loss functions. This limits the potential applications of the theory because many commonly used loss functions in statistical inference are unbounded. In this paper we develop asymptotic equivalence results for robust nonparametric regression with unbounded loss functions. The results imply that all the Gaussian nonparametric regression procedures can be robustified in a unified way. A key step in our equivalence argument is to bin the data and then take the median of each bin.

The asymptotic equivalence results have significant practical implications. To illustrate the general principles of the equivalence argument we consider two important nonparametric inference problems: robust estimation of the regression function and the estimation of a quadratic functional. In both cases easily implementable procedures are constructed and are shown to enjoy simultaneously a high degree of robustness and adaptivity. Other problems such as construction of confidence sets and nonparametric hypothesis testing can be handled in a similar fashion.

We consider the problem of reliably finding filaments in point clouds. Realistic data sets often have numerous filaments of various sizes and shapes. Statistical techniques exist for finding one (or a few) filaments but these methods do not handle noisy data sets with many filaments. Other methods can be found in the astronomy literature but they do not have rigorous statistical guarantees. We propose the following method. Starting at each data point we construct the steepest ascent path along a kernel density estimator. We locate filaments by finding regions where these paths are highly concentrated. Formally, we define the density of these paths and we construct a consistent estimator of this path density.

We formulate and investigate a statistical inverse problem of a random tomographic nature, where a probability density function on ℝ³ is to be recovered from observation of finitely many of its two-dimensional projections in random and unobservable directions. Such a problem is distinct from the classic problem of tomography where both the projections and the unit vectors normal to the projection plane are observable. The problem arises in single particle electron microscopy, a powerful method that biophysicists employ to learn the structure of biological macromolecules. Strictly speaking, the problem is unidentifiable and an appropriate reformulation is suggested hinging on ideas from Kendall’s theory of shape. Within this setup, we demonstrate that a consistent solution to the problem may be derived, without attempting to estimate the unknown angles, if the density is assumed to admit a mixture representation.

Situations of a functional predictor paired with a scalar response are increasingly encountered in data analysis. Predictors are often appropriately modeled as square integrable smooth random functions. Imposing minimal assumptions on the nature of the functional relationship, we aim to estimate the directional derivatives and gradients of the response with respect to the predictor functions. In statistical applications and data analysis, functional derivatives provide a quantitative measure of the often intricate relationship between changes in predictor trajectories and those in scalar responses. This approach provides a natural extension of classical gradient fields in vector space and provides directions of steepest descent. We suggest a kernel-based method for the nonparametric estimation of functional derivatives that utilizes the decomposition of the random predictor functions into their eigenfunctions. These eigenfunctions define a canonical set of directions into which the gradient field is expanded. The proposed method is shown to lead to asymptotically consistent estimates of functional derivatives and is illustrated in an application to growth curves.

When the spatial sample size is extremely large, which occurs in many environmental and ecological studies, operations on the large covariance matrix are a numerical challenge. Covariance tapering is a technique to alleviate the numerical challenges. Under the assumption that data are collected along a line in a bounded region, we investigate how the tapering affects the asymptotic efficiency of the maximum likelihood estimator (MLE) for the microergodic parameter in the Matérn covariance function by establishing the fixed-domain asymptotic distribution of the exact MLE and that of the tapered MLE. Our results imply that, under some conditions on the taper, the tapered MLE is asymptotically as efficient as the true MLE for the microergodic parameter in the Matérn model.

This paper is concerned with density estimation of directional data on the sphere. We introduce a procedure based on thresholding on a new type of spherical wavelets called needlets. We establish a minimax result and prove its optimality. We are motivated by astrophysical applications, in particular in connection with the analysis of ultra high-energy cosmic rays.

We investigate function estimation in nonparametric regression models with random design and heteroscedastic correlated noise. Adaptive properties of warped wavelet nonlinear approximations are studied over a wide range of Besov scales, $f\in\mathcal{B}_{\pi,r}^{s}$, and for a variety of L^p error measures. We consider error distributions with Long-Range-Dependence parameter α, 0<α≤1; heteroscedasticity is modeled with a design dependent function σ. We prescribe a tuning paradigm, under which warped wavelet estimation achieves partial or full adaptivity results with the rates that are shown to be the minimax rates of convergence. For p>2, it is seen that there are three rate phases, namely the dense, sparse and long range dependence phase, depending on the relative values of s, p, π and α. Furthermore, we show that long range dependence does not come into play for shape estimation f−∫f. The theory is illustrated with some numerical examples.

In Bayesian analysis of multi-way contingency tables, the selection of a prior distribution for either the log-linear parameters or the cell probabilities parameters is a major challenge. In this paper, we define a flexible family of conjugate priors for the wide class of discrete hierarchical log-linear models, which includes the class of graphical models. These priors are defined as the Diaconis–Ylvisaker conjugate priors on the log-linear parameters subject to “baseline constraints” under multinomial sampling. We also derive the induced prior on the cell probabilities and show that the induced prior is a generalization of the hyper Dirichlet prior. We show that this prior has several desirable properties and illustrate its usefulness by identifying the most probable decomposable, graphical and hierarchical log-linear models for a six-way contingency table.

Extracting useful information from high-dimensional data is an important focus of today’s statistical research and practice. Penalized loss function minimization has been shown to be effective for this task both theoretically and empirically. With the virtues of both regularization and sparsity, the L₁-penalized squared error minimization method Lasso has been popular in regression models and beyond.

In this paper, we combine different norms including L₁ to form an intelligent penalty in order to add side information to the fitting of a regression or classification model to obtain reasonable estimates. Specifically, we introduce the Composite Absolute Penalties (CAP) family, which allows given grouping and hierarchical relationships between the predictors to be expressed. CAP penalties are built by defining groups and combining the properties of norm penalties at the across-group and within-group levels. Grouped selection occurs for nonoverlapping groups. Hierarchical variable selection is reached by defining groups with particular overlapping patterns. We propose using the BLASSO and cross-validation to compute CAP estimates in general. For a subfamily of CAP estimates involving only the L₁ and L_∞ norms, we introduce the iCAP algorithm to trace the entire regularization path for the grouped selection problem. Within this subfamily, unbiased estimates of the degrees of freedom (df) are derived so that the regularization parameter is selected without cross-validation. CAP is shown to improve on the predictive performance of the LASSO in a series of simulated experiments, including cases with p≫n and possibly mis-specified groupings. When the complexity of a model is properly calculated, iCAP is seen to be parsimonious in the experiments.

Model selection and sparse recovery are two important problems for which many regularization methods have been proposed. We study the properties of regularization methods in both problems under the unified framework of regularized least squares with concave penalties. For model selection, we establish conditions under which a regularized least squares estimator enjoys a nonasymptotic property, called the weak oracle property, where the dimensionality can grow exponentially with sample size. For sparse recovery, we present a sufficient condition that ensures the recoverability of the sparsest solution. In particular, we approach both problems by considering a family of penalties that give a smooth homotopy between L₀ and L₁ penalties. We also propose the sequentially and iteratively reweighted squares (SIRS) algorithm for sparse recovery. Numerical studies support our theoretical results and demonstrate the advantage of our new methods for model selection and sparse recovery.

We formulate nonparametric and semiparametric hypothesis testing of multivariate stationary linear time series in a unified fashion and propose new test statistics based on estimators of the spectral density matrix. The limiting distributions of these test statistics under null hypotheses are always normal distributions, and they can be implemented easily for practical use. If null hypotheses are false, as the sample size goes to infinity, they diverge to infinity and consequently are consistent tests for any alternative. The approach can be applied to various null hypotheses such as the independence between the component series, the equality of the autocovariance functions or the autocorrelation functions of the component series, the separability of the covariance matrix function and the time reversibility. Furthermore, a null hypothesis with a nonlinear constraint like the conditional independence between the two series can be tested in the same way.

This paper generalizes a part of the theory of Z-estimation which has been developed mainly in the context of modern empirical processes to the case of stochastic processes, typically, semimartingales. We present a general theorem to derive the asymptotic behavior of the solution to an estimating equation θ↝Ψ_n(θ, ĥ_n)=0 with an abstract nuisance parameter h when the compensator of Ψ_n is random. As its application, we consider the estimation problem in an ergodic diffusion process model where the drift coefficient contains an unknown, finite-dimensional parameter θ and the diffusion coefficient is indexed by a nuisance parameter h from an infinite-dimensional space. An example for the nuisance parameter space is a class of smooth functions. We establish the asymptotic normality and efficiency of a Z-estimator for the drift coefficient. As another application, we present a similar result also in an ergodic time series model.

Regular factorial designs with randomization restrictions are widely used in practice. This paper provides a unified approach to the construction of such designs using randomization defining contrast subspaces for the representation of randomization restrictions. We use finite projective geometry to determine the existence of designs with the required structure and develop a systematic approach for their construction. An attractive feature is that commonly used factorial designs with randomization restrictions are special cases of this general representation. Issues related to the use of these designs for particular factorial experiments are also addressed.

It is known that all resolution IV regular 2^n−m designs of run size N=2^n−m where 5N/16<n<N/2 must be projections of the maximal even design with N/2 factors and, therefore, are even designs. This paper derives a general and explicit relationship between the wordlength pattern of any even 2^n−m design and that of its complement in the maximal even design. Using these identities, we identify some (weak) minimum aberration 2^n−m designs of resolution IV and the structures of their complementary designs. Based on these results, several families of minimum aberration 2^n−m designs of resolution IV are constructed.

New types of designs called nested space-filling designs have been proposed for conducting multiple computer experiments with different levels of accuracy. In this article, we develop several approaches to constructing such designs. The development of these methods also leads to the introduction of several new discrete mathematics concepts, including nested orthogonal arrays and nested difference matrices.

We introduce a semiparametric “tubular neighborhood” of a parametric model in the multinomial setting. It consists of all multinomial distributions lying in a distance-based neighborhood of the parametric model of interest. Fitting such a tubular model allows one to use a parametric model while treating it as an approximation to the true distribution. In this paper, the Kullback–Leibler distance is used to build the tubular region. Based on this idea one can define the distance between the true multinomial distribution and the parametric model to be the index of fit. The paper develops a likelihood ratio test procedure for testing the magnitude of the index. A semiparametric bootstrap method is implemented to better approximate the distribution of the LRT statistic. The approximation permits more accurate construction of a lower confidence limit for the model fitting index.

This paper uses the invariance principle to solve the incidental parameter problem of [Econometrica 16 (1948) 1–32]. We seek group actions that preserve the structural parameter and yield a maximal invariant in the parameter space with fixed dimension. M-estimation from the likelihood of the maximal invariant statistic yields the maximum invariant likelihood estimator (MILE). Consistency of MILE for cases in which the likelihood of the maximal invariant is the product of marginal likelihoods is straightforward. We illustrate this result with a stationary autoregressive model with fixed effects and an agent-specific monotonic transformation model.

Asymptotic properties of MILE, when the likelihood of the maximal invariant does not factorize, remain an open question. We are able to provide consistent, asymptotically normal and efficient results of MILE when invariance yields Wishart distributions. Two examples are an instrumental variable (IV) model and a dynamic panel data model with fixed effects.

The gapped local alignment score of two random sequences follows a Gumbel distribution. If computers could estimate the parameters of the Gumbel distribution within one second, the use of arbitrary alignment scoring schemes could increase the sensitivity of searching biological sequence databases over the web. Accordingly, this article gives a novel equation for the scale parameter of the relevant Gumbel distribution. We speculate that the equation is exact, although present numerical evidence is limited. The equation involves ascending ladder variates in the global alignment of random sequences. In global alignment simulations, the ladder variates yield stopping times specifying random sequence lengths. Because of the random lengths, and because our trial distribution for importance sampling occurs on a different sample space from our target distribution, our study led to a mapping theorem, which led naturally in turn to an efficient dynamic programming algorithm for the importance sampling weights. Numerical studies using several popular alignment scoring schemes then examined the efficiency and accuracy of the resulting simulations.