We present a path algorithm for the generalized lasso problem. This problem penalizes the ℓ₁ norm of a matrix D times the coefficient vector, and has a wide range of applications, dictated by the choice of D. Our algorithm is based on solving the dual of the generalized lasso, which greatly facilitates computation of the path. For D = I (the usual lasso), we draw a connection between our approach and the well-known LARS algorithm. For an arbitrary D, we derive an unbiased estimate of the degrees of freedom of the generalized lasso fit. This estimate turns out to be quite intuitive in many applications.

We study a random sampling technique to approximate integrals ∫_[0, 1]^s f(x) dx by averaging the function at some sampling points. We focus on cases where the integrand is smooth, which is a problem which occurs in statistics.

The convergence rate of the approximation error depends on the smoothness of the function f and the sampling technique. For instance, Monte Carlo (MC) sampling yields a convergence of the root mean square error (RMSE) of order N^−1/2 (where N is the number of samples) for functions f with finite variance. Randomized QMC (RQMC), a combination of MC and quasi-Monte Carlo (QMC), achieves a RMSE of order N^−3/2+ε under the stronger assumption that the integrand has bounded variation. A combination of RQMC with local antithetic sampling achieves a convergence of the RMSE of order N^{−3/2−1/s+ε} (where s ≥ 1 is the dimension) for functions with mixed partial derivatives up to order two.

Additional smoothness of the integrand does not improve the rate of convergence of these algorithms in general. On the other hand, it is known that without additional smoothness of the integrand it is not possible to improve the convergence rate.

This paper introduces a new RQMC algorithm, for which we prove that it achieves a convergence of the root mean square error (RMSE) of order N^{−α−1/2+ε} provided the integrand satisfies the strong assumption that it has square integrable partial mixed derivatives up to order α > 1 in each variable. Known lower bounds on the RMSE show that this rate of convergence cannot be improved in general for integrands with this smoothness. We provide numerical examples for which the RMSE converges approximately with order N^−5/2 and N^−7/2, in accordance with the theoretical upper bound.

Let (X_i)_i≥1 be a stationary mean-zero Gaussian process with covariances $\rho(k)=\mathbb {E}(X_{1}X_{k+1})$ satisfying ρ(0) = 1 and ρ(k) = k^−DL(k), where D is in (0, 1), and L is slowly varying at infinity. Consider the U-process {U_n(r), r ∈ I} defined as $$U_n(r) =\frac{1}{n(n−1)} \sum_{1≤i≠j≤n}1_{\{G(X_i, X_j)≤r\}},$$ where I is an interval included in ℝ, and G is a symmetric function. In this paper, we provide central and noncentral limit theorems for U_n. They are used to derive, in the long-range dependence setting, new properties of many well-known estimators such as the Hodges–Lehmann estimator, which is a well-known robust location estimator, the Wilcoxon-signed rank statistic, the sample correlation integral and an associated robust scale estimator. These robust estimators are shown to have the same asymptotic distribution as the classical location and scale estimators. The limiting distributions are expressed through multiple Wiener–Itô integrals.

A new time series bootstrap scheme, the time frequency toggle (TFT)-bootstrap, is proposed. Its basic idea is to bootstrap the Fourier coefficients of the observed time series, and then to back-transform them to obtain a bootstrap sample in the time domain. Related previous proposals, such as the “surrogate data” approach, resampled only the phase of the Fourier coefficients and thus had only limited validity. By contrast, we show that the appropriate resampling of phase and magnitude, in addition to some smoothing of Fourier coefficients, yields a bootstrap scheme that mimics the correct second-order moment structure for a large class of time series processes. As a main result we obtain a functional limit theorem for the TFT-bootstrap under a variety of popular ways of frequency domain bootstrapping. Possible applications of the TFT-bootstrap naturally arise in change-point analysis and unit-root testing where statistics are frequently based on functionals of partial sums. Finally, a small simulation study explores the potential of the TFT-bootstrap for small samples showing that for the discussed tests in change-point analysis as well as unit-root testing, it yields better results than the corresponding asymptotic tests if measured by size and power.

We consider the statistical experiment of functional linear regression (FLR). Furthermore, we introduce a white noise model where one observes an Itô process, which contains the covariance operator of the corresponding FLR model in its construction. We prove asymptotic equivalence of FLR and this white noise model in LeCam’s sense under known design distribution. Moreover, we show equivalence of FLR and an empirical version of the white noise model for finite sample sizes. As an application, we derive sharp minimax constants in the FLR model which are still valid in the case of unknown design distribution.

Testing covariance structure is of significant interest in many areas of statistical analysis and construction of compressed sensing matrices is an important problem in signal processing. Motivated by these applications, we study in this paper the limiting laws of the coherence of an n × p random matrix in the high-dimensional setting where p can be much larger than n. Both the law of large numbers and the limiting distribution are derived. We then consider testing the bandedness of the covariance matrix of a high-dimensional Gaussian distribution which includes testing for independence as a special case. The limiting laws of the coherence of the data matrix play a critical role in the construction of the test. We also apply the asymptotic results to the construction of compressed sensing matrices.

In this paper, a uniform (over some parameter space) moment bound for the inverse of Fisher’s information matrix is established. This result is then applied to develop moment bounds for the normalized least squares estimate in (nonlinear) stochastic regression models. The usefulness of these results is illustrated using time series models. In particular, an asymptotic expression for the mean squared prediction error of the least squares predictor in autoregressive moving average models is obtained. This asymptotic expression provides a solid theoretical foundation for some model selection criteria.

Within a Bayesian decision theoretic framework we investigate some asymptotic optimality properties of a large class of multiple testing rules. A parametric setup is considered, in which observations come from a normal scale mixture model and the total loss is assumed to be the sum of losses for individual tests. Our model can be used for testing point null hypotheses, as well as to distinguish large signals from a multitude of very small effects. A rule is defined to be asymptotically Bayes optimal under sparsity (ABOS), if within our chosen asymptotic framework the ratio of its Bayes risk and that of the Bayes oracle (a rule which minimizes the Bayes risk) converges to one. Our main interest is in the asymptotic scheme where the proportion p of “true” alternatives converges to zero.

We fully characterize the class of fixed threshold multiple testing rules which are ABOS, and hence derive conditions for the asymptotic optimality of rules controlling the Bayesian False Discovery Rate (BFDR). We finally provide conditions under which the popular Benjamini–Hochberg (BH) and Bonferroni procedures are ABOS and show that for a wide class of sparsity levels, the threshold of the former can be approximated by a nonrandom threshold.

It turns out that while the choice of asymptotically optimal FDR levels for BH depends on the relative cost of a type I error, it is almost independent of the level of sparsity. Specifically, we show that when the number of tests m increases to infinity, then BH with FDR level chosen in accordance with the assumed loss function is ABOS in the entire range of sparsity parameters p ∝ m^−β, with β ∈ (0, 1].

In this paper we study the consistency of different bootstrap procedures for constructing confidence intervals (CIs) for the unique jump discontinuity (change-point) in an otherwise smooth regression function in a stochastic design setting. This problem exhibits nonstandard asymptotics, and we argue that the standard bootstrap procedures in regression fail to provide valid confidence intervals for the change-point. We propose a version of smoothed bootstrap, illustrate its remarkable finite sample performance in our simulation study and prove the consistency of the procedure. The m out of n bootstrap procedure is also considered and shown to be consistent. We also provide sufficient conditions for any bootstrap procedure to be consistent in this scenario.

We address the problem of density estimation with "$\mathbb{L}_{s}$-loss by selection of kernel estimators. We develop a selection procedure and derive corresponding $\mathbb{L}_{s}$-risk oracle inequalities. It is shown that the proposed selection rule leads to the estimator being minimax adaptive over a scale of the anisotropic Nikol’skii classes. The main technical tools used in our derivations are uniform bounds on the $\mathbb{L}_{s}$-norms of empirical processes developed recently by Goldenshluger and Lepski [Ann. Probab. (2011), to appear].

This paper deals with the consistency of the nonparametric least squares estimator of a convex regression function when the predictor is multidimensional. We characterize and discuss the computation of such an estimator via the solution of certain quadratic and linear programs. Mild sufficient conditions for the consistency of this estimator and its subdifferentials in fixed and stochastic design regression settings are provided.

Single-index models are natural extensions of linear models and circumvent the so-called curse of dimensionality. They are becoming increasingly popular in many scientific fields including biostatistics, medicine, economics and financial econometrics. Estimating and testing the model index coefficients β is one of the most important objectives in the statistical analysis. However, the commonly used assumption on the index coefficients, ‖β‖ = 1, represents a nonregular problem: the true index is on the boundary of the unit ball. In this paper we introduce the EFM approach, a method of estimating functions, to study the single-index model. The procedure is to first relax the equality constraint to one with (d − 1) components of β lying in an open unit ball, and then to construct the associated (d − 1) estimating functions by projecting the score function to the linear space spanned by the residuals with the unknown link being estimated by kernel estimating functions. The root-n consistency and asymptotic normality for the estimator obtained from solving the resulting estimating equations are achieved, and a Wilks type theorem for testing the index is demonstrated. A noticeable result we obtain is that our estimator for β has smaller or equal limiting variance than the estimator of Carroll et al. [J. Amer. Statist. Assoc. 92 (1997) 447–489]. A fixed-point iterative scheme for computing this estimator is proposed. This algorithm only involves one-dimensional nonparametric smoothers, thereby avoiding the data sparsity problem caused by high model dimensionality. Numerical studies based on simulation and on applications suggest that this new estimating system is quite powerful and easy to implement.

We propose statistical tests to discriminate between the finite and infinite activity of jumps in a semimartingale discretely observed at high frequency. The two statistics allow for a symmetric treatment of the problem: we can either take the null hypothesis to be finite activity, or infinite activity. When implemented on high-frequency stock returns, both tests point toward the presence of infinite-activity jumps in the data.

Fully nonparametric methods for regression from functional data have poor accuracy from a statistical viewpoint, reflecting the fact that their convergence rates are slower than nonparametric rates for the estimation of high-dimensional functions. This difficulty has led to an emphasis on the so-called functional linear model, which is much more flexible than common linear models in finite dimension, but nevertheless imposes structural constraints on the relationship between predictors and responses. Recent advances have extended the linear approach by using it in conjunction with link functions, and by considering multiple indices, but the flexibility of this technique is still limited. For example, the link may be modeled parametrically or on a grid only, or may be constrained by an assumption such as monotonicity; multiple indices have been modeled by making finite-dimensional assumptions. In this paper we introduce a new technique for estimating the link function nonparametrically, and we suggest an approach to multi-index modeling using adaptively defined linear projections of functional data. We show that our methods enable prediction with polynomial convergence rates. The finite sample performance of our methods is studied in simulations, and is illustrated by an application to a functional regression problem.

We develop a method to carry out MAP estimation for a class of Bayesian regression models in which coefficients are assigned with Gaussian-based spike and slab priors. The objective function in the corresponding optimization problem has a Lagrangian form in that regression coefficients are regularized by a mixture of squared l₂ and l₀ norms. A tight approximation to the l₀ norm using majorization–minimization techniques is derived, and a coordinate descent algorithm in conjunction with a soft-thresholding scheme is used in searching for the optimizer of the approximate objective. Simulation studies show that the proposed method can lead to more accurate variable selection than other benchmark methods. Theoretical results show that under regular conditions, sign consistency can be established, even when the Irrepresentable Condition is violated. Results on posterior model consistency and estimation consistency, and an extension to parameter estimation in the generalized linear models are provided.

Inference for partially observed Markov process models has been a longstanding methodological challenge with many scientific and engineering applications. Iterated filtering algorithms maximize the likelihood function for partially observed Markov process models by solving a recursive sequence of filtering problems. We present new theoretical results pertaining to the convergence of iterated filtering algorithms implemented via sequential Monte Carlo filters. This theory complements the growing body of empirical evidence that iterated filtering algorithms provide an effective inference strategy for scientific models of nonlinear dynamic systems. The first step in our theory involves studying a new recursive approach for maximizing the likelihood function of a latent variable model, when this likelihood is evaluated via importance sampling. This leads to the consideration of an iterated importance sampling algorithm which serves as a simple special case of iterated filtering, and may have applicability in its own right.

When considering d possibly dependent random variables, one is often interested in extreme risk regions, with very small probability p. We consider risk regions of the form {z ∈ ℝ^d : f(z) ≤ β}, where f is the joint density and β a small number. Estimation of such an extreme risk region is difficult since it contains hardly any or no data. Using extreme value theory, we construct a natural estimator of an extreme risk region and prove a refined form of consistency, given a random sample of multivariate regularly varying random vectors. In a detailed simulation and comparison study, the good performance of the procedure is demonstrated. We also apply our estimator to financial data.