Consistency of the Frequency Domain Bootstrap for differentiable functionals

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Consistency of the Frequency Domain Bootstrap for differentiable functionals Patrice Bertail, Anna Dudek


Introduction
Bootstrap for dependent data has been developed over the last three decades. Most of existing bootstrap approaches are designed for the time domain. Widely applied for stationary time series are block bootstrap methods. For instance, the Moving Block Bootstrap ( [25], [27]), the Circular Block Bootstrap ( [37]), the Stationary Bootstrap ( [38]), the Tapered Block Bootstrap ( [33]), the Regenerative Block Bootstrap ( [3]). Some of these techniques can be also adapted for nonstationary data. Additionally, there exist methods introduced for particular classes of nonstationary time series. Among them we have the Seasonal Block Bootstrap ( [35]), the Periodic Block Bootstrap ( [5]), the Generalized Seasonal Block Bootstrap ( [15]), the Generalized Seasonal Tapered Block Bootstrap ( [16]), the Extension of Moving Block Bootstrap ( [13], [14]). Sometimes in the parametric setting it is also possible to apply to dependent sequences the techniques designed for i.i.d. data like the i.i.d. bootstrap of Efron [17] or wild bootstrap of Wu [45] (see e.g., [26] and [43]).
Alternatively, one may bootstrap the time series in the frequency domain. In that case the usual approach is to apply the i.i.d. bootstrap to studentized periodogram estimates ( [19], [18], [29]). The most classical example of this idea is the Frequency Domain Bootstrap (FDB). Other method called the Local Bootstrap (LB) was proposed in [32]. In this approach one bootstraps the periodogram ordinates locally around the frequency of interest. In contrary to the FDB, the LB does not require estimation of the spectral density function. Both methods share same limitations of applicability, i.e. they are consistent only for some classes of functionals. To extend applicability of the bootstrap in the frequency domain a few other bootstrap methods were proposed: the Autoregressive Aided Periodogram Bootstrap ( [24]), the Convolved Bootstrapped Periodograms of Subsamples (CBPS) ( [28]) and the Time Frequency Toggle (TFT)-bootstrap ( [23]). All these approaches are much more difficult to implement than the FDB. They depend on unknown tuning parameters. Moreover, in contrary to other techniques the TFT-bootstrap is not purely a frequency domain technique. Indeed, the idea is to bootstrap Fourier coefficients obtained after applying a fast Fourier transform to the considered time series, and at the end, to back-transformed these quantities to obtain a bootstrap sample in the time domain. It should be also noticed that the CBPS is asymptotically valid in quite general framework for linear functionals and could be adapted to the general functional considered in this paper, but it seems very challenging to study its second order properties.
In this paper we focus on the classical Frequency Domain Bootstrap. Till now its consistency/inconsistency was proven in some particular cases, mainly for stationary linear processes. Franke and Härdle [18] considered the problem of spectral density estimation, while Dahlhaus and Janas [10] obtained validity of the FDB for ratio statistics and Whittle estimator. Finally, Kim and Nordman [22] extended its applicability for Whittle estimator to long-range dependent linear models. It is worth to indicate that the FDB works for Whittle estimator since the functional corresponding to the Whittle estimator may be expressed approximately as a ratio statistic as will be seen later. Moreover, it is known that the FDB is not consistent for some functionals e.g., the autocovariance function. This originates from the fact that the FDB assumes that periodogram ordinates are independent while for non-Gaussian processes this condition usually does not hold. As a result bootstrap variance for the considered statistics not always converges to the asymptotic one (see e.g., [20], [31]). However, till now in the literature there is no result stating the form of the class of functionals for which the FDB is consistent, while for particular functional it is important to know if the simplest bootstrap approach is valid or some generalization is essential. The main advantage of the FDB method is the fact that it is based on the i.i.d. bootstrap and does not require choosing any tuning parameters like for instance block length. Moreover, till now the second order correctness was proven only for the FDB in some specific cases.
The aim of this paper is twofold. At first, we revisit some well known bootstrap consistency results for spectral density function. We focus on smooth functions of linear functionals of spectral density. In particular we give necessary and sufficient conditions for the validity of the FDB. Essentially, in the general (non-Gaussian) case, the FDB works for functions (of linear functionals), which are homogeneous of degree 0. We then generalize existing results to differentiable functionals of spectral density function in the framework of stationary linear processes. For that purpose we introduce a concept of influence function in the time domain analogously to the i.i.d. case, but on spectral measures instead of cumulative distribution functions. These influence functions behave quite differently than in the usual i.i.d. set-up and may not be automatically centered. Moreover, we define the notion of the Fréchet differentiability of functionals of spectral measures. The FDB is asymptotically valid if and only if the kurtosis of the process is 0 (for instance in the Gaussian case) or if the functional of interest has a centered influence function, which is the case for ratio statistics as well as some Whittle estimators. We then study under what conditions the empirical process in the frequency domain, as considered in [8], converges to some Gaussian process and when its bootstrap version is valid. We essentially show that this holds if and only if all the functions are centered with respect to the given spectral density. In other cases, the bootstrap will fail to give the correct asymptotic distributions. These results allow us to prove the validity of the bootstrap for large classes of interesting statistics. In details, we discuss case of quantile and range. For this purpose we introduce a version of the Hadamard differentiability in the time domain.
Paper is organized as follows. In Section 2 notation is introduced and the FDB algorithm is recalled. Consistency of the FDB for differentiable functions of linear functionals and its second order correctness is discussed in Section 3. Influence function in the time domain is introduced in Section 4 and some ex-amples are presented. Section 5 is dedicated to the sufficient and necessary conditions for asymptotic validity of the FDB for Fréchet differentiable functionals in the time domain. The analogous conditions for empirical processes are formulated in Section 6. In Section 7 results of a simulation study are provided. Section 8 contains short discussion of the obtained results. All proofs can be found in the Appendix.

Problem formulation
Let {X t , t ∈ Z} be a real valued stationary time series. For simplicity we assume that EX t = 0. Moreover, let X t admit an infinite moving average representation where (ζ t ) t∈Z is an i.i.d. sequence with Eζ 2 t = σ 2 and Eζ 8 t < ∞. Such conditions allow to verify easily the conditions for asymptotic normality of the estimators that we are going to consider (see [7], Corollary 3.2).
Let R(k) = Cov(X 1 , X 1+k ) be the autocovariance function of the process X t and let be its spectral density function.
By I n (ω) we denote the periodogram i.e., is the discrete Fourier transform of {X t , t ∈ Z}. It is known that I n (ω) is not a consistent estimator of the spectral density f but it is asymptotically unbiased and may serve as a basis for estimating many parameters.
In this paper we are interested in functionals of the spectral density T (f ) and in particular, smooth functions g (second order differentiable) of linear functionals We usually estimate T (f ) using the plug-in estimator T (I n ). Its bootstrap counterpart is obtained using the Frequency Domain Bootstrap (FDB), which we recall in Section 2.2.

Discretized versions
In general, to compute integrals in A(ξ, f ) we estimate the functional by using the Riemann approximation of the integral at specific frequencies, most of the time the Fourier frequencies λ jn = 2πj/n, j = 1, . . . , n 0 , where n 0 = [n/2] is the integer part of n/2. We denote However, it is known that the error of approximation in this case is of order where a n ∼ b n means that a n /b n −→ 1 as n −→ ∞. Assume in addition that ξ 1 (ω)f (ω) is twice differentiable, which, under the condition that we introduce later, will reduce to the assumption that the spectral density is twice differentiable. Then one can rather choose an approximation at the midpoint frequencies λ jn = (2πj +π)/n. By the well known Polya's theorem, we have If the spectral density is not twice differentiable, then the approximation will be of order O(n −1 ). Notice that the periodogram is infinitely differentiable as a function of ω. Thus, when we replace f by I n we automatically get As a consequence we will always gain in using the discretization at midpoint frequencies λ jn = (2πj + π)/n and hence such λ jn are chosen in the sequel. This is a minor point if we are interested only in the first order asymptotics but it can have important consequence for one and two sided confidence intervals. The approximation error in the case of the standardized version of typically implies a bias of a size 1/ √ n, when the standard Fourier frequencies are used. To our knowledge, this has not been discussed before in the bootstrap literature.

Frequency Domain Bootstrap
The main idea underlying the FDB is based on the observation that the periodogram values evaluated at different frequencies 0 < ω 1 < ω 2 < · · · < ω k < π, I n (ω j ), j = 1, . . . , k are asymptotically independent and exponentially distributed i.e., asymptotically we have The FDB essentially consists in "bootstrapping" these standardized frequencies, once f is estimated by some convergent estimator.
Recall that for linear processes considered here, we have (see [41], [26], p. 235) where is the kurtosis of the innovations. This means that even if the random variables are asymptotically independent, they are not independent for finite n and hence we expect the i.i.d. bootstrap of the frequencies to have problem when k 4 = 0.
Below we recall the FDB algorithm together with its simple modification and the consistency result.
Step 0 Compute an estimator of the spectral density f , for instance the kernel estimator where n 0 = [n/2] and λ jn = (2πj + π)/n. Moreover, k is a kernel on [−π, π], that is a real valued, non-negative, symmetric function such that ∞ −∞ k(x)dx = 1. The smoothing window parameter h = h n converges to 0 as n → ∞.
Step 1 Approximate the functional by the sequence of T n = T n (I n ) = g(A n (ξ, I n )), where The correcting factor 2π appears because we are evaluating the functions at "the Fourier frequencies".
Because of the estimated standardization, the mean of jn may be neither equal nor close to 1. To solve this problem, compute the rescaled values Step 3 Generate * jn i.i.d. from the empirical distribution P n0 = 1 n0 n0 i=1 δ jn (by construction E Pn 0 * jn = 1 and Var Pn 0 ( * jn ) → 1) that is, draw the bootstrap values randomly with replacement from these rescaled values and compute the bootstrapped periodogram values I * n (λ jn ) = f n (λ jn ) * jn , j = 1, . . . , n 0 .
In Step 0 of the FDB algorithm we used the standard kernel estimator with a normalization of the kernel to 1 on R ( ∞ −∞ k(x)dx = 1) instead of 2π, which may be found in [26] (see p. 299 below expression (9.20)) or i.e., in [18], [31], [24]. This standardization by 2π is a source of confusion in many applications and in several expressions, all the more than the conventions are not same in signal theory.
The (2π) 2 in the expression for f n (ω) (see formula (2.3)) may be surprising, but is essentially due to the fact that the functions are taken at the Fourier )dx, which converges to 1/(2π) as h → 0. Integrating locally I n (2πx) also results in an additional 1/2π term.
In Step 1 in some cases it is possible to consider the estimator where f n is an estimator of f . In the parametric case one may use the parametric estimator where θ n is a convergent estimator of the parameter θ (for instance a Whittle estimator of θ, under specific assumptions).
Once the frequencies are resampled, then compute the corresponding value of the statistics T n , that is

Consistency of FDB for differentiable functions of linear functionals
The following theorem, which summarizes the main known results, shows that the FDB is valid only for specific processes or specific functionals (see Lahiri (2003) ch. 9.2 and references therein).
t ∈ Z} is a stationary linear process of the form where (ζ t ) t∈Z is an i.i.d. white noise with Eζ 2 t = σ 2 and Eζ 8 t < ∞; (ii) the spectral density estimator f n converges to f uniformly over [0, π]; .

Moreover, the FDB satisfies
.
Notice that we have Thus, the bootstrap asymptotically works when either κ 4 = 0 or if the functional of interest is such that the quantities π 0 ξ i (ω)f (ω)dω = 0. The first condition is for instance satisfied in the Gaussian case, but is very restrictive in the framework of bootstrap.
Let us recall that we are interested in a smooth functional T (f ) = g(A(ξ, f )), which is differentiable around A(ξ, f ). Using Slutsky's lemma (or the delta method) we derive the corresponding equation to (3.1). We have where ∇ is the gradient operator. Thus, the FDB has the correct asymptotic variance iff k 4 = 0 or In that case we have that (under appropriate assumptions) In particular it is easy to check that for p = 2, if g(x, y) = x/y then (3.2) holds immediately. This explains why the bootstrap is asymptotically valid for any ratio statistics for which the second coordinate of A(ξ, f ) is nonzero (see [20], [26]). But actually this result covers more functionals for general p. Indeed, if we want to have for any value of the parameter A(ξ, f ) in an open set, then equivalently we have in an open set. This equation is known as the Euler differential equation and is equivalent to the fact that g is homogeneous of degree zero. Let us recall that g is homogeneous of degree zero iff which yields by derivation in λ to Recall also that if at least one of the coordinates does take the value 0 (a.s.), then the function g can always be expressed as a function of ratio (which explains why ratio plays such an important role in the validity of the FDB).

Examples:
1. the FDB for the spectral distribution function, of the variance estimator or of the autocovariance function R(k) fails unless the kurtosis of the innovations k 4 = 0; 2. the FDB for autocorrelation, which can be expressed as ratio statistics, asymptotically works; x αi i with p i=1 α i = 0, then the FDB works asymptotically; 4. the FDB is also valid for functionals that are not directly ratios. For instance for p = 2 one can take g(x, y) = xy (x 2 +y 2 ) , or for p = 3, g(x, y, z) = xyz (x 2 z+y 2 x+z 2 y) . These functions are differentiable outside the set of points for which the denominator equals 0 and they are homogeneous of degree zero. Moreover, they all can be expressed as functions of different types of ratios.

Second order theory for the FDB
Dahlhaus and Janas in [10] proved the second order validity of the FDB in the particular case when the statistics of interest are ratio of linear functionals.
Following their proof it is easy to show that the same result holds for any function of linear functionals, that is smooth and homogenous of degree 0. Recall that we consider the process with linear representation Its transfer function is given by and the spectral density is such that for any ω ∈ [−π, π], Moreover, let f be a tapered estimator of f of the form where for some ρ ∈ (0, 1] (the proportion of tapered data), we define the taper To obtain the second order validity of the FDB we consider the following assumptions.

A1
The function g is twice differentiable, homogeneous of degree 0, such that Moreover, the tapered estimator f (ω) is uniformly strongly consistent i.e., a vector of bounded functions having bounded variation, which are extended to the whole real line in a way that the extension is symmetric around 0 and periodic (with period 2π). A6 The proportion of tapered data ρ = ρ n (see (3.5)) is such that

A7
The filter coefficients {a j } j∈Z and the Fourier coefficients ξ(ω) of ξ(ω) are decreasing exponentially that is, there exists C > 0 such that for all large ω. A8 The following Cramér condition holds: for some some 0 < δ < 1 and some M > 0, for any t = (t 1 , t 2 ), ||t|| > M, Then the limits of Cov(W n,1 ) and Cov (W n,2 ) exist and are nonsingular. Moreover, Assumption A3 is very strong. It is clearly satisfied in the Gaussian case but not for general processes. Condition A8 is automatically satisfied if ξ has an absolutely continuous part with respect to Lebesgue measure on R. Finally, A9 ensures that the cumulants of order 4 are not degenerate and that their empirical versions are close to the true ones.
Theorem below states the second order correctness of the FDB for functions or ratio functionals of the spectral density (when standardized by the true variance).
where C P (f ) is a constant depending on P and f . If in addition A3 holds (the skewness of the residuals is 0), then the bootstrap is second order correct.
Theorem 3.2 shows that the FDB is asymptotically valid and/or second order correct under very specific conditions: • for any smooth functional, the bootstrap will be second order correct if E P ζ 3 1 = 0 and k 4 = E P ζ 4 1 / E P ζ 2 1 2 − 3 = 0 that is typically in the Gaussian case; • only homogeneous functions of degree 0 of linear functionals (including ratios of linear functionals) are candidates for the asymptotic validity when k 4 = 0; • only linear time series with i.i.d. innovations such that Eζ 3 1 = 0 can be second order correct without corrections.
Thus, one should be careful while applying the FDB method on specific functionals and should not expect second order corrections without some further modification of the procedure.
A solution to obtain second order valid confidence intervals via calibration of the quantile of the bootstrap distribution is to use the Edgeworth expansion inversion (see [1]) when M 3 = 0. Indeed it is easy to see with their results that if one has estimators of the quantities M 3 , σ 2 and C P (f ) say M 3 , σ 2 and C such that then we can correct either the original statistics or the bootstrap quantiles to get second order correction. However, such methods may require some complicated computations to obtain a valid estimator of C P (f ).

Influence function in the time domain
In this section we introduce a concept of an influence function in the time domain that will allow us later to state sufficient and necessary conditions for consistency of the FDB. Note that the functional T (f ) = g(A(ξ, f )) can be seen as a functional of the spectral measure F on [0, π]. We have We denote A(ξ, f ) and T (f ) by A(ξ, F ) and T(F ), respectively, to stress the dependence of F rather than of f . The natural estimator of T(F ) is simply may be also seen by extension as a positive measure Since we know that it is easier to get asymptotic distribution of the process √ n( F n − F ), than of the corresponding process based on non-integrated periodogram, it is natural to try to study the differentiability property of T in the time domain, to get an analogue of the functional delta-method. In this case we will see that it is possible to introduce a contaminated version of F by some Dirac measure to compute an equivalent of the influence function but in the time domain.
Let T be a functional defined on a vectorial space of positive measure on [0, π] including Dirac measures, denoted by F. We define the following notion of influence function in the time domain analogously to the i.i.d. case, but on spectral measures instead of cdf's.
It is known that the purely periodic process where EA = EB = 0 and Var(A) = Var(B) = 1, has spectral measure given by Then for a general process X t with spectral measure F on [0, π], the contaminated process X t + √ 2εη t , where X t and η t are independent, has spectral measure F + εδ ω0 on [0, π]. Thus, the influence function can be interpreted as the infinitesimal variation of the functional T(F ) when F is contaminated by a purely periodic process with variance going to 0 at rate √ 2ε.

Examples 1-3:
1. The linear functional A(ξ, F ) has influence function given by Notice that in general A particular case is the covariance function of order k and for some k π 0 cos(ωk)f (ω)dω = 0.

Whittle estimators.
For simplicity all the calculus are made for θ ∈ R, but the final result actually also holds for the multidimensional case. A Whittle estimator can be seen as a M-estimator that is a solution of the equation Thus, we can define θ = T(F ), the functional solution of the equation To compute the influence function of T(F ), we consider θ • ε = T(F + εδ ω0 ) the solution of the equation Then calculating the derivative with respect to ε, we get that which yields the influence function Moreover, one may note that in the particular case of ARMA or FARIMA models, we have π 0 ∂ ∂θ 1 f θ (ω) F (dω) = 0 (see the comment under Remark 1, p. 409 in [22]. Thus, we get

General contrasts.
Notice that the Whittle estimators are a simple case of contrast estimators satisfying some estimating equation Assuming that ψ is twice differentiable and that there exists a unique solution to this problem for any F in F, it is easy to compute the corresponding influence function. Consider θ By derivation, we get and hence the influence function is of the form The main benefit of having the influence function is that it allows to linearize the functional of interest. Typically we expect that where R n is a remainder, which needs to be controlled either by choosing an adequate metric or directly by hand. In many applications (e.g., Whittle estimator) this remainder is typically of order R n = o P (n −1/2 ). As a consequence the limiting behavior of √ n(T( F n ) − T(F )) is determined by the linear part i.e., Notice that this linearization allows to study the behavior of the contrast whether the true model is really F θ (in this case the influence function becomes very simple) or under a misspecified model (notice the shift in the denominator).

Sufficient and necessary conditions for asymptotic validity of FDB
A von Mises' type of theorem follows immediately from the representation above. Moreover, under an additional assumption controlling the behavior of the remainder evaluated at F * n , we establish a necessary and sufficient condition for the asymptotic validity of the FDB of a non-degenerate general functional.
Theorem 5.1. Assume that the assumptions (i)-(iii) of Theorem 3.1 hold and that T : in probability along the sample. Then the bootstrap is asymptotically valid iff

Remark 5.2. The condition (5.1) essentially means that either k 4 = 0, which is true in the Gaussian case, or the influence function is centered. As already noticed the latter may not be the case with our definition of the influence function as shown in the following examples.
Examples: 1. The FDB for a linear functional with only one function (p = 1) i.e., for A(ξ 1 , F ), is valid iff k 4 π 0 ξ 1 (ω)f (ω)dω = 0. In particular one can notice that for the autocovariance the FDB does not work if k 4 = 0. 2. The influence function of the ratio T (F ) = π 0 ξ 1 (ω)F (dω)/ π 0 ξ 2 (ω)F (ω) is given by .
Note that the influence function is automatically centered.

Whittle estimators (continuation of Example 2 from Section 4).
In this case, for ARMA or FARIMA models, we have the recentering property T (1) (ω 0 , F θ ) is precisely the linear part obtained in [10]. It is automatically centered under their assumptions. This explains why the limiting distribution does not depend on k 4 and why the bootstrap works (asymptotically) in that case. However, notice that in models where the variance depends on θ, the influence function given by (4.1) should be considered and may not be centered, so that the bootstrap may fail in that case!

(continuation of Example 3 from Section 4).
If the M-estimator is constructed in the way that then under the assumptions of Theorem 5.1 the FDB is valid. In particular, notice that the Whittle estimator satisfies this property.

Invalidity/validity of the bootstrap for empirical processes in the time domain
Empirical spectral processes indexed by some class of real functions H satisfying some integrability conditions are studied in [8]. The framework in the time domain is a bit different from the usual one. We consider class of functions of the following form We are interested in the behavior of the (infinite) dimensional vectors of the form and more precisely in the discretized version of this quantity at the Fourier frequencies. We put Let us introduce d H the pseudo-distance between spectral densities defined by In the following, we will also be interested in the rate of convergence d H (I n , f). Indeed typically to ensure that Theorem 5.1 yields a CLT or to study general M-estimators including the Whittle estimators, we end-up with controlling d H (I n , f) for a specific class of function (see examples in [9]). Moreover, we want to check under which conditions we have convergence of the bootstrap versions to the same limit or at least when we have d( F * n , F ) = O P (n −1/2 ) as assumed in Theorem 5.1 for this kind of metrics.
Additionally, we introduce the computable discretized version of d F (f 1 , f 2 ) given by which obviously converges to d F (f 1 , f 2 ) as n → ∞.
As in [8] the process (Z n (h), h ∈ H) is a random element of l ∞ (H) (the space of all bounded functions from H to R) equipped with the metric, ||z|| H = sup h∈H |z(h)|. Moreover, as in the usual case (l ∞ (H), ||z|| H ) is a (generally non-separable) Banach space. It was proven in [8] that under the conditions discussed below (Z n (h)) h∈H converges to a Gaussian process in l ∞ (H).
Define the semi-metric on H When f is bounded (as will be the case later), it is possible to use as in [9]. However, notice that if one wants to generalize the results presented below to fractional stationary times series with a singularity at 0, then ρ 2 should be used. The bracketing number N (δ, H, ρ 2 ) is defined as the smallest number m such that, there exist functions g 1 , g 2 , . . . , g m ∈ H such that for any g ∈ H, inf 1≤i≤m ρ 2 (g, g i ) ≤ δ. At this point we refer the reader to the discussion in [9] explaining the link between bracketing numbers and regular covering numbers (the number of balls needed to cover H with balls of size δ) and providing examples of calculus of this quantity for many classes of functions. In many examples, when f is bounded, and these quantities can be bounded by a polynomial Cδ −V for some positive constants C and V . Following [8] we assume the following conditions: B0 the process (X t ) t∈Z is strictly stationary and centered; B1 function f is continuous and Hölder of order k > 1/2 (and thus bounded) that is, for some positive constant K B2 the fourth order spectrum is continuous and the spectrum of all order m ≥ 2 are bounded by C m , where C is some positive constant; B3 (H, ρ 2 ) is totally bounded and is a permissible subset of the set of all real functions such that B4 the covering number satisfies the integrability condition In this section similarly to [8] we make a stronger assumption B0 on the generating process. The reason for this is that exponential inequalities (here on weighted sums of the periodogram at the Fourier frequencies) are essential to obtain maximal inequalities and to control increments of empirical processes as was done in the i.i.d. case. Condition B4 comes from the paper by Dahlhaus and Polonik [9] who have improved the original condition of Dahlhaus in [8]. Notice that B4 is stronger than the usual assumption (in the i.i.d. case) which would rather be of the form 1 0 (log (N (δ, H, ρ 2 )) 1/2 < ∞. This means that N (δ, H, ρ 2 ) should be of an order much smaller than exp(δ −1/2 ) (rather than the "Gaussian" rate exp(δ −2 )). In most applications (in particular if the bracketing number is polynomial) this condition will be satisfied. This restriction is due to the fact that the exponential inequalities obtained in this framework are typically proved with a suboptimal rate of order exp(−t 1/2 ) instead of the Gaussian rate exp(−t 2 ).
In the following we consider the bootstrap version of (Z n (f ), h ∈ H), say (Z * n (f ), h ∈ H), obtained using the standard FDB procedure described before (either with a parametric estimator of the spectral density or with a nonparametric one), where

P. Bertail and A. E. Dudek
Moreover, the bootstrap empirical spectral process Z * n (h) also converges in (l ∞ (H), ||z|| H ) to a (different) centered Gaussian process with continuous sample paths and covariance operator given, for all h ∈ H, g ∈ H, As a consequence the FDB of the empirical spectral process is only asymptotically valid on classes of functions satisfying the additional conditions, for ALL h ∈ H Moreover, in any case we have d H (I * n , I n ) = O P (n −1/2 ) in probability along the sample. These results allow us to prove the validity of the bootstrap for large classes of interesting statistics. For this purpose we introduce a version of the Hadamard differentiability in the time domain (more precisely in (l ∞ (H), ||z|| H )), which is weaker than the Fréchet differentiability introduced before. This notion is a time domain version of the Hadamard differentiability definition proposed in [40] for regular functionals in probability spaces, which has been successfully used in many applications. As an illustration we discuss the notions of quantile and range in the time domain, which may be useful to detect with high probability the important frequencies in a signal. Finally, we state the validity of the FDB for these functionals.
For a given spectral measure F in ∞ (H), define B(H, F) as the set of spectral measures Q in ∞ (H) whose paths f ∈ H → Qf = π 0 f (w)Q(dw) are ρ 2uniformly continuous and bounded. This is the smallest natural space containing the Gaussian limiting process of the empirical process studied before. 0.
Notice that T may not be defined on the entire space ∞ (H), but on a subset F only. In this case, one must check that Recall that Hadamard differentiability is weaker than Fréchet differentiability and if the influence function is easily computable as T (1) (ω, F ), then dT F may be express as The following result establishes a CLT for Hadamard differentiable functionals in the time domain and the necessary and sufficient conditions for the validity of the FDB. F tangentially to B(H, F) for a class of functions H which satisfies  assumptions B3-B4. Assume in addition that the additional hypotheses of Theorem 6.1 are satisfied, then we have where G F is the centered Gaussian process with continuous sample paths and covariance operator, given by c (h, g), h ∈ H, g ∈ H in (see (6.1)). If in addition, Hadamard differentiability holds for a set H such that for all h ∈ H, k 4 π 0 h(ω)f (ω)dω = 0 or if the functional admits a non-degenerate influence function such that k 4 π 0 T (1) (ω, F )f (ω)dω = 0, then the limit distribution is given by N 0, 2π π 0 T (1) (ω, F ) 2 f (ω) 2 dω and the FDB is asymptotically valid.
Again the condition k 4 π 0 T (1) (ω, F )f (ω)dω = 0, which is easier to check in practice, holds either if k 4 = 0 (typically for Gaussian processes) or for differentiable functionals with centered influence function in the time domain. Below we provide two important functionals, for which the FDB is valid.

Example 5: Quantiles in the frequency domain
Since the spectral measure F may not be a cumulative distribution function, it is tempting to define its normalized version by The corresponding empirical version based on the integrated periodogram is then defined by Since it is a ratio statistics, this functional (at fixed λ) enjoys the nice properties stressed before: the bootstrap will be automatically valid for this quantity at any λ. Thus, it is easy to define the quantile in the time domain as In particular the range functional F −1 (1 − α/2) − F −1 (α/2) can be used in practice to detect which frequencies are included in the signal with probability 1−α. Let us prove that this functional is Hadamard differentiable with respect to a well chosen set of functions. Assume that we are working in a set of functions F that are continuous with continuous spectral density f , which is lower bounded on [0, π]. Let us first compute "formally" the influence function. Despite the fact that in most situation computing the influence function is easy, in practice, this is the first step that must be taken. Since F ( T α (F )) = α, we have as ε −→ 0, and by derivation at ε = 0 Since F ( F −1 (α)) = αF ([0, π]), we get the expression (which is actually totally similar to the usual influence function of the quantile in an i.i.d. case) Notice that this influence function is automatically centered with respect to f , because we have It is known that the inverse operator is Hadamard differentiable in the set of all functions, which have non-zero derivatives at the point of interest (see [42] and hence is Hadamard differentiable tangentially to B(H, F). It follows that the quantiles associated to the normalized spectral density are Hadamard differentiable with differential given by As a consequence, F −1 n (α) is asymptotically Gaussian and the FDB is consistent, because the influence function is centered. The same result holds for the range F −1 (1 − α/2)− F −1 (α/2) with influence function in the time domain given by and the FDB is consistent even in the non-Gaussian case.

Example 6: Comparing spectra
In many fields, it is interesting to compare the spectral measure or spectral density of two independent signals to test whether they come for instance from the same source (see for instance [6,11]) or to test if is the spectral density is equal to some given spectral density f 2 (see [30]). The idea is simply to introduce a distance between the spectrum densities or the spectrum measures. For instance, metrics of the following forms have been suggested These quantities may be seen respectively as a L 2 metric and a L 1 -Wasserstein distance in the time domain. The idea is simply to use the plug-in versions of (6.2) and (6.3) based on the periodogram to get an estimator of the distance.
Under H 0 : f 1 = f 2 , estimates of these quantities should be close to 0. However, one should be aware of the fact that the test is going to reject the null hypothesis H 0 : f 1 = f 2 against H 1 : f 1 = f 2 even if the spectra of the signals are the same, but the scaling constants for the signal are different. Indeed if (X t ) t∈Z and (Y t ) t∈Z are two independent copies of the same signal (with common spectrum density f ), then (X t ) t∈Z and c(Y t ) t∈Z (c -constant, c = 1), then they have spectral densities f and cf , respectively. Similarly, if one considers scaling by a random variable A > 0 (independent of the signal) with mean 0 and variance σ 2 A , then (X t ) t∈Z and A(Y t ) t∈Z have spectral densities respectively f and σ 2 A f . In many fields, one wants to detect if the spectrum (the frequencies in the signal) is (are) the same, while the scaling is considered as a side effect. Thus, in practice it is recommended to use the standardized versions of the quantities introduced before say where F −1 i , i = 1, 2 are the standardized quantiles introduced in Example 5. It is not difficult to see that these functionals are homogeneous of degree 0 in term of the joint spectrums (f 1 , f 2 ). The corresponding plug-in versions are asymptotically normal and the FDB is asymptotically valid under H 0 .
To explain why it works, consider for instance the simple case, where one wants to test if the signal comes from the standardized spectrum F associated to the spectral density f , based on the functional W 1 ( F 1 , F ). Assume that we observe n consecutive observations from the signal (X t ) t∈Z . Denote by I n,X the periodogram. Then W 1 ( F 1 , F ) is estimated by

P. Bertail and A. E. Dudek
(or its discretized version) and the bootstrap version is given by The functional h → π 0 |h(ω)| dω is a continuous function. By the continuous mapping theorem (see for instance [44]), it is sufficient to prove that √ n ω 0 I n,X (λ)dλ have the same limiting distributions (seen as processes indexed by ω ∈ [0, π]) say GI(ω) and GI * (ω), respectively. But these quantities are actually ratios and hence one can apply Theorem 6.1. Thus, we have which validates the use of the bootstrap for such quantities.
Testing equality between two spectra may be treated similarly (by separating the two estimators under H 0 ). It follows that the quantile of the FDB distribution can be successfully used to test equality to a given spectrum or equality of two spectra. The general case with L p metric is more difficult and requires more sophisticated tools (as is in the i.i.d. case for Wasserstein distance) but provided that we use standardized spectra, we still expect the FDB to be valid.

A simulation study
In this section we perform some simulations to study the performance of the FDB for some Whittle estimator. We generate Gaussian and non-Gaussian, linear and non-linear time series.

1−φ 2
and is independent of ε t , which are i.i.d.
Notice that the model S3 is not linear and hence is not fulfilling assumptions of our theorems. Moreover, all correlations are 0 and hence the spectral density is similar to the one of a white noise and is unable to capture the dependence structure of the original data. Thus, we expect that the FDB will perform badly in that case. For S4 assume that |φ 1 | < 1. Note that S4 is a Markov chain with a Gaussian noise, but is not a Gaussian process. It has a stationary distribution with a very strong dissymmetry. For large values of |X t |, it is easy to see that in mean X t+1 behaves like a simple AR(1) model with coefficient φ 1 . The condition |φ 1 | < 1 ensures that the process will come back to its mean, equal to 0. Conversely, for small values of |X t | (close to 0), the process behaves like an AR(1) model with coefficient φ 1 + φ 2 . We may even assume that φ 1 + φ 2 ≥ 1, which makes the process almost explosive in the short term until it reaches a level, which ensures that it will return to 0. This kind of process is able to describe bursting bubbles in finance (see [4]). It has been shown in [4] that most bootstrap methods including the Moving Block Bootstrap and the Sieve Bootstrap perform very badly for this kind of processes, in a case when the considered process exhibits a lot of bubbles. Model S4 is an interesting benchmark to study bootstrap methods in a dependent framework.
For S1-S4 we set σ 2 = 1. One can easily notice that the performance of the method deteriorates as σ 2 is growing. We choose φ = 0, 0.5, 0.9 and 0.99 to evaluate the effect of being more and more dependent and close to the unit root process. Additionally, for models S3 and S4, β = 0.5, 0.9 and φ 2 = 0.5.
Let the parameter θ be the correlation between X t and X t−1 . In our study we calculate the actual coverage probabilities of two-sided confidence intervals for θ (when the nominal coverage is 95%). For that purpose we apply the Whittle procedure assuming that the true spectral density has the form of an AR(1) model with parameter θ. Thus, for S1 and S2 we have θ = φ. For S3, we have θ = 0. For S4 the true value of θ is computed by Monte-Carlo simulation (over 50 000 simulations, the relative precision is of order 10 −3 ).
To have an idea of the accuracy of the FDB method in comparison to other bootstrap methods, we additionally perform all simulation using the Circular Block Bootstrap (CBB). Independently of the chosen method a clear difficulty is the choice of the tuning parameters. For the FDB, from the simulation, it appears that the choice of the smoothing parameter for estimating the original spectral density is critical. Indeed, we realized that when the process becomes more and more dependent the nonparametric estimator tends to be strongly biased resulting in a severe bias on the bootstrap distribution. This was also recently stressed in a simulation paper for the standardized mean by Kim and Im [21]. A fine analysis of simulations also shows that the usual nonparametric automatic choice of the smoothing parameter results in over-smoothing of the spectral density, especially when the process is close to nonstationarity (S1-S2 for ρ ≥ 0.9). Despite this bias the bootstrap distribution still recovers the correct variance of the statistics of interest. Thus, we decided to compare the performance of the standard percentile bootstrap confidence intervals (based on the α/2 and 1 − α/2 quantiles) with the bootstrap-t confidence intervals. We chose a more rough estimator of the spectral density estimator by taking The idea is to take a bandwidth smaller than n −1/5 with a constant roughly proportional to the variance of the noise, but which becomes small when the dependence increases. We tried to apply many bandwidths, including adaptive bandwidth choice (see bwadap in R based on [36], the program gives actually an incorrect choice for the bandwidth) and the chosen by us bandwidth provides satisfactory results. For the CBB the choice of the block size is also problematic. We use an automatic block length choice based on the method of [39] (see corrections in [34]). It should be noticed that in term of computation, the FDB in this setting is more efficient (almost 20 times faster), than the CBB, which requires the computation of a different spectral density estimators for each replication. Our simulation show (see Tables 1-2) that none of the considered bootstrap methods is a panacae. It suggests to use several methods to test the robustness of the procedures. First of all, we noticed that in most cases the bootstrapt confidence intervals for the FDB and the CBB have relatively good coverage probabilities, whereas the percentile confidence intervals may be quite inaccurate especially, when the dependence becomes stronger. It means that both methods (except for the FDB in model S3) are able to estimate correctly the variance of the statistics of interest. However, depending of the generating process an important bias (essentially due to the estimation of the spectral density) may jeopardize the confidence intervals. The percentile confidence intervals become totally irrelevant when one is close to the unit root (it is known that most bootstrap methods fail in this case). To explain this phenomenon one must recall that the true distribution of the estimator of θ is highly skewed when alpha becomes close to 1 and none of the methods is able to reproduce correctly this skewness. That can be clearly observed for the strongly skewed process S2. Interestingly, for the difficult process S4, the FDB provides better results than the CBB. In term of coverage we are close to the nominal level, whereas the CBB tends to be too conservative (due to too high estimates of the variance). Indeed, for most simulations, intervals lengths are quite similar for both bootstrap methods except for the model S4 for which the CBB provides wider intervals than the FDB. As was expected, the FDB totally fails for the ARCH process. It can be explained by the fact that the second order structure of the process is similar to the one of a white noise so that the FDB is unable to reproduce the dependence structure and hence fails to provide even the correct variance of the estimators. Note that the usual percentile CBB confidence interval completely fails for S4 model when the bubbles are very explosive (α 1 = 0.9, α 2 = 0.5). This can be explained by the fact that the CBB introduces a lot of artificial breaks in the reconstructed bootstrap time series.
Summarizing, before applying the FDB (and even the CBB) procedure for some parameters for which the FDB is supposed to work asymptotically (function of ratios or function of differentiable functionals with centered influence function) we recommend to verify if the considered times series does not present some conditionally heteroscedastic effects or is not too strongly skewed.

Summary and conclusions
In this paper we provided sufficient and necessary conditions for the consistency of the FDB in the case of linear stationary time series. For this purpose we defined the influence function in the time domain on spectral measures, which allowed us to linearize the functional of interest. Moreover, we introduced a notion of Fréchet differentiability of functionals of spectral measures. We discussed consistency of the FDB and its second order correctness for differentiable functionals of spectral density function. Finally, we stated sufficient and necessary conditions for the FDB validity in the case of empirical processes.
Our results allow to understand why the FDB is valid for some functionals (e.g., the Whittle estimator) or empirical processes and for which functionals or empirical processes it can be consistent. For instance only homogeneous functions of degree 0 of linear functionals (including ratios of linear functionals) are candidates for the asymptotic validity of the FDB when kurtosis of innovations is equal to zero. Moreover, we indicated that the second order correctness can be obtained only for linear time series with i.i.d. innovations such that Eζ 3 1 = 0. Thus, one should carefully apply the FDB method for particular functionals. Our simulation study shows that the FDB performs well for standard AR models but may have problems with ARCH models. Moreover, it outperforms the block bootstrap method in a case of an ETAR model.
Proof of Theorem 6.1. The proof follows standard arguments from the empirical process literature.
The marginal distribution converges obviously to the marginal distribution of the limit process by Theorem 3.1. The result concerning (Z n (h), h ∈ H) is a special case of those presented in [8] (with no tapering and for the univariate time series). Thus, essentially we have to prove the result for the bootstrap empirical process.
We are going to show that II and III are uniformly small of order o P (1), so that the limiting distribution is essentially given by I. But notice that this is simply a process with i.i.d. random variables and hence it is sufficient to verify for instance the assumptions of Theorem 2.11.9 (p. 211) in [44] to get the convergence to the Gaussian process given before. We have that H(λ jn )|ε * jn − 1|.
Since the ε * jn are i.i.d., we get that 2π Note that the first term on the right-hand side in the above expression is bounded by Thus, it follows that uniformly in h, we have that II + III = o P (1) along the sample.
To prove the conclusion of the theorem, now we investigate the behaviour of I. Note that the entropy condition is automatically satisfied under the stronger entropy condition (needed for Z n (f )) since we have by the Cauchy-Schwartz inequality The Lindeberg-Feller condition of Theorem 2.11.9 p. 211 in [44] is fulfilled because the moments of order 3 of |ε * jn − 1| are finite in probability along the sample. Thus, it is sufficient to verify an equicontinuity condition. h(λ jn ) − g(λ jn )) 2 f (λ jn 2 E * ε * jn − 1 2 . Note that E * (ε * jn − 1) 2 is converging to 1 in probability, it is bounded and as n −→ ∞ 2π n n0 j=1 (h(λ jn ) − g(λ jn )) 2 f (λ jn ) 2 −→ ρ 2 (f, g).
This convergence is uniform over the set H ηn ={h, g ∈ H, ρ 2 (h, g) ≤ η n }, because by the same arguments as in [44] p. 128, we have that N (δ, H ηn , ρ 2 ) ≤ 4N (δ/2, H, ρ 2 ) 2 < ∞, ensuring the validity of the Glivenko-Cantelli theorem over the class H ηn . It follows that for n large enough, there exists a constant C > 0 such that sup h,g∈ H,ρ2(h,g)≤δ E|Z * n (h) − Z * n (g)| 2 ≤ C δ n in probability, which converges to 0 when δ n −→ 0. Thus, Z * n (h) converges in (l ∞ (H), ||z|| H ) to a (different) centered Gaussian process with covariance given by the limit of covariance, for all h ∈ H, g ∈ H, The two limits of the empirical processes coincide iff the second term in the covariance c(h, g) vanishes that is iff k 4 π 0 h(ω)f (ω)dω = 0. Proof of Theorem 6.2. The proof follows the standard reasoning usually used in the literature on Hadamard differentiability. It is sufficient to apply the Hadamard differentiability property to the sequence h n = √ n( F n − F ), which converges to h = G in B(H, F), with t N = 1/ √ N −→ 0. We then have, as In the particular case when the influence function is non-degenerate and the variance is finite, we get To obtain the bootstrap convergence we use this Hadamard property again and the continuity of the differential operator. We have where G * F is a Gaussian process with covariance operator c 1 (h, g) = 2π π 0 h(ω)g(ω)f (ω) 2 dω. When the influence function is non-degenerate, this distribution becomes N 0, 2π π 0 T (1) (ω, F )T (1) (ω, F ) f (ω) 2 dω and again the bootstrap is valid in that case iff the influence function is centered.