SPECTRAL NORM OF CIRCULANT TYPE MATRICES WITH HEAVY TAILED ENTRIES

We ﬁrst study the probabilistic properties of the spectral norm of scaled eigenvalues of large dimensional Toeplitz, circulant and symmetric circulant matrices when the input sequence is independent and identically distributed with appropriate heavy tails. When the input sequence is a stationary two sided moving average process of inﬁnite order, we scale the eigenvalues by the spectral density at appropriate ordinates and study the limit for their maximums.


Introduction
Matrices with suitable patterned random inputs where the dimension tends to infinity are known as large dimensional random matrices. In this article we focus on the (symmetric) Toeplitz, circulant, reverse circulant and symmetric circulant matrices defined as follows: let {Z 0 , Z 1 , . . .} be a sequence of real random variables, which will be called the input sequence. (Symmetric) Toeplitz matrix T n . This n × n symmetric matrix with input {Z i } is the matrix with (i, j)-th entry Z |i− j| for all i, j. Toeplitz matrices appear as the covariance matrix of stationary processes, in shift-invariant linear filtering and in many aspects of combinatorics, time series and har-monic analysis. [Bai(1999)] proposed the study of the large Toeplitz matrix with independent inputs. For results in the non random situation we refer the reader to [Grenander and Szegő(2001)]. Circulant matrix C n . The first row is (Z 0 Z 1 . . . Z n−1 ) and the ( j + 1)-th row is obtained by giving its j-th row a right circular shift by one position. This is not a symmetric matrix and its (i, j)-th element is given by Z ( j−i+n)mod n . One of the usefulness of the circulant matrix is due to its deep connection to the Toeplitz matrix. The former has an explicit easy formula for its eigenvalues. The spectral analysis of the latter is much harder and challenging. If the input {Z l } l≥0 is square summable, then the circulant matrix approximates the corresponding Toeplitz matrix in several senses when the dimension grows. See [Gray (2006)] for a recent and relatively easy account. The circulant matrices are diagonalized by the Fourier matrix F = ((F s,t )), F s,t = e 2πist/n , 0 ≤ s, t < n. Their eigenvalues are the discrete Fourier transform of the input sequence {a l } 0≤l<n and are given by λ t = n−1 l=0 a l e −2πi t/n , 0 ≤ t < n. The eigenvalues of the circulant matrices arise crucially in time series analysis. For instance, the periodogram of a sequence {a l } l≥0 is defined as n −1 | n−1 l=0 a l e 2πi j/n | 2 , − n−1 2 ≤ j ≤ n−1 2 and is a straightforward function of the eigenvalues of the corresponding circulant matrix. The study of the properties of the periodogram is fundamental in the spectral analysis of time series. See for instance [Fan and Yao(2003)]. The maximum of the perdiogram, in particular, has been studied in [Davis and Mikosch(1999)]. Some recent developments on random circulant matrices are available in [Meckes (2009)] and [Bose et al.(2009a)]. Symmetric circulant matrix SC n . This matrix is a symmetric version of the usual circulant matrix. The first row (Z 0 Z 1 Z 2 . . . Z 2 Z 1 ) is a palindrome and the ( j + 1)-th row is obtained by giving its j-th row a right circular shift by one position. Its (i, j)-th element is given by Z n/2−|n/2−|i− j|| . When the input sequence is i.i.d. with positive variance, then it is no longer square summable. In that case, the spectral behaviour of the symmetric circulant and the symmetric Toeplitz are quite different. Compare for example, the limiting spectral distribution results of [Bose and Mitra(2002)] and [Massey et al.(2007)] for the symmetric circulant matrix, and of [Bryc, Dembo and Jiang (2006)] and [Hammond and Miller(2005)] for the Toeplitz matrix. On the other hand, consider the random symmetric band Toeplitz matrix, where the banding parameter m, which essentially is a measure of the number of nonzero entries, satisfies m → ∞ and m/n → 0. Then again its spectral distribution is approximated well by the corresponding banded symmetric circulant matrix. See for example [Kargin(2009)] and [Basak and Bose(2009)]. Reverse circulant matrix RC n . The first row of this matrix is (Z 0 Z 1 . . . Z n−1 ) and the ( j + 1)-th row is obtained by giving its j-th row a left circular shift by one position. This is a symmetric matrix and its (i, j)-th entry is given by Z (i+ j−2)mod n . This matrix arises in various applications of time series. The eigenvalue structure of this matrix is very closely related to the periodogram of the input sequence. The LSD of the reverse circulant was derived in [Bose and Mitra(2002)]. This has been used in the study of the symmetric band Hankel matrices. See [Basak and Bose(2009)] for details. Spectral norm. The spectral norm A of a matrix A with complex entries is the square root of the largest eigenvalue of the positive semidefinite matrix A * A: where A * denotes the conjugate transpose of A. Therefore if A is an n × n real symmetric matrix or A is a normal matrix then A = max 1≤i≤n |λ i | where λ 1 , λ 2 , . . . , λ n are the eigenvalues of A.
Existing limit results for the spectral norm. For the existing limit results on spectral norms of the Toeplitz and circulant matrices, see [Bose and Sen(2007)], [Meckes(2007)], [Bryc and Sethuraman(2009)], [Bose et al.(2009b)] and [Adamczak(2010)]. For spectral norm and radii of non-central random matrices see [Silverstein(1994)]. The maximum eigenvalues of the Wigner and the sample covariance matrix have been extensively studied, see [Bai and Yin (1988)] and [Yin et al.(1988)] for details. All these works are for the situation when the entries have finite moment of at least order two. [Soshnikov (2004)] shows the distributional convergence of the maximum eigenvalue of an appropriately scaled Wigner matrix with heavy tailed entries . A similar result was proved for the sample covariance matrices in [Soshnikov (2006)] with Cauchy entries. These results on the Wigner and the sample covariance matrices were extended in [Auffinger et al.(2009)] to the case 0 ≤ α ≤ 4. Our results. We focus on the above listed four matrices when the input sequence is heavy tailed, and 0 < α < 1. We establish the distributional convergence of the spectral norm of the three circulant matrices. Though we are unable to obtain the exact limit in the Toeplitz case, we provide upper and lower bounds. Our approach is to exploit the structure of the matrices and use existing methods on the study of maximum of periodograms for heavy tailed sequences. It seems to be a nontrivial problem to derive properties of the spectral norm in the case of moving average process inputs. We resort to scaling each eigenvalue by the power transfer function (defined in Section 3) at the appropriate ordinate as described below and then consider their maximum. For any of the above mentioned matrix A n , we define M(A n , f ) = max 1≤k≤n where f is the power transfer function corresponding to {x n } and {λ k } are the eigenvalues of A n . Similar scaling has been used in the study of periodograms (see [Davis and Mikosch(1999)], [Mikosch et al.(2000)], [Lin and Liu(2009)]). We show the distributional convergence of M(A n , f ) for the three circulant matrices. Any general result without the scaling seems difficult to derive without further assumptions. However in this setup the results are immediate from the results on the spectral norm of their i.i.d. counterparts.

Results for i.i.d. input
Notation and preliminaries. Let {Z t , t ∈ } be a sequence of i.i.d random variables with common distribution F where F is in the domain of attraction of an α-stable random variable with 0 < α < 1. Thus, there exist p, q ≥ 0 with p + q = 1 and a slowly varying function L(x), such that and µ real such that its characteristic function has the form For details on stable processes see [Samorodnitsky and Taqqu (1994)].
In the description of our results, we shall need the following: let {Γ j }, {U j } and {B j } be three independent sequences defined on the same probability space where {Γ j } is the arrival sequence of a unit rate poisson process on , U j are i.i.d U(0, 1) and B j are i.i.d. satisfying where p and q are as defined in (1). We also define For a nondecreasing function f on , let f ← ( y) = inf{s : f (s) > y}. Then the scaling sequence {b n } is defined as The reverse circulant and the circulant. The eigenvalues {λ k , 0 ≤ k ≤ n − 1} of b −1 n RC n are given by (see [Bose and Mitra(2002) where The eigenvalues of b −1 n C n are given by From the eigenvalue structure of C n and RC n , it is clear that b −1 n C n = b −1 n RC n and therefore they have identical limiting behavior which is stated in the following result.
(ii) for n even: with λ n−k = λ k in both cases. (3). Remark 1. (i) Theorem 1 and 2 are rather easy to derive when p = 1, that is, when the left tail is negligible compared to the right tail. Let us consider b −1 n RC n and note from the eigenvalue structure that, For the lower bound note that When α = 1, and {Z i } are non negative where Y α is a S 1 (2/π, 1, 0) random variable. Similar results hold for symmetric circulant matrices.
The Toeplitz. Resolving the question of the exact limit of the Toeplitz spectral norm seems to very difficult. Here we provide a good upper and lower bound in the distribution sense.
Theorem 3. Suppose that the input sequence is i.i.d. {Z t } satisfying (1). Then for α ∈ (0, 1) and γ > 0, Remark 2. The case when α ∈ [1, 2) and p = 1 and {Z i } are not necessarily non negative appears to be a non trivial problem. In the reverse circulant case we saw that the eigenvalue structure is similar to the square root of the periodogram and the maximum of the periodogram is not tight with the scaling b 1/α n when α ≥ 1 (even with input sequence as i.i.d. SαS random variables). Instead it is tight with a different scaling (see [Mikosch et al.(2000)], Section 3 for details).

Results for dependent inputs
Now suppose that the input sequence is a linear process {X t , t ∈ } given by Suppose that {Z i } are i.i.d random variables satisfying (1) with 0 < α < 1. Using E |Z| α−ε < ∞ and the assumption on the {a j } we have, Hence X t is finite a.s. Let be the transfer function of the linear filter {a j } and f X (x) be the power transfer function of {X t }. Then where in each case {λ k } are the eigenvalues of the corresponding matrix. From the eigenvalue structure of C n and RC n , M(C n , f X ) = M(RC n , f X ).

Proofs of the results
Some auxiliary results. The main idea of the proofs is taken from [Mikosch et al.(2000)] who show weak convergence of the maximum of the periodogram based on heavy tailed sequence for α < 1. Let ε x (·) denote the point measure which gives unit mass to any set containing x and let Suppose f is a bounded continuous complex valued function defined on and without loss of generality assume | f (x)| ≤ 1 for all x ∈ . Now pick η > 0 and define T η : M p (E) −→ C[0, ∞) as follows: The following Lemma was proved in [Mikosch et al.(2000)] (Lemma 2.3) with the function f (x) = exp(−i x). Same proof works in our case.
is continuous a.s. with respect to the distribution of N given in (8).
The proof of the following result is similar to the proof of Proposition 2.2 of [Mikosch et al.(2000)].
We briefly sketch the proof in our case.
Proof of Theorem 1. We use Lemma 1 and 2 with f (x) = exp(−i x). It is immediate that It is well known that Hence it remains to show that for γ > 0, Now observe that for any integer K and sufficiently large n, Now the theorem follows from Lemma 3 given below.
This lemma is similar to Lemma 2.4 of [Mikosch et al.(2000)] and hence we skip the proof.
Proof of Theorem 2. The proof is similar to the proof of Theorem 1. We provide the proof for n odd, and for n even the changes needed are minor. Define where q = q n = [ n 2 ]. Since b −1 n SC n − M n,Z → 0 almost surely, it is enough to show M n,Z ⇒ 2 1−1/α Y α . Note that (8) holds with [0, 1] replaced by [0, 1/2], and letting N n = q k=1 ε (k/n,Z k /b q ) , N = ∞ j=1 ε (U j ,B j Γ −1/α j ) and U j to be i.i.d. U[0, 1/2]. Now following the argument given in Lemma 2.3 of [Mikosch et al.(2000)], Lemma 2 and taking f (x) = cos x it is easy to establish that J n,Z (x/n) = 2b −1 n q k=1 Z k cos 2πkx It is obvious that It remains to show that for η > 0, Now following the arguments given to prove (12), we can establish this relation. This completes the proof of the theorem. (2007)], T n is a submatrix of the infinite Laurent matrix L n = Z | j−k| 1 | j−k|≤n−1 j,k∈ so T n ≤ L n , where L n denotes the operator norm of L n acting in the standard way on l 2 ( ).

Proof of Theorem 3. Following [Meckes
If we use the Fourier basis to identify l 2 ( ) with L 2 [0, 1], it turns out that L n corresponds to a multiplication operator with the multiplier Hence as n → ∞, By another argument of [Meckes(2007)], we get the following estimate where v x ∈ n is defined as (v x ) j = e 2πi x j for j = 1, 2, . . . , n and 〈·, ·〉 is the standard inner product on n . Therefore To find the limit in the last expression, pick η > 0 and define T η : M p (E) −→ C[0, ∞), as follows: Following the argument given in Lemma 2.3 of [Mikosch et al.(2000)], it is easy to see that T η is continuous a.s. with respect to the distribution of N and then using an argument from Lemma 2, we can show that for fixed x Now for any fixed T where n > T > 0, using (15) ( and hence lim inf Since this is true for any T , we obtain lim inf Now to identify the distribution of the random variable appearing in the right side of the inequality, we follow Lemma 2.4 of [Mikosch et al.(2000)]. Here we use the fact This completes the proof.

Remark 3.
The assumption of α < 1 is crucially used only in the lower bound argument. It is clear from the above proof that the upper bound can be derived when α ∈ (1, 2). Indeed, it easily follows that where Y α is as in Remark 1(ii).

Proof of Theorem 4(a).
The proof is along the lines of the proof of Lemma 2.6 in [Mikosch et al.(2000)].
Let C n be the circulant matrix formed with independent entries {Z i }. To prove the result it is enough to show that where Since f X is bounded away from 0 and (16) holds, it is enough to show that max 1≤k≤n |Y n (k/n)| → 0. Now j=−n a j exp(−i2πx j)V n, j = S 1 (x) + S 2 (x) + S 3 (x) + S 4 (x).
Now following an argument similar to that given in the proof of Lemma 2.6 in [Mikosch et al.(2000)], we can show that max 1≤k≤n |S i (k/n)| → 0 for i = 1, 2.
The behavior of S 3 (x) and S 4 (x) are similar to S 1 (x) and S 2 (x) respectively. Therefore, following similar argument we can show that max 1≤k≤n |S j (k/n)| → 0 for j = 3, 4. This completes the proof of part (a).

Proof of Theorem 4(b).
Let SC n be the symmetric circulant matrix formed with independent entries {Z i }. In view of Theorem 2, it is enough to show that M(b −1 n SC n , f X ) − b −1 n SC n → 0.
Let q = q n = [ n 2 ] and J n,Z (x) := 2b −1 n q t=1 Z t cos(2πx t) Then using a j = a − j we have Since f X is bounded away from 0, it is enough to show that sup 1≤k≤q J n,X (k/n) − ψ(k/n)J n,Z (k/n) ≤ sup 1≤k≤q Y 1n (k/n) + sup 1≤k≤q Y 2n (k/n) → 0. Again following an argument similar to that in the proof of Lemma 2.6 in [Mikosch et al.(2000)], we can show that sup 1≤k≤q S i (k/n) → 0 for 1 ≤ i ≤ 4. Hence sup 1≤k≤q Y 1n (k/n) → 0. Similarly sup 1≤k≤q Y 2n (k/n) → 0. This completes the proof of part (b).