Journal URL

Limiting spectral distribution (LSD) of scaled eigenvalues of circulant, symmetric circulant and a class of k-circulant matrices are known when the input sequence is independent and identically distributed with finite moments of suitable order. We derive the LSD of these matrices when the input sequence is a stationary, two sided moving average process of infinite order. The limits are suitable mixtures of normal, symmetric square root of the chisquare, and other mixture distributions, with the spectral density of the process involved in the mixtures.


Introduction
Suppose λ 1 , λ 2 , ..., λ k are all the eigenvalues of a square matrix A of order k.Then the empirical spectral distribution (ESD) of A is defined as where for any z ∈ , (z), (z) denote the real and imaginary part of z respectively.Let {A n } ∞ n=1 be a sequence of square matrices (with growing dimension) with the corresponding ESD {F A n } ∞ n=1 .The limiting spectral distribution (or measure) (LSD) of the sequence is defined as the weak limit of the sequence {F A n } ∞ n=1 , if it exists.Suppose elements of {A n } are defined on some probability space (Ω, , ), that is {A n } are random.Then {F A n (•)} are random and are functions of ω ∈ Ω but we suppress this dependence.Let F be a nonrandom distribution function.We say the ESD of A n converges to F in L 2 if at all continuity points (x, y) of F , Ω F A n (x, y) − F (x, y) 2 d (ω) → 0 as n → ∞. (1.1) If the eigenvalues are real then it is understood that {F A n } and F are functions of one variable.It may be noted that (1.1) holds if [F A n (t)] → F (t) and V [F A n (t)] → 0 at all continuity points t of F .We often write F n for F A n when the matrix under consideration is clear from the context.
For detailed information on existence and identification of LSD in different contexts, see Bai [1999] and also Bose and Sen [2008].There are many universality results available in this context.However, most of the existing work on LSD assumes the input sequence {x i } to be independent.With the current methods used to establish LSD, such as the moment method or the Stietljes transform method, it does not appear to be easy to extend the known results on LSD to general dependent situations for all types of matrices.There are very few works dealing with dependent inputs.For instance, Bose and Sen [2008] establishes LSD for some specific type of dependent entries for a few matrices and Bai and Zhou [2008] establishes LSD of large sample covariance matrices with AR(1) entries.
We focus on the LSD of circulant type matrices-the circulant, the symmetric circulant, the reverse circulant and the so called k-circulant for suitable values of k.We assume that {x i } is a stationary linear process.Stationary linear process is an important class of dependent sequence.For instance the widely used stationary time series models such as AR, MA, ARMA are all linear processes.Under very modest conditions on the process, we are able to establish LSD for these matrices.These LSD are functions of the spectral density of the process and hence are universal.Consider the following condition.
We first describe the different matrices that we deal with.These may be divided into four classes: (i) The circulant matrix is given by x 0 x 1 . . .x n−3 x n−2 x n−2 x n−1 x 0 . . .x n−4 x n−3 . . .
Sen [2006] shows that if Assumption A is satisfied then the ESD of C n converges in L 2 to the two-dimensional normal distribution given by N(0, D) where D is a 2 × 2 diagonal matrix with diagonal entries 1/2.Meckes [2009] shows similar type of result for independent complex entries.
In particular, if (x j ) = 0, |x j | 2 = 1 and lim n→∞ 1 for every ε > 0, then the ESD converges in L 2 to the standard complex normal distribution.
(ii) The symmetric circulant matrix is defined as Bose and Mitra [2002] show that if {x i } satisfies Assumption A, then the ESD of SC n converges weakly in L 2 to the standard normal distribution.
The palindromic Toeplitz matrix is the palindromic version of the usual symmetric Toeplitz matrix.It is defined as (see Massey et.al. [2007]), Its behavior is closely related to the symmetric circulant matrix.It may be noted that the n×n principal minor of P T n+1 is SC n . Massey et.al. [2007] establish the Gaussian limit for F P T n .Bose and Sen [2008] show that if the input sequence {x i } is independent with mean zero and variance 1 and are either (i) uniformly bounded or (ii) identically distributed, then the LSD of P T n is standard Gaussian.They also observe that if the LSD of any one of P T n and SC n exist, then the other also exists and they are equal.
(iii) The reverse circulant matrix is given by Bose and Mitra [2002] show that if {x i } satisfies Assumption A then the ESD of RC n converges weakly in L 2 to F , which is the symmetric square root of the chisquare with two degrees of freedom, having density (1.2) (iv) For positive integers k and n, the n × n k-circulant matrix is defined as We emphasize that all subscripts appearing above are calculated modulo n.For 1 ≤ j < n − 1, its ( j + 1)-th row is obtained by giving its j-th row a right circular shift by k positions (equivalently, k mod n positions).We have dropped the n −1/2 factor from the definition of the matrix so that subsequent formulae for eigenvalues remain simple.Establishing the LSD for general k-circulant matrices appears to be a difficult problem.Bose, Mitra and Sen [2008] show that if {x i } are i.i.d N (0, 1), k = n o(1) (≥ 2) and gcd(k, n) = 1 then the LSD of F n −1/2 A k,n is degenerate at zero, in probability.They also derive the following LSD.Suppose {x i } are i.i.d.satisfying Assumption C given below.Let {E i } be i.i.d.E x p(1), U 1 be uniformly distributed over (2g)-th roots of unity, U 2 be uniformly distributed over the unit circle where {U i }, {E i } are mutually independent.Then as n → ∞, F n −1/2 A k,n converges weakly in probability to We investigate the existence of LSD of the above matrices under the following situation.
Assumption B {x n ; n ≥ 0} is a two sided moving average process Assumption C {ε i }, i ∈ are i.i.d.random variables with mean zero, variance one and |ε i | 2+δ < ∞ for some δ > 0.
We show that the LSD of the above matrices continue to exist in this dependent situation under appropriate conditions on the spectral density of the process.The LSD turn out to be appropriate mixtures of, the normal distribution, the "symmetric" square root of the chisquare distribution, and, some other related distributions.Quite expectedly, the spectral density of the process is involved in these mixtures.Our results also reduce to the results quoted above for the i.i.d.situation.
In Section 2 we describe the nature of the eigenvalues of the above matrices, describe the spectral density and set up notation.In Section 3 we state the main results and report some simulation which demonstrate our theoretical results.The main proofs are given in Section 4 and the proofs of some auxiliary Lemma are given in the Appendix.
Some of the results reported in this article have been reported in the (not to be published) technical reports Bose and Saha [2008a], Bose and Saha [2008b], Bose and Saha [2009].

Spectral density and related facts
Under Assumptions B and C, γ h = C ov x t+h , x t is finite and j∈ |γ j | < ∞.The spectral density function f of {x n } exists, is continuous, and is given by x t e −i tω k 2 , k = 0, 1, . . ., n − 1, (2.1) denote the periodogram of {x i } where ω k = 2πk/n are the Fourier frequencies.Let where a i 's are the moving average coefficients in the definition of x n .It is easy to see that and for g ≥ 2, The above functions will play a crucial role in the statements and proofs of the main results later.

Description of eigenvalues
We now describe the eigenvalues of the four classes of matrices.Let [x] be the largest integer less than or equal to x.
(2.4) (ii) Symmetric circulant matrix.The eigenvalues {λ i } of SC n are given by: (a) for n odd: (b) for n even: (iii) Palindromic Toeplitz matrix.As far as we know, there is no formula solution for the eigenvalues of the palindromic Toeplitz matrix.As pointed out already, since the n × n principal minor of P T n+1 is SC n , by interlacing inequality P T n and SC n have identical LSD.
(iv) Reverse circulant matrix.The eigenvalues are given in Bose and Mitra [2002]: (v) k-circulant matrix.The structure of its eigenvalues is available in Zhou (1996).A more detailed analysis and related properties of the eigenvalues, useful in the present context, have been developed in Section 2 of Bose, Mitra and Sen [2008].Let x l ν kl , 0 ≤ j < n. (2.5) For any positive integers k, n, let p 1 < p 2 < . . .< p c be all their common prime factors so that, Here α q , β q ≥ 1 and n ′ , k ′ , p q are pairwise relatively prime.For any positive integer s, let s = {0, 1, 2, . . ., s − 1}.Define the following sets For any set A, let |A| denote its cardinality.Let g x = |S(x)| and (2.6) We observe the following about the sets S(x).
As a consequence, the distinct sets from the collection {S(x) : 0 ≤ x < n ′ } forms a partition of n ′ .

Main results
For any Borel set B, Leb(B) will denote its Lebesgue measure in the appropriate dimension.

Lemma 1. (i) For fixed x, y, H C is a bounded continuous function in ω.
(ii) F C defined as follows is a proper distribution function.
continuous everywhere and can be expressed as The proof of the Lemma is easy and we omit it.The normality claim in (iii) follows by applying Cauchy Schwartz inequality to compare fourth moment and square of the variance and using the fact that for the normal distribution their ratio equals 3. We omit the details.

Theorem 1. Suppose Assumptions B and C hold. Then the ESD of C n converges in L
, and F C reduces to the bivariate normal distribution whose covariance matrix is diagonal with entries 1/2 each.This agrees with Theorem 15, page 57 of Sen [2006] who proved the result under Assumption A.

Symmetric Circulant Matrix
For x ∈ and ω ∈ [0, π] define, As before, we now have the following Lemma.We omit the proof.

Lemma 2. (i) For fixed x, H S is a bounded continuous function in ω and H
(ii) F S defined below is a proper distribution function and F S (x) + F S (−x) = 1.
is continuous everywhere and may be expressed as Further, F S is normal if and only if f is constant almost everywhere (Lebesgue).
(iv) If Leb(C ′ 0 ) = 0 then F S is discontinuous only at x = 0. Theorem 2. Suppose Assumptions B and C hold and Then the ESD of SC n converges in L 2 to F S given in (3.4)-(3.5).The same limit continues to hold for P T n .
(ii) It is easy to check that the variance, µ 2 and the fourth moment µ 4 of F S equal In the latter case, F S is standard normal.This agrees with Remark 2 of Bose and Mitra [2002] (under Assumption A).

Reverse circulant matrix
where G(x) = 1 − e −x for x > 0, is the standard exponential distribution function.
is continuous everywhere and can be expressed as Further, F R is the distribution of the symmetric version of the square root of chisquare variable with two degrees of freedom if and only if f is constant almost everywhere (Lebesgue). (iv The proof of the above lemma is omitted.

k-Circulant matrix
As mentioned before, it appears difficult to prove general results for all possible pairs (k, n).We investigate two subclasses of the k-circulant.

n
where (ii) F d defined below is a valid continuous distribution function.
The proof of lemma is omitted.
where {E i } are i.i.d. with distribution function F g given in (3.9) and U 1 is uniformly distributed over the (2g)th roots of unity, independent of the {E i }.
where {E i } are i.i.d.E x p(1), U 1 is as in Theorem 4 and independent of {E i }.This limit agrees with Theorem 3 of Bose, Mitra and Sen [2008].
Remark 5. Using the expression (2.8) for the characteristic polynomial, it is then not difficult to manufacture {k = k(n)} such that the LSD of n −1/2 A k,n has some positive mass at the origin.For example, suppose the sequences k and n satisfy k g = −1 + sn where g ≥ 1 is fixed and s = o(n 1/3 ).Fix primes p 1 , p 2 , . . ., p t and positive integers β 1 , β 2 , . . ., β t .Define mass at zero, and rest of the probability mass is distributed as where U 1 and {E i } are as in Theorem 4.
(ii) g defined below is a proper distribution function.
= 0 then g is continuous everywhere and may be expressed as where t = (t 1 , t 2 , . . ., t 2g−1 , t 2g ) and dt = d t i .Further g is multivariate (with independent components) if and only if f is constant almost everywhere (Lebesgue).
(iv) If Leb(C 0 ) = 0 then g is discontinuous only on D g = {(z i , w i , i = 1, . . ., g) : The proof of lemma is omitted.1), U 2 is uniformly distributed over the unit circle independent of {E i }.This agrees with Theorem 4 of Bose, Mitra and Sen [2008].

Simulations
To demonstrate the limits we did some modest simulations with MA(1) and MA(2) processes.We performed numerical integration to obtain the LSD.In case of k-circulant (n = k 2 + 1), we have plotted the density of F 2 defined in (3.9).(ii) (right) dashed line represents the LSD of symmetric circulant matrix with entries x t = 0.3ε t + ε t+1 + 0.5ε t+2 where {ε i } i.i.d.N (0, 1) and the continuous line represents the kernel density estimate of the ESD of the same matrix of order 5000 × 5000 and same {x t }.where {ε i } i.i.d.N (0, 1).The continuous line represents the kernel density estimate of ESD of the same matrix of order 5000 × 5000 with same {x t }. (ii) same graphs with centered and scaled Bernoulli(1, 0.5).

Proofs of main results
Throughout c and C will denote generic constants depending only on d.We use the notation at all continuity points t of F .This is what we shall show in every case.
If the eigenvalues have the decomposition where y k → 0 in probability then {λ k } and {η k } have similar behavior.We make this precise in the following lemma.
Lemma 6. Suppose {λ n,k } 1≤k≤n is a triangular sequence of d -valued random variables such that Assume the following holds: Then,

lim n→∞
Proof.We define new random variables Λ n with P(Λ n = λ n,k ) = 1/n for k = 1, . . ., n.Then Similarly define by assumption (iii).Therefore Λ n and E n have the same limiting distribution.Now as n → ∞, and this is conclusion (i).To prove (ii) we use similar type of argument.Here we define new random variables Λn with P( Λn = (λ n,k , λ n,l )) = 1/n 2 for 1 ≤ k, l ≤ n.Similarly define Ẽn and Ỹn .Again Λn = Ẽn + Ỹn and So Λn and Ẽn will have same limiting distribution and hence conclusion (ii) holds.
We use normal approximation heavily in our proofs.Lemma 7 is a fairly standard consequence of normal approximation and follows easily from Bhattacharya and Ranga Rao [1976] (Corollary 18.1, page 181 and Corollary 18.3, page 184).We omit its proof.Part (i) will be used in Section 4.1-4.4 and Part (ii) will be used in Section 4.4.
Lemma 7. Let X 1 , . . ., X k be independent random vectors with values in d , having zero means and an average positive-definite covariance matrix where Φ d is the standard d dimensional normal distribution function, is the class of all Borel measurable convex subsets of d , (2+δ) and η = Cρ 2+δ n −δ/2 .

Proof of Theorem 1
The proof for circulant matrix mainly depends on Lemma 7 which helps to use normal approximation and, Lemma 8 given below which allows us to approximate the eigenvalues by appropriate partial sums of independent random variables.The latter follows easily from Fan and Yao [2003] (Theorem 2.14(ii), page 63).We have provided a proof in Appendix for completeness.For Lemma 8. Suppose Assumption B holds and {ε t } are i.i.d random variables with mean 0, variance 1.
Proof of Theorem 1: We first assume Leb(C 0 ) = 0. Note that we may ignore the eigenvalue λ n and also λ n/2 whenever n is even since they contribute atmost 2/n to the ESD F n (x, y).So for x, y ∈ , where Y n (ω k ) are same as defined in Lemma 8. Then To show this, define for 1 where for 1 So to show (4.6), it is enough to show as n → ∞, Along the lines of the proof used to show (4.5) one may now extend the vectors of two coordinates defined above to ones with four coordinates and proceed exactly as above to verify this.We omit the routine details.This completes the proof for the case Leb(C 0 ) = 0.When Leb(C 0 ) = 0, we have to show (4.1)only on D c 1 (of Lemma 1).All the above steps in the proof will go through for all (x, y) in D c 1 .Hence if Leb(C 0 ) = 0, we have our required LSD.This completes the proof of Theorem 1.

Proof of Theorem 2
For convenience, we prove the result for symmetric circulant matrix only for odd n = 2m + 1.The even case follows by appropriate easy changes in the proof.The partial sum approximation now takes the following form.For the interested reader, we provide a proof in the Appendix.

Proof of Theorem 2:
As before, we provide the detailed proof only when Leb(C ′ 0 ) = 0. Note that we may ignore the eigenvalue λ 0 since it contributes 1/n to the ESD F n (•).Further the term x 0 n can be ignored from the eigenvalue {λ k }.So for x ∈ , Following the argument given in circulant case and using Lemma 6 and Lemma 9 it is sufficient to show that 2 where Note that where Let α k be the minimum eigenvalue of V k .Then α k ≥ α m for 1 ≤ k ≤ m and Since {X l,k } satisfies (4.7) and C k is a convex set in 2 , we can apply Part (i) of Lemma 7 for where Φ 0,V k is a bivariate normal distribution with mean zero and covariance matrix V k .Note that for large m, where On the other hand, for every n, Therefore, by first letting n → ∞ and then letting p → 1, (4.8) holds.Hence Rest of the argument in the proof for symmetric circulant is same as in the proof of Theorem 1.
To prove the result for P T n , we use Cauchy's interlacing inequality (see Bhatia, 1997, page 59): As a consequence We have already observed that the n × n principal minor of P T n+1 is SC n .The result for P T n follows immediately.

Proof of Theorem 3
For reverse circulant we need the following Lemma.Its proof is given in Fan and Yao [2003] (Theorem 2.14(ii), page 63).
Lemma 10.Suppose Assumption B holds and {ε t } are i.i.d random variables with mean 0, variance and then we have as n → ∞, max 1≤k≤ Proof of Theorem 3: As earlier, we give the proof only for the case Leb(C ′ 0 ) = 0. From the structure of the eigenvalues, the LSD, if it exists, is going to be that of a symmetric distribution.So, it is enough to concentrate on the case x > 0. As before we may ignore the two eigenvalues λ 0 and λ n/2 .Hence for x > 0, Along the same lines as in the proof of Theorem 1, using Lemma 6 and Lemma 10 it is sufficient to show 1 n where Hence for x ≥ 0, Now rest of the argument in the proof is same as in the proof of Theorem 1.

Proofs for Theorems 4 and 5
The proofs of the two theorems for k circulant matrices draw substantially from the method of Bose and Mitra [2002] and uses the eigenvalue description given earlier.

n
Proof of Theorem 4: For simplicity we first prove the result when g = 2.Note that gcd(k, n) = 1 and hence in this case n ′ = n in (2.8).Thus the index of each eigenvalue belongs to exactly one of the sets l in the eigenvalue partition of {0, 1, 2, ..., n− 1}.Recall that v k,n is the total number of eigenvalues γ j of A k,n such that j ∈ l and | l | < g 1 .In view of Lemma 7 of Bose, Mitra and Sen [2008], we have v k,n /n ≤ 2/n → 0 and hence these eigenvalues do not contribute to the LSD.Hence it remains to consider only the eigenvalues corresponding to the sets l which have size exactly equal to g 1 .
Let 1, ω, ω 2 , ω 3 be all the fourth roots of unity.Note that for every j, the eigenvalues of A k,n corresponding to the set j are: y 1/4 j ω 3 .Hence it suffices to consider only the modulus of eigenvalues y 1/4 j as j varies: if these have an LSD F , say, then the LSD of the whole sequence will be (r, θ ) in polar coordinates where r is distributed according to F and θ is distributed uniformly across all the fourth roots of unity and r and θ are independent.With this in mind and remembering the scaling n, we consider for x > 0, Since the set of λ values corresponding to any j is closed under conjugation, there exists a set Combining each λ j with its conjugate, we may write y j in the form, where {b t } and {c t } are given in (2.4).Note that for x > 0, Now our aim is to show We can write Now using Lemma 10 it is easy to see that for any ε > 0, max 1≤ j≤|I n | (|R n, j | > ε) → 0 as n → ∞.So in view of Lemma 6 it is enough to show To do this we will use normal approximation.Define Note that {X l, j } satisfies (4.2)-(4.4)and L n, j ≤ x 4 = n −1/2 n−1 l=1 X l, j ∈ A n, j .Now using Part (ii) of Lemma 7 and Lemma 4 of Bose, Mitra and Sen [2008], arguing as in the previous proofs, To identify the limit, recall the structure of the sets S(x), j , j and their properties.Since |I n |/n → 1/4, v k,n ≤ 2 and either S(x) = S(u) or S(x) ∩ S(u) = φ, we have Also for n = k 2 + 1 we can write {1, 2, . . ., n − 1} as {ak + b; 0 ≤ a ≤ k − 1, 1 ≤ b ≤ k} and using the construction of S(x) we have (except for at most two values of j) Recall that for fixed x, H 2 (ω, ω ′ , x) is uniformly continuous on [0, 2π] × [0, 2π], .Therefore given any positive number ρ we can choose N large enough such that for all n Finally using (4.10), (4.11) we have To show that V [F n (x)] → 0, since the variables involved are all bounded, it is enough to show that Along the lines of the proof used to show [F n (x)] → F 2 (x), one may now extend the vectors with 4 coordinates defined above to ones with 8 coordinates and proceed exactly as above to verify this.We omit the routine details.This proves the Theorem when g = 2.The above argument can be extended to cover the general (g > 2) case.We highlight only a few of the technicalities and omit the other details.We now need the following lemma.
Lemma 11.Given any ε, η > 0 there exist an N ∈ such that where L n (ω j ), R n (ω j ) are as defined in Lemma 10.
and iterating this argument, Combining all the above we get First term in the right side can be made smaller than η/2 by choosing M large enough and since max 1≤k≤n |R n (ω k )| → 0 as n → ∞, we can choose N ∈ such that the second term is less than η/2 for all n ≥ N , proving the lemma.
For general g ≥ 2, as before, n ′ = n, the index of each eigenvalue belongs to one of the sets l in the eigenvalue partition of {0, 1, 2, ..., n − 1} and v k,n /n → 0. Hence it remains to consider only the eigenvalues corresponding to the sets l which have size exactly equal to g 1 and it follows from the argument in the proof of Lemma 3(i) of Bose, Mitra and Sen [2008] that g 1 = 2g.We can now proceed as in g = 2 case.First we show (4.12) Now write y j n g as follows Using Lemma 11 it is easy show that for any ε > 0, max 1≤ j≤|I n | (|R n, j | > ε) → 0 as n → ∞.So, by Lemma 6, to show (4.12) it is sufficient to show that For this we use normal approximation as we did in g = 2 case and define Now using Part (ii) of Lemma 7 and Lemma 4 of Bose, Mitra and Sen [2008], we have So we can write the sets j explicitly using this decomposition of {1, 2, . . ., n − 1} as done in g = 2 case, that is, n = k 2 + 1 case.For example if for finitely many j, bounded by v k,n and they do not contribute to this limit).Using this fact and proceeding as before we conclude that the LSD is now F g (•), proving Theorem 4 completely.
4.4.2n = k g − 1 for some fixed g ≥ 2 Proof of Theorem 5. First we assume Leb(C 0 ) = 0. Note that gcd(k, n) = 1.Since k g = 1 + n = 1 mod n, we have g 1 |g.If g 1 < g, then g 1 ≤ g/α where α = 2 if g is even and α = 3 if g is odd.In either case, it is easy to check that Hence, g = g 1 .By Lemma 3(ii) of Bose, Mitra and Sen [2008] the total number of eigenvalues γ j of A k,n such that j ∈ l and | l | < g is asymptotically negligible.
Unlike the previous theorem, here the partition sets l are not necessarily self-conjugate.However, the number of indices l such that l is self-conjugate is asymptotically negligible compared to n.To show this, we need to bound the cardinality of the following set for 1 ≤ l < g: D l = {t ∈ {1, 2, . . ., n} : t k l = −t mod n} = {t ∈ {1, 2, . . ., n} : n|t(k l + 1)}.
Note that t 0 = n/ gcd(n, k l + 1) is the minimum element of D l and every other element is a multiple of t 0 .Thus |D l | ≤ n t 0 ≤ gcd(n, k l + 1).
Theorem 3. Suppose Assumptions B and C hold.Then the ESD of RC n converges in L 2 to F R given in (3.7)-(3.8).