Random walks with $k$-wise independent increments

We construct examples of a random walk with pairwise-independent steps which is almost-surely bounded, and for any $m$ and $k$ a random walk with $k$-wise independent steps which has no stationary distribution modulo $m$.


INTRODUCTION
Consider a simulation of a simple random walk on a graph. How will the simulation be affected if the source of randomness is not truly random but only pseudo random in the following specific sense, the random bits are kwise independent for some k > 1 and not independent as a family? The first question to ask is, does the pseudo walk converge to the same stationary measure? The most simple graph to consider might be a cycle of size m. This suggests the following problem: given a random walk S n = n i=1 X i , where the X i are random signs, plus or minus 1 with equal probability, and the X i 's are k-wise independent, what can be said about the behavior of the partial sums and in particular modulo some fixed number m? It turns out that there is a fundamental difference between the cases k = 2 and k > 2.
Examine first the case k = 2. For this case we shall show Theorem 1. There exists a sequence of random variables {X i } ∞ i=1 taking values ±1, pairwise independent, such that S n is bounded almost surely.
This result should be contrasted against the fact that we know that ES n = 0 and ES 2 n = n, just like in the completely independent case. In other words, S n for large n must have extreme "fat tail" behavior. Naturally, M := max n S n satisfies EM = ∞. The example we will demonstrate is essentially the Walsh system, which is pairwise independent. We will discuss this in section 2.
Such behavior cannot occur when k ≥ 4 for the simple reason that in this case we know that ES 4 n = 3n 2 − 2n and this gives We could not settle k = 3, Problem 1. Is there a sequence of random variables {X i } ∞ i=1 taking values ±1 with equal probability 3-wise independent, such that S n is bounded almost surely.
The higher k is, the more moments we know and we can approximate the large scale shape of S n . However, this does not necessarily mean we can say something about S n mod m. And indeed, in section 3 we show the following Theorem 2. Let m and k be some natural numbers, and let ǫ > 0. Then there exists a sequence of random variables {X i } ∞ i=1 taking values ±1, kwise independent, and a sequence I j such that Notice that the requirement that the condition holds only for some I j is unavoidable, since, say, if k ≥ 10m 2 then the distribution of S Ij +10m 2 is approximately uniform, since X Ij +1 , . . . , X Ij +10m 2 are independent.
Explicit constructions of k-wise independent 0-1 probability spaces and estimates on their sizes are the focus of several papers in combinatorics and complexity, see e.g. [3] for the first construction and [1] for a recent discussion with additional references. Sums of pairwise independent random variable were extensively studied, see e.g. [2] for some interesting examples. Thus there are already many interesting examples of pairwise independent processes. But it seems behavior modulo m was not studied.

PAIRWISE INDEPENDENT PROCESSES
We term the construction we use a "gray walk", as it is based on the well-known Gray code construction in combinatorics. The nth Gray code is a Hamiltonian path on the discrete n-cube, or a listing of all 2 n binary strings of length n, starting with the string 00...0, where every string appears exactly once, and any two consecutive strings differ in exactly one place. The construction (and hence also proof that this is possible) is done recursively, namely: To construct the nth Gray code, write down two copies of the (n − 1)th code, where the order of the strings in the second copy is reversed, and add to the first 2 n−1 strings a zero at the nth place, and to the second 2 n−1 strings a one at the nth place.
The Gray codes of all orders n can be combined into an infinite Gray code, by listing them sequentially for increasing n's, and converting all strings into infinite strings by padding with zeros. The first few strings in the infinite code are: To construct the gray walk, we now consider each string A i as specifying a finite subset (also denoted A i ) of the natural numbers N, where a 1 in the jth place signifies that j ∈ A i , and define where ξ 1 , ξ 2 , ... is a seed sequence of independent ±1 variables. It is easy to verify that the X i 's are pairwise independent. Therefore to finish theorem 1 we only need Proposition. The gray walk S n is bounded almost surely. More precisely Proof. For simplicity we add the element X 0 ≡ 1, and prove that for The recursive definition of the Gray code is reflected in the path: Assume we have defined the first steps S ′ 0 , S ′ 1 , S ′ 2 , ..., S ′ 2 j−1 −1 of the walk, then the next 2 j−1 steps will be the previous steps listed in reverse order, and multiplied by the random sign ξ j . This implies that the path up to time 2 j − 1, where j is the first value for which ξ j = −1, is 0, 1, 2, 3, 4, ..., 2 j−1 , 2 j−1 − 1, 2 j−1 − 2, ..., 3, 2, 1, 0, and all the subsequent values are bounded between −2 j and 2 j .
Remark. Theorem 1 holds also if one takes the strings A i in lexicographic order, which is simpler and is also the standard ordering of the Walsh system. However, we believe the gray walk is interesting in its own right. For example, it satisfies that Xi+1 Xi = ξ j(i) , i.e. the seed sequence is exposed directly in the quotients of the X i 's.
On the other hand, there is a vast body of knowledge about the behavior of "weighted walks" with the lexicographical order, which is simply the question when ∞ i=0 c i W i converges where W i is the Walsh system. The general philosophy is that it is similar to the behavior of the Fourier system, with the exception of the symmetry of the Fourier system which has no equivalent in the Walsh system. Of course, this is very different from the simple behavior of independent variables. See [5] for a survey, or the book [4].

PROOF OF THEOREM 2
Before embarking on the general proof, let us demonstrate the case m = 4 which is far simpler. Let L > k satisfy be a sequence of i.i.d ±1 variables. Next define new variables by conditioning the ξ i -s: The fact that L > k clearly shows that the X i 's are k-independent. The fact that ξ aL+b = 1 shows that in each block of size L the number of −1's is even, and since 4|L we get that ξ aL+b ≡ 0 mod 4 for every block. Therefore if we define I j = jL then the condition in (1) is actually satisfied combinatorially and not only in high probability.
How can we generalize this to m = 4? The remaining ingredient in the proof is based on the following well known fact: It is possible to construct variables with probability 1 3 , 1 3 , 1 3 given a sequence of independent variables with probabilities 1 2 , 1 2 .
The algorithm is simple: take two such variables: if they give 11, "output" 1, if they give 1, −1 "output" 2 and if they give −1, 1, "output" 3. If they give −1, −1, take a new set of two variables and repeat the process. The output is 1, 2 or 3, each with probability 1 3 . The time required for each step is unbounded but the probability that it is large is exponentially small.
The proof below combines these two ideas, which we nickname "generating hidden symmetries" and "simulating uniform variables" to get the result.
Proof of theorem 2, general m. We may assume without loss of generality that m is even (otherwise take 2m instead). Let λ = λ(k, ǫ) be some sufficiently large parameter which we shall fix later. Let L := 2(k + 1) λm 2 where ⌈·⌉ stands, as usual, for the upper integer value.
is a sequence of k-wise independent variables taking values ±1 with probabilities 1 2 .
Properties 3 and 4 are obvious from the construction. Property 1 is a direct consequence of the estimate P(Ξ i ∈ S) ≤ Ce −cλ , if only λ is large enough. Define λ so as to satisfy this condition. Therefore we need only prove property 2.
Let therefore i 1 < · · · < i k and δ 1 , . . . , δ k ∈ {±1} and examine the events We know that P(X ) = 2 −k and we need to show that P(X |Y) = 2 −k . Let b be some number and let σ ⊂ {1, . . . , b − 1} be a set with #σ = b − (k + 1) and examine the event The point of the proof is that the event Y is independent of X ∩ S. This is because X depends only on k places, but Y depends on k + 1 Ξ i 's each of which is distributed uniformly. In other words Returning to the proof of the theorem, we let {Y µ,ν , L µ,ν } (µ,ν)∈{0,2,...,m−2}×N be an independent family of couples of variables constructed using the lemma. We now construct our sequence X i inductively as follows: assume that at the νth step we have defined X 1 , . . . , X ρ where ρ = ρ(ν) is random. Define µ = µ(ν) ≡ − ρ i=1 X i mod m and then define X ρ+1 , . . . , X ρ+L µ,ν using X ρ+i = y µ,ν i and ρ(ν + 1) = ρ(ν) + L µ,ν . This creates a sequence of ±1 variables. To see (1), define r(a) = max{ν : ρ(ν) < aL} and properties 1 and 4 of the lemma will give that Therefore we need only show that the X i 's are k-wise independent. While being strongly related to the k-wise independence of the y µ,ν i it is not an immediate consequence of it because of the dependence between the y µ,ν i and the L µ,ν 's.
We prove that the X i 's are k-wise independent inductively. Let l and I be some integers and assume that we have shown that X i1 , . . . , X im are independent for every m < l and for m = l and i 1 < I. Let I = i 1 < i 2 < · · · < i l and let δ 1 , . . . , δ l ∈ {±1}. Examine L 0,1 and define n = n(L 0,1 ) = #{m : i m ≤ L 0,1 } .

Remarks.
(1) It is easy to generalize the theorem to force Ij i=1 ǫ i to have almost arbitrary distributions. The only obstacle is the evenodd symmetry of the walk. Thus if m is odd, any distribution whatsoever on the congruence classes modulus m might be achieved, while if m is even, any distribution supported on the set of even or odd congruence classes can be achieved.
(2) It is easy to see from the proof that L ≈ km 2 log(k/ǫ). In other words, the precision is asymptotically exponential in the stepping of the subsequence I j . We remind the reader a fact that was discussed in the introduction: if k > m 2 log 1/ǫ then necessarily I j+1 − I j > cm 2 log 1/ǫ because immediately after I j we get a sequence of independent variables. (3) Another direction in which k-wise independent random variables may be very different from truly independent variables is their entropy. It is known that the entropy of n such variables may be as low as k log n, see [3] and [1]. Our pairwise example is optimal in this respect: n truly random bits create 2 n pairwise independent random variables. The example above is far from it: the entropy is linear in the number of bits. However, it turns out that it is possible to create an example of k-wise independent variables with low entropy satisfying the conditions of theorem 2. The example, while amusing, if a bit off topic hence we will skip it. The proof is similar to that of theorem 2, but using a different L in each step. However there are some additional technical subtleties. Essentially, if in the main lemma of the previous theorem we defined a variable Y µ where the parameter µ was used to "return to sync" the series if ever the congruence is no longer 0, here we would need variables Y µ,τ where τ is some additional parameter needed to "return to sync" in the L domain. While being only moderately more complicated, we felt it better to present the simpler proof of the previous theorem.

FUR THER RELATED PROBLEMS
We end by suggesting further problems related to k-wise independent random variables.

2D Percolation.
What about other functions of k-wise independent random variables? For example, instead of considering Bernoulli percolation, the random variables determining the configuration are only kwise independent. Here is an "unimpressive" example: k-wise independent bond percolation on an n × n(k + 1)box in Z 2 such that the probability to have a crossing of the narrow side is 1. Of course, even in usual percolation the probability 1 is 1 − e −ck so that the improvement is not exciting.
The construction is as follows: divide into k + 1 disjoint boxes of size n × n. Take the usual independent percolation, conditioned such that the number of boxes with crossings is odd. This is like conditioning k + 1 Bernoulli variables to have an odd sum, so that you get a k-wise independent structure. If the number is odd, then it is non zero. Can it be improved, say to guarantee crossing of the n × n-box?