Recurrence and transience of excited random walks on $\Z^d$ and strips

We investigate excited random walks on $\Z^d, d\ge 1,$ and on planar strips $\Z\times\{0,1,...,L-1\}$ which have a drift in a given direction. The strength of the drift may depend on a random i.i.d. environment and on the local time of the walk. We give exact criteria for recurrence and transience, thus generalizing results by Benjamini and Wilson for once-excited random walk on $\Z^d$ and by the author for multi-excited random walk on $\Z$.


Introduction
We consider excited random walks (ERWs), precisely to be defined below, which move on either Z d or strips, i.e. which have state space In general, ERWs are not Markovian. Instead, the transition probabilities may depend on how often the walk has previously visited its present location and additionally on the environment at this location.
To be more precise, let us first fix two quantities for the rest of the paper: A direction ℓ and the so-called ellipticity constant κ. In the case Y = Z or Y = Z × {0, 1, . . . , L − 1} we always choose ℓ = e 1 ∈ Y to be the first standard unit vector. In the case Y = Z d , d ≥ 2, we let ℓ ∈ R d be any direction with |ℓ| 1 = 1. The ellipticity constant κ ∈ (0, 1/(2d)] will be a uniform lower bound for the probability of the walk to jump from x to any nearest neighbor of x. Then an environment ω for an ERW is an element of ω(x, e, i) = 1, ω(x, e, i) e · ℓ ≥ 0 .
Here in the case of Y being a strip, d = 2 and x + e is modulo L in the second coordinate.
Besides this it is also shown in [BW03] that BW-ERW has positive liminf speed if d ≥ 4. Kozma extended this result to Z 3 in [K03] and very recently even to Z 2 in [K05]. Unfortunately, we were not able to adapt the technique of proof introduced in [BW03], which was also used in [K05], to the more general setting described above as we shall explain now.
Firstly, the proof in [BW03] relies on coupling the BW-ERW (X n ) n to a simple symmetric random walk (Y n ) n in such a way that 0 ≤ (X n − Y n ) · e 1 is non-decreasing in n and X n · e i = Y n · e i for all n and i ≥ 2. It is not clear to us how to achieve such a coupling if one allows the drift to point into a direction other than a coordinate direction e i , e.g. by letting ω(x, ±e j , 1) = 1/(2d) ± ε for j = 1, 2 and ω(x, e, i) = 1/(2d) for all other (x, e, i).
Secondly, the proof uses so-called tan points, which are points x to the right of which no other point has been visited prior to x. It is easy to see that any tan point for (Y n ) n is also a tan point for (X n ) n . Moreover, any tan point for (X n ) n will be a point with a cookie, which will push (X n ) n to the right. Then, roughly speaking, using a lower bound on the number of tan points for (Y n ) n , one gets a lower bound on the number of cookies consumed by (X n ) n , which Benjamini and Wilson show to be sufficient to ensure transience to the right. We do not see how this line of proof could be adapted to other settings, for instance if the excitement occurs not on the first but only on the second visit to a site.
In the present paper we suggest an alternative method of proof, based on martingales and on the environment viewed from the particle, which applies to BW-ERW and to other more general settings, in which ω is sampled from Ω according to a probability measure P on Ω such that the family Throughout the paper we will assume (1) and denote the expectation with respect to P by E. Note that we do not assume independence between different cookies at the same site nor between transition probabilities to different neighbors of the same site, but only between cookies at different sites. An important quantity will be the total drift δ x in direction ℓ of all the cookies stored at site x ∈ Y , i.e.
Note that by definition of Ω, δ x (ω) ≥ 0 for all x ∈ Y and ω ∈ Ω. We shall generalize Theorem A as follows.
The technique of proof improves methods used in [Ze05] to show the following result for d = 1. Some simulation studies for Y = Z can be found in [AR05].
In fact, [Ze05, Theorem 12] is more general since it does not need any ellipticity condition and allows the environment to be stationary and ergodic only instead of i.i.d.. In the present paper we shall generalize Theorem B to strips as follows.
So if the strip is made wider and wider while the distribution of ω(x, ·, ·) is kept fixed, the walk will eventually become transient if E[δ 0 ] > 0. This provides some additional support for Theorem 1.

Preliminaries
For z ∈ R, n ∈ N ∪ {∞} we let ω(x, e, i)e · ℓ denote the drift absorbed by the walk by time n while visiting the slab S z := {x ∈ Y | z ≤ x · ℓ < z + 1}. Then D n := z∈Z D z n is the total drift encountered by the walk up to time n. Observe that D z n ≥ 0 and therefore also D n ≥ 0 for all ω ∈ Ω and all paths (X m ) m .
By standard arguments, for any ω ∈ Ω the process (M n ) n≥0 defined by is a martingale under P 0,ω with respect to the filtration generated by (X n ) n≥0 . Indeed, (2) is just the Doob-Meyer decomposition of the submartingale (X n · ℓ) n .
In the setting considered in [BW03] and [Ze05] part of the following fact was achieved by coupling the ERW to a simple symmetric random walk staying always to the left of the ERW. For the present more general setting we need a different argument.
In particular, for all x ≥ 0, Proof. It follows from ellipticity and the Borel-Cantelli lemma that lim inf n X n · ℓ / ∈ R, which implies the first statement.
For the statement about lim sup, let x ≥ 0. Since D n ≥ 0 for all n, the martingale (M n∧Tx ) n is bounded from above by x and hence converges P 0,ωa.s. to a finite limit as n → ∞. Therefore, it suffices to show, that (M n ) n itself P 0,ω -a.s. does not converge, because then the convergence of (M n∧Tx ) n can only be due to T x being P 0,ω -a.s. finite.
However, this is impossible. Indeed, let e 0 ∈ Y be a unit vector which maximizes e 0 ·ℓ. Then due to ellipticity and the Borel-Cantelli lemma, |(X n+1 − X n ) · ℓ| = e 0 · ℓ infinitely often, whereas, again by ellipticity, for all n and some random i = i(n) ∈ N, Proof. By the Optional Stopping Theorem for all n ∈ N, 0 = E 0,ω [M Tx∧n ] and consequently by (2), The statement now follows from monotone convergence. Now we introduce some notation taken from [Ze05] for the cookie environment left over by the walk. For ω ∈ Ω and any finite sequence ( This is the environment created by the ERW by following the path (x n ) n≤m and removing all the first cookies encountered, except for the last visit to x m . Finiteness of T 1 , guaranteed by Lemma 3, implies that the Markov transition kernel Here θ z denotes the spatial shift of the environment by z, i.e. θ z (ω(x, ·, ·)) := ω(x + z, ·, ·). The probability measure R(ω, ·) is the distribution of the modified environment ω viewed from the particle at time T 1 . Note that it is supported on those countably many ω ′ ∈ Ω ℓ , which are obtained from ω by removing finitely many cookies from ω.
Lemma 5. R is weak Feller, i.e. convergence w.r.t. the product topology on Ω of ω n ∈ Ω towards ω ∈ Ω as n → ∞ implies for any bounded continuous function f : Ω → R.
Note that due to the discreteness of R(ω, ·) only countably many terms in the sums in (3) do not vanish.
Lemma 6. There is a probability measure P on Ω which is invariant under R and under which For the remainder of this paper we fix P according to Lemma 6 and let E be its expectation operator. We also introduce the annealed probability measures P 0 = P×P 0,ω and P 0 = P×P 0,ω with expectation operators E 0 and E 0 , respectively, which one gets by averaging the so-called quenched measure P 0,ω over E and E, respectively, i.e.
Consider the stopping times defined by τ 0 := 0 and τ n+1 := inf{n > τ n : X n · ℓ ≥ X τn · ℓ + 1} for n ≥ 0. Note that because in the second case, due to |ℓ| = 1, X τ n+1 · ℓ ≤ X τn · ℓ + 2. Since the slabs S Xτ n ·ℓ , n ≥ 0, are disjoint, we have D T K ≥ n≥0 D Xτ n ·ℓ T K for all K ≥ 0. Therefore, for all 0 ≤ k < K/2, where m = m(K, k) := K − k for Y being a strip or Z and m(K, k) := ⌊K/2⌋ − k for Y = Z d , d ≥ 2. Indeed, in both cases τ n+k ≤ T K for all n ≤ m due to (9). Consequently, by Lemma 4 and (10), By conditioning on the history up to time τ n and using the strong Markov property we get where R n denotes the n-th iteration of R and the last identity holds due to PR n = P. Consequently, we obtain from (11) Monotone convergence as k → ∞ then yields the claim.

Transience on Z d and strips
We denote by A ℓ := lim n→∞ X n · ℓ = +∞ and B ℓ := {∀n ≥ 1 X n · ℓ > X 0 · ℓ} the event that the walk tends to the right and the event that it stays forever strictly to the right of its initial point, respectively. As a preliminary result, we are now going to prove Theorem 1 with P instead of P.
Proof. On A c ℓ the walk changes sign P 0 -a.s. infinitely often due to Lemma 3. Therefore, because of ellipticity, on A c ℓ it also visits P 0 -a.s. infinitely many sites in the slab S 0 . Among these sites x there are P 0 -a.s. infinitely many ones with |e|=1,i≤I ω(x, e, i)e·ℓ > ε for some ε > 0 and some finite I due to the assumption of independence in the environment and E[δ 0 ] > 0. Again by ellipticity, on A c ℓ , P 0 -a.s. infinitely many of those sites will be visited at least I times. This yields that on A c ℓ , P 0 -a.s.  Proof. By assumption there is a finite nearest-neighbor path π 1 starting at 0 and ending at some a with a · ℓ > d such that with positive P 0,ω -probability the walk first follows π 1 and then stays to the right of a, while tending to the right, i.e. Figure 1. In particular, the second factor in (13) is positive. Now on A ℓ , the walk can visit sites on the path π 1 only finitely often. Therefore, there is another path π 2 of length m 2 entirely to the right of a which starts at a and ends at some b such that However, on the event that the walk never visits π 1 the walk does not feel whether it moves in the environment ψ(ω, (π 1 , π 2 )) or ψ(ω, π 2 ). Therefore, Since P a,ω [(X n ) n follows π 2 ] > 0 due to ellipticity, we get from this Now because of a · ℓ > d there is a nearest-neighbor path π 0 from 0 to a with 0 < x · ℓ < a · ℓ for all sites x on π 0 except for its starting and its end point. By ellipticity, the walk will follow π 0 with positive P 0,ω -probability. Therefore, due to (14) and since P a, by the strong Markov property.
We are now ready to prove a 0-1-law. We shall apply this result tō P ∈ {P, P}.
Proof. For short setP 0 =P × P 0,ω . Let us assumeP 0 [A ℓ ] > 0. We need to showP 0 [A ℓ ] = 1. By Lemma 9,P 0 [B ℓ ] > 0. The following argument is well-known, see e.g. [SzZe99, Lemma 1.1] and [ZeM01, Proposition 3]. Fix M ∈ N. We define recursively possibly infinite stopping times (S k ) k≥0 and (R k ) k≥0 by S 0 := T M , Due to Lemma 3 S 0 isP 0 -a.s. finite and any subsequent S k+1 isP 0 -a.s. finite as well provided R k is finite. Moreover, at each finite time S k the walk has reached a half space it has never touched before. The environment (ω(x + X S k , ·, ·)) x·ℓ≥0 in this half space is independent of the environment visited so far and has the same distribution as (ω(x, ·, ·)) x·ℓ≥0 . Hence the walk has probabilityP 0 [B ℓ ] never to leave this half space again. Therefore, by induction,P 0 [R k < ∞] ≤P 0 [B c ℓ ] k , which goes to 0 as k → ∞. Consequently, there is a random integer K with R K = ∞. This means that X n · ℓ ≥ M for all n ≥ S K . Since this holds for all M ,P 0 [A ℓ ] = 1.
The following is the counterpart of Lemma 8 for Z and strips. Proof. Assume that P 0 [A ℓ ] < 1. Then by Proposition 10, P 0 [A ℓ ] = 0. Therefore, the walk changes sign P 0 -a.s. infinitely often due to Lemma 3. However, if the walk crosses the finite set S 0 infinitely often then by ellipticity it will eventually eat all the cookies in S 0 , i.e. P 0 -a.s.

Recurrence on strips
Proof of recurrence in Theorem 2. Let LE[δ 0 ] ≤ 1. We need to show that P 0 -a.s. lim inf n X n · e 1 ≤ 0, since then, by ellipticity, X n = 0 infinitely often. Assume the contrary. Then by Lemma 3, P 0 [A ℓ ] > 0. Consequently, by Hence, by Proposition 10, Now let T −i := inf{n | X n · ℓ ≤ −i} for i > 0. Then we have by the Optional Stopping Theorem for all i, k, n ∈ N and all ω ∈ Ω, Using dominated convergence as n → ∞ for both terms in (16), we obtain Hence, due to (15), P-a.s. lim i→∞ lim k→∞ k −1 E 0,ω [D T k ∧T −i ] = 1. Splitting D n into D + n := k≥0 D k n and D − n := k<0 D k n then yields , which is P-a.s. finite, does not depend on k and thus vanishes when divided by k → ∞. However, by Lemma 4. Therefore, (17) implies By a calculation similar to the one in (12), Consequently, we can proceed like in the proof of [Ze05, Theorem 12] as follows and get Dominated convergence for K → ∞, justified by (18), and (19) then yield Now consider the event S := x∈S 0 δ x > ω(0, e 1 , 1) − ω(0, −e 1 , 1) that not all the drift contained in the slab S 0 is stored in the first cookie at 0. Observe that P[S] > 0. Indeed, for L ≥ 2 this follows from independence of the environment at different sites and for L = 1 the opposite would imply E[δ 0 ] ≤ 1 − κ, contradicting (20). Now according to (15) we have P-a.s. P 0,ω [A ℓ ] = 1. Therefore, by Lemma 9, P-a.s. P 0,ω [A ℓ ∩ B ℓ ] > 0. Hence, since P[S] > 0, as shown above, . Along with (20) this contradicts the assumption LE[δ 0 ] ≤ 1.
We conclude with some remarks, discussing the assumption of uniform ellipticity and some relation to branching processes with immigration.
2. It is well-known that recurrence of the simple symmetric random walk (Y n ) n on Z corresponds to extinction of the Galton-Watson process (Z m ) m with geometric(1/2) offspring distribution. Indeed, let the walk start at Y 0 = 1, set Z 0 = 1 and denote by Z m , m ≥ 1, the number of transitions of (Y n ) n from m to m + 1 before the walk hits 0. Since for ERW transitions to the right are more likely than for (Y n ) n , ERW can be viewed as a Galton-Watson process with immigration. Pakes [P71, Theorem 1] and Zubkov [Zu72,Theorem 3] showed that adding to each non-empty generation of a critical Galton-Watson process an i.i.d. number of immigrants makes it supercritical if the mean number of immigrants is above a certain critical threshold. This is reminiscent of Theorem B. However, since the immigration component of the Galton-Watson process derived from ERW is not independent these results do not directly translate into results for ERW.