The Annals of Applied Probability

Nonconvex homogenization for one-dimensional controlled random walks in random potential

Atilla Yilmaz and Ofer Zeitouni

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We consider a finite horizon stochastic optimal control problem for nearest-neighbor random walk $\{X_{i}\}$ on the set of integers. The cost function is the expectation of the exponential of the path sum of a random stationary and ergodic bounded potential plus $\theta X_{n}$. The random walk policies are measurable with respect to the random potential, and are adapted, with their drifts uniformly bounded in magnitude by a parameter $\delta\in[0,1]$. Under natural conditions on the potential, we prove that the normalized logarithm of the optimal cost function converges. The proof is constructive in the sense that we identify asymptotically optimal policies given the value of the parameter $\delta$, as well as the law of the potential. It relies on correctors from large deviation theory as opposed to arguments based on subadditivity which do not seem to work except when $\delta=0$.

The Bellman equation associated to this control problem is a second-order Hamilton–Jacobi (HJ) partial difference equation with a separable random Hamiltonian which is nonconvex in $\theta$ unless $\delta=0$. We prove that this equation homogenizes under linear initial data to a first-order HJ equation with a deterministic effective Hamiltonian. When $\delta=0$, the effective Hamiltonian is the tilted free energy of random walk in random potential and it is convex in $\theta$. In contrast, when $\delta=1$, the effective Hamiltonian is piecewise linear and nonconvex in $\theta$. Finally, when $\delta\in(0,1)$, the effective Hamiltonian is expressed completely in terms of the tilted free energy for the $\delta=0$ case and its convexity/nonconvexity in $\theta$ is characterized by a simple inequality involving $\delta$ and the magnitude of the potential, thereby marking two qualitatively distinct control regimes.

Article information

Ann. Appl. Probab., Volume 29, Number 1 (2019), 36-88.

Received: May 2017
Revised: January 2018
First available in Project Euclid: 5 December 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K37: Processes in random environments 93E20: Optimal stochastic control 35B27: Homogenization; equations in media with periodic structure [See also 74Qxx, 76M50]

Random walk in random potential stochastic optimal control Hamilton–Jacobi homogenization corrector large deviations tilted free energy


Yilmaz, Atilla; Zeitouni, Ofer. Nonconvex homogenization for one-dimensional controlled random walks in random potential. Ann. Appl. Probab. 29 (2019), no. 1, 36--88. doi:10.1214/18-AAP1395.

Export citation


  • [1] Alili, S. (1999). Asymptotic behaviour for random walks in random environments. J. Appl. Probab. 36 334–349.
  • [2] Armstrong, S. and Cardaliaguet, P. (2018). Stochastic homogenization of quasilinear Hamilton–Jacobi equations and geometric motions. J. Eur. Math. Soc. (JEMS) 20 797–864. Available at arXiv:1504.02045.
  • [3] Armstrong, S. N., Tran, H. V. and Yu, Y. (2015). Stochastic homogenization of a nonconvex Hamilton–Jacobi equation. Calc. Var. Partial Differential Equations 54 1507–1524.
  • [4] Armstrong, S. N., Tran, H. V. and Yu, Y. (2016). Stochastic homogenization of nonconvex Hamilton–Jacobi equations in one space dimension. J. Differential Equations 261 2702–2737.
  • [5] Artstein, Z. (1980). Discrete and continuous bang-bang and facial spaces or: Look for the extreme points. SIAM Rev. 22 172–185.
  • [6] Berger, N., Mukherjee, C. and Okamura, K. (2018). Quenched large deviations for simple random walks on percolation clusters including long-range correlations. Comm. Math. Phys. 358 633–673. Available at arXiv:1612.01393.
  • [7] Bertsekas, D. P. and Shreve, S. E. (1978). Stochastic Optimal Control: The Discrete Time Case. Mathematics in Science and Engineering 139. Academic Press, New York.
  • [8] Daley, D. J. and Vere-Jones, D. (2008). An Introduction to the Theory of Point Processes. Vol. II: General Theory and Structure, 2nd ed. Springer, New York.
  • [9] Davini, A. and Kosygina, E. (2017). Homogenization of viscous and non-viscous HJ equations: A remark and an application. Calc. Var. Partial Differential Equations 56 Art. 95.
  • [10] Dembo, A. and Zeitouni, O. (2010). Large Deviations Techniques and Applications. Stochastic Modelling and Applied Probability 38. Springer, Berlin. Corrected reprint of the second (1998) edition.
  • [11] Evans, L. C. (1989). The perturbed test function method for viscosity solutions of nonlinear PDE. Proc. Roy. Soc. Edinburgh Sect. A 111 359–375.
  • [12] Fehrman, B. J. A partial homogenization result for nonconvex viscous Hamilton–Jacobi equations. Preprint. Available at arXiv:1402.5191.
  • [13] Flury, M. (2007). Large deviations and phase transition for random walks in random nonnegative potentials. Stochastic Process. Appl. 117 596–612.
  • [14] Gao, H. (2016). Random homogenization of coercive Hamilton–Jacobi equations in 1d. Calc. Var. Partial Differential Equations 55 Art. 30.
  • [15] Grimmett, G. R. and Stirzaker, D. R. (2001). Probability and Random Processes, 3rd ed. Oxford Univ. Press, New York.
  • [16] Kosygina, E. (2007). Homogenization of stochastic Hamilton–Jacobi equations: Brief review of methods and applications. In Stochastic Analysis and Partial Differential Equations. Contemp. Math. 429 189–204. Amer. Math. Soc., Providence, RI.
  • [17] Kosygina, E., Rezakhanlou, F. and Varadhan, S. R. S. (2006). Stochastic homogenization of Hamilton–Jacobi–Bellman equations. Comm. Pure Appl. Math. 59 1489–1521.
  • [18] Kozlov, S. M. (1985). The averaging method and walks in inhomogeneous environments. Uspekhi Mat. Nauk 40 61–120, 238.
  • [19] Liggett, T. M. (1985). An improved subadditive ergodic theorem. Ann. Probab. 13 1279–1285.
  • [20] Lions, P.-L., Papanicolaou, G. and Varadhan, S. R. S. Homogenization of Hamilton–Jacobi equation. Unpublished manuscript, circa 1987.
  • [21] Lions, P.-L. and Souganidis, P. E. (2005). Homogenization of “viscous” Hamilton–Jacobi equations in stationary ergodic media. Comm. Partial Differential Equations 30 335–375.
  • [22] Rassoul-Agha, F. and Seppäläinen, T. (2014). Quenched point-to-point free energy for random walks in random potentials. Probab. Theory Related Fields 158 711–750.
  • [23] Rassoul-Agha, F., Seppäläinen, T. and Yilmaz, A. (2013). Quenched free energy and large deviations for random walks in random potentials. Comm. Pure Appl. Math. 66 202–244.
  • [24] Rassoul-Agha, F., Seppäläinen, T. and Yilmaz, A. (2017). Variational formulas and disorder regimes of random walks in random potentials. Bernoulli 23 405–431.
  • [25] Rosenbluth, J. M. (2006). Quenched Large Deviation for Multidimensional Random Walk in Random Environment: A Variational Formula. ProQuest LLC, Ann Arbor, MI. Ph.D. thesis, New York Univ.
  • [26] Sion, M. (1958). On general minimax theorems. Pacific J. Math. 8 171–176.
  • [27] Spitzer, F. (1976). Principles of Random Walk, 2nd ed. Graduate Texts in Mathematics 34. Springer, New York.
  • [28] Sznitman, A.-S. (1994). Shape theorem, Lyapounov exponents, and large deviations for Brownian motion in a Poissonian potential. Comm. Pure Appl. Math. 47 1655–1688.
  • [29] Varadhan, S. R. S. (2003). Large deviations for random walks in a random environment. Comm. Pure Appl. Math. 56 1222–1245.
  • [30] Yilmaz, A. (2009). Quenched large deviations for random walk in a random environment. Comm. Pure Appl. Math. 62 1033–1075.
  • [31] Zerner, M. P. W. (1998). Directional decay of the Green’s function for a random nonnegative potential on ${\mathbf{Z}}^{d}$. Ann. Appl. Probab. 8 246–280.
  • [32] Ziliotto, B. (2017). Stochastic homogenization of nonconvex Hamilton–Jacobi equations: A counterexample. Comm. Pure Appl. Math. 70 1798–1809.