Sequential Monte Carlo methods, also known as particle methods, are a popular set of techniques for approximating high-dimensional probability distributions and their normalizing constants. These methods have found numerous applications in statistics and related fields; for example, for inference in nonlinear non-Gaussian state space models, and in complex static models. Like many Monte Carlo sampling schemes, they rely on proposal distributions which crucially impact their performance. We introduce here a class of controlled sequential Monte Carlo algorithms, where the proposal distributions are determined by approximating the solution to an associated optimal control problem using an iterative scheme. This method builds upon a number of existing algorithms in econometrics, physics and statistics for inference in state space models, and generalizes these methods so as to accommodate complex static models. We provide a theoretical analysis concerning the fluctuation and stability of this methodology that also provides insight into the properties of related algorithms. We demonstrate significant gains over state-of-the-art methods at a fixed computational complexity on a variety of applications.
References
[1] Andrieu, C., Doucet, A. and Holenstein, R. (2010). Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 269–342. 1411.65020 10.1111/j.1467-9868.2009.00736.x[1] Andrieu, C., Doucet, A. and Holenstein, R. (2010). Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 269–342. 1411.65020 10.1111/j.1467-9868.2009.00736.x
[2] Bertsekas, D. P. and Tsitsiklis, J. N. (1996). Neuro-dynamic Programming. Athena Scientific, Belmont, MA. 0924.68163[2] Bertsekas, D. P. and Tsitsiklis, J. N. (1996). Neuro-dynamic Programming. Athena Scientific, Belmont, MA. 0924.68163
[3] Beskos, A., Jasra, A., Kantas, N. and Thiery, A. (2016). On the convergence of adaptive sequential Monte Carlo methods. Ann. Appl. Probab. 26 1111–1146. 1342.82127 10.1214/15-AAP1113 euclid.aoap/1458651829[3] Beskos, A., Jasra, A., Kantas, N. and Thiery, A. (2016). On the convergence of adaptive sequential Monte Carlo methods. Ann. Appl. Probab. 26 1111–1146. 1342.82127 10.1214/15-AAP1113 euclid.aoap/1458651829
[4] Bichteler, K. (2002). Stochastic Integration with Jumps. Encyclopedia of Mathematics and Its Applications 89. Cambridge Univ. Press, Cambridge. 1002.60001[4] Bichteler, K. (2002). Stochastic Integration with Jumps. Encyclopedia of Mathematics and Its Applications 89. Cambridge Univ. Press, Cambridge. 1002.60001
[5] Bresler, Y. (1986). Two-filter formula for discrete-time non-linear Bayesian smoothing. Internat. J. Control 43 629–641. 0586.93067 10.1080/00207178608933489[5] Bresler, Y. (1986). Two-filter formula for discrete-time non-linear Bayesian smoothing. Internat. J. Control 43 629–641. 0586.93067 10.1080/00207178608933489
[6] Briers, M., Doucet, A. and Maskell, S. (2010). Smoothing algorithms for state-space models. Ann. Inst. Statist. Math. 62 61–89. 1422.62297 10.1007/s10463-009-0236-2[6] Briers, M., Doucet, A. and Maskell, S. (2010). Smoothing algorithms for state-space models. Ann. Inst. Statist. Math. 62 61–89. 1422.62297 10.1007/s10463-009-0236-2
[8] Chopin, N. (2002). A sequential particle filter method for static models. Biometrika 89 539–552. 1036.62062 10.1093/biomet/89.3.539[8] Chopin, N. (2002). A sequential particle filter method for static models. Biometrika 89 539–552. 1036.62062 10.1093/biomet/89.3.539
[9] Chopin, N. (2004). Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference. Ann. Statist. 32 2385–2411. 1079.65006 10.1214/009053604000000698 euclid.aos/1107794873[9] Chopin, N. (2004). Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference. Ann. Statist. 32 2385–2411. 1079.65006 10.1214/009053604000000698 euclid.aos/1107794873
[10] Christensen, O. F., Roberts, G. O. and Rosenthal, J. S. (2005). Scaling limits for the transient phase of local Metropolis–Hastings algorithms. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 253–268. 1075.65012 10.1111/j.1467-9868.2005.00500.x[10] Christensen, O. F., Roberts, G. O. and Rosenthal, J. S. (2005). Scaling limits for the transient phase of local Metropolis–Hastings algorithms. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 253–268. 1075.65012 10.1111/j.1467-9868.2005.00500.x
[11] Del Moral, P. (2004). Feynman–Kac Formulae. Probability and Its Applications (New York). Springer, New York. 1130.60003[11] Del Moral, P. (2004). Feynman–Kac Formulae. Probability and Its Applications (New York). Springer, New York. 1130.60003
[12] Del Moral, P., Doucet, A. and Jasra, A. (2006). Sequential Monte Carlo samplers. J. R. Stat. Soc. Ser. B. Stat. Methodol. 68 411–436. 1105.62034 10.1111/j.1467-9868.2006.00553.x[12] Del Moral, P., Doucet, A. and Jasra, A. (2006). Sequential Monte Carlo samplers. J. R. Stat. Soc. Ser. B. Stat. Methodol. 68 411–436. 1105.62034 10.1111/j.1467-9868.2006.00553.x
[13] Del Moral, P. and Guionnet, A. (1999). Central limit theorem for nonlinear filtering and interacting particle systems. Ann. Appl. Probab. 9 275–297. 0938.60022 10.1214/aoap/1029962742 euclid.aoap/1029962742[13] Del Moral, P. and Guionnet, A. (1999). Central limit theorem for nonlinear filtering and interacting particle systems. Ann. Appl. Probab. 9 275–297. 0938.60022 10.1214/aoap/1029962742 euclid.aoap/1029962742
[14] Diaconis, P. and Freedman, D. (1999). Iterated random functions. SIAM Rev. 41 45–76. 0926.60056 10.1137/S0036144598338446[14] Diaconis, P. and Freedman, D. (1999). Iterated random functions. SIAM Rev. 41 45–76. 0926.60056 10.1137/S0036144598338446
[16] Doucet, A. and Johansen, A. M. (2011). A tutorial on particle filtering and smoothing: Fifteen years later. In The Oxford Handbook of Nonlinear Filtering (D. Crisan andB. L. Rozovsky, eds.) 656–704. Oxford Univ. Press, Oxford. 05919872[16] Doucet, A. and Johansen, A. M. (2011). A tutorial on particle filtering and smoothing: Fifteen years later. In The Oxford Handbook of Nonlinear Filtering (D. Crisan andB. L. Rozovsky, eds.) 656–704. Oxford Univ. Press, Oxford. 05919872
[17] Gelman, A. and Meng, X.-L. (1998). Simulating normalizing constants: From importance sampling to bridge sampling to path sampling. Statist. Sci. 13 163–185. 0966.65004 10.1214/ss/1028905934 euclid.ss/1028905934[17] Gelman, A. and Meng, X.-L. (1998). Simulating normalizing constants: From importance sampling to bridge sampling to path sampling. Statist. Sci. 13 163–185. 0966.65004 10.1214/ss/1028905934 euclid.ss/1028905934
[18] Gerber, M., Chopin, N. and Whiteley, N. (2019). Negative association, ordering and convergence of resampling methods. Ann. Statist. 47 2236–2260. 1429.62154 10.1214/18-AOS1746 euclid.aos/1558425644[18] Gerber, M., Chopin, N. and Whiteley, N. (2019). Negative association, ordering and convergence of resampling methods. Ann. Statist. 47 2236–2260. 1429.62154 10.1214/18-AOS1746 euclid.aos/1558425644
[19] Girolami, M. and Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 123–214. 1411.62071 10.1111/j.1467-9868.2010.00765.x[19] Girolami, M. and Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 123–214. 1411.62071 10.1111/j.1467-9868.2010.00765.x
[20] Gobet, E. (2016). Monte-Carlo Methods and Stochastic Processes: From Linear to Non-linear. CRC Press, Boca Raton, FL. 1359.65012[20] Gobet, E. (2016). Monte-Carlo Methods and Stochastic Processes: From Linear to Non-linear. CRC Press, Boca Raton, FL. 1359.65012
[21] Gordon, N. J., Salmond, D. and Smith, A. F. M. (1993). A novel approach to non-linear/non-Gaussian Bayesian state estimation. IEE Proc. Radar Signal Process. 140 107–113.[21] Gordon, N. J., Salmond, D. and Smith, A. F. M. (1993). A novel approach to non-linear/non-Gaussian Bayesian state estimation. IEE Proc. Radar Signal Process. 140 107–113.
[23] Gupta, A., Jain, R. and Glynn, P. (2018). A fixed point theorem for iterative random contraction operators over Banach spaces. Available at arXiv:1804.01195. 1804.01195[23] Gupta, A., Jain, R. and Glynn, P. (2018). A fixed point theorem for iterative random contraction operators over Banach spaces. Available at arXiv:1804.01195. 1804.01195
[24] Heng, J., Bishop, A. N, Deligiannidis, G. and Doucet, A. (2020). Supplement to “Controlled sequential Monte Carlo.” https://doi.org/10.1214/19-AOS1914SUPP.[24] Heng, J., Bishop, A. N, Deligiannidis, G. and Doucet, A. (2020). Supplement to “Controlled sequential Monte Carlo.” https://doi.org/10.1214/19-AOS1914SUPP.
[25] Higham, N. J. (1988). Computing a nearest symmetric positive semidefinite matrix. Linear Algebra Appl. 103 103–118. 0649.65026 10.1016/0024-3795(88)90223-6[25] Higham, N. J. (1988). Computing a nearest symmetric positive semidefinite matrix. Linear Algebra Appl. 103 103–118. 0649.65026 10.1016/0024-3795(88)90223-6
[26] Jacob, P. E., Murray, L. M. and Rubenthaler, S. (2015). Path storage in the particle filter. Stat. Comput. 25 487–496. 1332.62359 10.1007/s11222-013-9445-x[26] Jacob, P. E., Murray, L. M. and Rubenthaler, S. (2015). Path storage in the particle filter. Stat. Comput. 25 487–496. 1332.62359 10.1007/s11222-013-9445-x
[27] Jasra, A., Stephens, D. A., Doucet, A. and Tsagaris, T. (2011). Inference for Lévy-driven stochastic volatility models via adaptive sequential Monte Carlo. Scand. J. Stat. 38 1–22. 1246.91149 10.1111/j.1467-9469.2010.00723.x[27] Jasra, A., Stephens, D. A., Doucet, A. and Tsagaris, T. (2011). Inference for Lévy-driven stochastic volatility models via adaptive sequential Monte Carlo. Scand. J. Stat. 38 1–22. 1246.91149 10.1111/j.1467-9469.2010.00723.x
[28] Kappen, H. J. and Ruiz, H.-C. (2016). Adaptive importance sampling for control and inference. J. Stat. Phys. 162 1244–1266. 1338.93166 10.1007/s10955-016-1446-7[28] Kappen, H. J. and Ruiz, H.-C. (2016). Adaptive importance sampling for control and inference. J. Stat. Phys. 162 1244–1266. 1338.93166 10.1007/s10955-016-1446-7
[31] Lin, M., Chen, R. and Liu, J. S. (2013). Lookahead strategies for sequential Monte Carlo. Statist. Sci. 28 69–94. 1332.62144 10.1214/12-STS401 euclid.ss/1359468409[31] Lin, M., Chen, R. and Liu, J. S. (2013). Lookahead strategies for sequential Monte Carlo. Statist. Sci. 28 69–94. 1332.62144 10.1214/12-STS401 euclid.ss/1359468409
[32] Liu, J. S. (2001). Monte Carlo Strategies in Scientific Computing. Springer Series in Statistics. Springer, New York. 0991.65001[32] Liu, J. S. (2001). Monte Carlo Strategies in Scientific Computing. Springer Series in Statistics. Springer, New York. 0991.65001
[33] Liu, J. S. and Chen, R. (1998). Sequential Monte Carlo methods for dynamic systems. J. Amer. Statist. Assoc. 93 1032–1044. 1064.65500 10.1080/01621459.1998.10473765[33] Liu, J. S. and Chen, R. (1998). Sequential Monte Carlo methods for dynamic systems. J. Amer. Statist. Assoc. 93 1032–1044. 1064.65500 10.1080/01621459.1998.10473765
[35] Mattingly, J. C., Stuart, A. M. and Higham, D. J. (2002). Ergodicity for SDEs and approximations: Locally Lipschitz vector fields and degenerate noise. Stochastic Process. Appl. 101 185–232. 1075.60072 10.1016/S0304-4149(02)00150-3[35] Mattingly, J. C., Stuart, A. M. and Higham, D. J. (2002). Ergodicity for SDEs and approximations: Locally Lipschitz vector fields and degenerate noise. Stochastic Process. Appl. 101 185–232. 1075.60072 10.1016/S0304-4149(02)00150-3
[36] Milstein, G. N. and Tretyakov, M. V. (2004). Stochastic Numerics for Mathematical Physics. Scientific Computation. Springer, Berlin. 1085.60004[36] Milstein, G. N. and Tretyakov, M. V. (2004). Stochastic Numerics for Mathematical Physics. Scientific Computation. Springer, Berlin. 1085.60004
[37] Møller, J., Syversveen, A. R. and Waagepetersen, R. P. (1998). Log Gaussian Cox processes. Scand. J. Stat. 25 451–482. 0931.60038 10.1111/1467-9469.00115[37] Møller, J., Syversveen, A. R. and Waagepetersen, R. P. (1998). Log Gaussian Cox processes. Scand. J. Stat. 25 451–482. 0931.60038 10.1111/1467-9469.00115
[38] Murray, L. M., Singh, S., Jacob, P. E. and Lee, A. (2016). Anytime Monte Carlo. Available at arXiv:612.03319.[38] Murray, L. M., Singh, S., Jacob, P. E. and Lee, A. (2016). Anytime Monte Carlo. Available at arXiv:612.03319.
[40] Nemoto, T., Bouchet, F., Jack, R. L. and Lecomte, V. (2016). Population-dynamics method with a multicanonical feedback control. Phys. Rev. E 93 062123.[40] Nemoto, T., Bouchet, F., Jack, R. L. and Lecomte, V. (2016). Population-dynamics method with a multicanonical feedback control. Phys. Rev. E 93 062123.
[41] Pitt, M. K. and Shephard, N. (1999). Filtering via simulation: Auxiliary particle filters. J. Amer. Statist. Assoc. 94 590–599. 1072.62639 10.1080/01621459.1999.10474153[41] Pitt, M. K. and Shephard, N. (1999). Filtering via simulation: Auxiliary particle filters. J. Amer. Statist. Assoc. 94 590–599. 1072.62639 10.1080/01621459.1999.10474153
[42] Richard, J.-F. and Zhang, W. (2007). Efficient high-dimensional importance sampling. J. Econometrics 141 1385–1411. 1420.65005 10.1016/j.jeconom.2007.02.007[42] Richard, J.-F. and Zhang, W. (2007). Efficient high-dimensional importance sampling. J. Econometrics 141 1385–1411. 1420.65005 10.1016/j.jeconom.2007.02.007
[43] Roberts, G. O. and Stramer, O. (2002). Langevin diffusions and Metropolis–Hastings algorithms. Methodol. Comput. Appl. Probab. 4 337–357. 1033.65003 10.1023/A:1023562417138[43] Roberts, G. O. and Stramer, O. (2002). Langevin diffusions and Metropolis–Hastings algorithms. Methodol. Comput. Appl. Probab. 4 337–357. 1033.65003 10.1023/A:1023562417138
[44] Roberts, G. O. and Tweedie, R. L. (1996). Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli 2 341–363. 0870.60027 10.2307/3318418 euclid.bj/1178291835[44] Roberts, G. O. and Tweedie, R. L. (1996). Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli 2 341–363. 0870.60027 10.2307/3318418 euclid.bj/1178291835
[45] Ruiz, H.-C. and Kappen, H. J. (2017). Particle smoothing for hidden diffusion processes: Adaptive path integral smoother. IEEE Trans. Signal Process. 65 3191–3203. 1414.94525 10.1109/TSP.2017.2686340[45] Ruiz, H.-C. and Kappen, H. J. (2017). Particle smoothing for hidden diffusion processes: Adaptive path integral smoother. IEEE Trans. Signal Process. 65 3191–3203. 1414.94525 10.1109/TSP.2017.2686340
[47] Scharth, M. and Kohn, R. (2016). Particle efficient importance sampling. J. Econometrics 190 133–147. 1419.62247 10.1016/j.jeconom.2015.03.047[47] Scharth, M. and Kohn, R. (2016). Particle efficient importance sampling. J. Econometrics 190 133–147. 1419.62247 10.1016/j.jeconom.2015.03.047
[48] Temereanca, S., Brown, E. N. and Simons, D. J. (2008). Rapid changes in thalamic firing synchrony during repetitive whisker stimulation. J. Neurosci. 28 11153–11164.[48] Temereanca, S., Brown, E. N. and Simons, D. J. (2008). Rapid changes in thalamic firing synchrony during repetitive whisker stimulation. J. Neurosci. 28 11153–11164.
[49] Theodorou, E. A. and Todorov, E. (2012). Relative entropy and free energy dualities: Connections to path integral and KL control. In Proceedings 51st IEEE Conference on Decision and Control (CDC), 1466–1473. Oxford Univ. Press, Oxford.[49] Theodorou, E. A. and Todorov, E. (2012). Relative entropy and free energy dualities: Connections to path integral and KL control. In Proceedings 51st IEEE Conference on Decision and Control (CDC), 1466–1473. Oxford Univ. Press, Oxford.
[51] Zhou, Y., Johansen, A. M. and Aston, J. A. D. (2016). Toward automatic model comparison: An adaptive sequential Monte Carlo approach. J. Comput. Graph. Statist. 25 701–726.[51] Zhou, Y., Johansen, A. M. and Aston, J. A. D. (2016). Toward automatic model comparison: An adaptive sequential Monte Carlo approach. J. Comput. Graph. Statist. 25 701–726.
