Probability Surveys

Stein’s method for comparison of univariate distributions

Christophe Ley, Gesine Reinert, and Yvik Swan

Full-text: Open access

Abstract

We propose a new general version of Stein’s method for univariate distributions. In particular we propose a canonical definition of the Stein operator of a probability distribution which is based on a linear difference or differential-type operator. The resulting Stein identity highlights the unifying theme behind the literature on Stein’s method (both for continuous and discrete distributions). Viewing the Stein operator as an operator acting on pairs of functions, we provide an extensive toolkit for distributional comparisons. Several abstract approximation theorems are provided. Our approach is illustrated for comparison of several pairs of distributions: normal vs normal, sums of independent Rademacher vs normal, normal vs Student, and maximum of random variables vs exponential, Fréchet and Gumbel.

Article information

Source
Probab. Surveys Volume 14 (2017), 1-52.

Dates
Received: April 2016
First available in Project Euclid: 9 January 2017

Permanent link to this document
https://projecteuclid.org/euclid.ps/1483952471

Digital Object Identifier
doi:10.1214/16-PS278

Subjects
Primary: 60B10: Convergence of probability measures
Secondary: 60E15: Inequalities; stochastic orderings 60E05: Distributions: general theory 60F05: Central limit and other weak theorems

Keywords
Density approach Stein’s method comparison of distributions

Citation

Ley, Christophe; Reinert, Gesine; Swan, Yvik. Stein’s method for comparison of univariate distributions. Probab. Surveys 14 (2017), 1--52. doi:10.1214/16-PS278. https://projecteuclid.org/euclid.ps/1483952471.


Export citation

References

  • [1] Afendras, G., Balakrishnan, N. and Papadatos, N. (2014). Orthogonal polynomials in the cumulative Ord family and its application to variance bounds, preprint arXiv:1408.1849.
  • [2] Afendras, G., Papadatos, N. and Papathanasiou, V. (2011). An extended Stein-type covariance identity for the Pearson family with applications to lower variance bounds, Bernoulli 17, 507–529.
  • [3] Arras, B., Azmoodeh, E., Poly, G. and Swan, Y. (2016). Stein’s method on the second Wiener chaos: 2-Wasserstein distance, preprint arXiv:1601.03301.
  • [4] Baldi, P., Rinott, Y. and Stein, C. (1989). A normal approximation for the number of local maxima of a random function on a graph, Probability, Statistics, and Mathematics, Academic Press, Boston, MA, 59–81.
  • [5] Barbour, A. D. (1990). Stein’s method for diffusion approximations, Probability Theory and Related Fields 84, 297–322.
  • [6] Barbour, A. D. and Chen L. H. Y. (2005). An introduction to Stein’s method, Lecture Notes Series Inst. Math. Sci. Natl. Univ. Singap. 4, Singapore University Press, Singapore.
  • [7] Barbour, A. D., Gan, H. L. and Xia, A. (2015). Stein factors for negative binomial approximation in Wasserstein distance, Bernoulli 21, 1002–1013.
  • [8] Barbour, A. D., Holst, L. and Janson, S. (1992). Poisson approximation, Oxford Studies in Probability, 2, The Clarendon Press Oxford University Press, New York, Oxford Science Publications.
  • [9] Barbour, A. D. and Eagleson, G. K. (1985). Multiple comparisons and sums of dissociated random variables, Advances in Applied Probability 17, 147–162.
  • [10] Barbour, A. D., Luczak M. J. and Xia A. (2015). Multivariate approximation in total variation, preprint arXiv:1512.07400.
  • [11] Bonis T. (2015). Stable measures and Stein’s method: rates in the Central Limit Theorem and diffusion approximation, preprint arXiv:1506.06966.
  • [12] Brown, T. C. and Xia, A. (1995). On Stein-Chen factors for Poisson approximation, Statistics & Probability Letters 23, 327–332.
  • [13] Cacoullos, T., Papadatos, N. and Papathanasiou, V. (2001). An application of a density transform and the local limit theorem,Teor. Veroyatnost. i Primenen. 46, 803–810.
  • [14] Cacoullos, T. and Papathanasiou, V. (1989). Characterizations of distributions by variance bounds, Statistics & Probability Letters 7, 351–356.
  • [15] Cacoullos, T., Papadatos, N. and Papathanasiou, V. (1998). Variance inequalities for covariance kernels and applications to central limit theorems, Theory of Probability & Its Applications 42, 149–155.
  • [16] Cacoullos, T. and Papathanasiou, V. (1995). A generalization of covariance identity and related characterizations, Mathematical Methods of Statistics 4, 106–113.
  • [17] Cacoullos, T., Papathanasiou, V. and Utev, S. A. (1994). Variational inequalities with examples and an application to the central limit theorem, Annals of Probability 22, 1607–1618.
  • [18] Chatterjee, S. (2014). A short survey of Stein’s method, Proceedings of ICM 2014, to appear.
  • [19] Chatterjee, S., Fulman, J. and Röllin, A. (2011). Exponential approximation by exchangeable pairs and spectral graph theory, ALEA Latin American Journal of Probability and Mathematical Statistics 8, 1–27.
  • [20] Chatterjee, S. and Meckes, E. (2008). Multivariate normal approximation using exchangeable pairs, ALEA Latin American Journal of Probability and Mathematical Statistics 4, 257–283.
  • [21] Chatterjee, S. and Shao, Q.-M. (2011). Nonnormal approximation by Stein’s method of exchangeable pairs with application to the Curie-Weiss model, Annals of Applied Probability 21, 464–483.
  • [22] Chen, L. H. Y. (1975). Poisson approximation for dependent trials, Annals of Probability 3, 534–545.
  • [23] Chen, L. H. Y. (1980). An inequality for multivariate normal distribution, Technical report, MIT.
  • [24] Chen, L. H. Y., Goldstein, L. and Shao, Q.-M. (2011). Normal approximation by Stein’s method, Probability and its Applications (New York), Springer, Heidelberg.
  • [25] Daly, F. (2008). Upper bounds for Stein-type operators, Electronic Journal of Probability 13, 566–587.
  • [26] Diaconis, P. and Zabell, S. (1991). Closed form summation for classical distributions: variations on a theme of de Moivre, Statistical Science 6, 284–302.
  • [27] Döbler, C. (2012). On rates of convergence and Berry-Esseen bounds for random sums of centered random variables with finite third moments, preprint arXiv:1212.5401.
  • [28] Döbler, C. (2015). Stein’s method of exchangeable pairs for the beta distribution and generalizations, Electronic Journal of Probability 20, 1–34.
  • [29] Döbler, C., Gaunt, R. E. and Vollmer, S. J. (2015). An iterative technique for bounding derivatives of solutions of Stein equations, preprint arXiv:1510.02623.
  • [30] Eden, R. and Viquez, J. (2015). Nourdin-Peccati analysis on Wiener and Wiener-Poisson space for general distributions, Stochastic Processes and their Applications 125, 182–216.
  • [31] Ehm, W. (1991). Binomial approximation to the Poisson binomial distribution, Statistics & Probability Letters 11, 7–16.
  • [32] Eichelsbacher, P. and Löwe, M. (2010). Stein’s method for dependent random variables occurring in statistical mechanics, Electronic Journal of Probability 15, 962–988.
  • [33] Eichelsbacher, P. and Martschink, B. (2014). Rates of convergence in the Blume–Emery–Griffiths model, Journal of Statistical Physics 154, 1483–1507.
  • [34] Eichelsbacher, P. and Reinert, G. (2008). Stein’s method for discrete Gibbs measures, Annals of Applied Probability 18, 1588–1618.
  • [35] Fulman, J. and Goldstein, L. (2015). Stein’s method and the rank distribution of random matrices over finite fields, Annals of Probability 43, 1274–1314.
  • [36] Fulman, J. and Goldstein, L. (2014). Stein’s method, semicircle distribution, and reduced decompositions of the longest element in the symmetric group, preprint arXiv:1405.1088.
  • [37] Gaunt, R. E. (2016). On Stein’s method for products of normal random variables and zero bias couplings, Bernoulli, to appear.
  • [38] Gaunt, R. E. (2014). Variance-Gamma approximation via Stein’s method, Electronic Journal of Probability 19, 1–33.
  • [39] Gaunt, R. E., Mijoule, G. and Swan, Y. (2016). Stein operators for product distributions, with applications. preprint arXiv:1604.06819.
  • [40] Gibbs, A. L. and Su, F. E. (2002). On choosing and bounding probability metrics, International Statistical Review / Revue Internationale de Statistique 70, 419–435.
  • [41] Goldstein, L. and Reinert, G. (1997). Stein’s method and the zero bias transformation with application to simple random sampling, Annals of Applied Probability 7, 935–952.
  • [42] Goldstein, L. and Reinert, G. (2005). Distributional transformations, orthogonal polynomials, and Stein characterizations, Journal of Theoretical Probability 18, 237–260.
  • [43] Goldstein, L. and Reinert, G. (2013). Stein’s method for the Beta distribution and the Pólya-Eggenberger urn, Journal of Applied Probability 50, 1187–1205.
  • [44] Goldstein, L. and Rinott, Y. (1996). Multivariate normal approximations by Stein’s method and size bias couplings, Journal of Applied Probability 33, 1–17.
  • [45] Gorham, J. and Mackey, L. (2015). Measuring sample quality with Stein’s method. In: Advances in Neural Information Processing Systems (NIPS), 226–234.
  • [46] Götze, F. and Tikhomirov, A. N. (2003). Rate of convergence to the semi-circular law, Probability Theory and Related Fields 127, 228–276.
  • [47] Götze, F. and Tikhomirov, A. N. (2006). Limit theorems for spectra of random matrices with martingale structure, Teor. Veroyatnost. i Primenen. 51, 171–192.
  • [48] Götze, F. (1991). On the rate of convergence in the multivariate CLT, Annals of Probability 19, 724–739.
  • [49] Haagerup, U. and Thorbjørnsen, S. (2012). Asymptotic expansions for the Gaussian unitary ensemble, Infinite Dimensional Analysis, Quantum Probability and Related Topics 15, no. 01.
  • [50] Hall, W. J. and Wellner, J. A. (1979). The rate of convergence in law of the maximum of an exponential sample, Statistica Neerlandica 33, 151–154.
  • [51] Hillion, E., Johnson, O. and Yu, Y. (2014). A natural derivative on [0, n] and a binomial Poincaré inequality. ESAIM: Probability and Statistics 18, 703–712.
  • [52] Holmes, S. (2004). Stein’s method for birth and death chains, Stein’s Method: Expository Lectures and Applications, IMS Lecture Notes Monogr. Ser., 46, 45–67.
  • [53] Johnson, O. (2004). Information Theory and the Central Limit Theorem, Imperial College Press, London.
  • [54] Johnson, O. and Barron, A. (2004). Fisher information inequalities and the central limit theorem, Probability Theory and Related Fields 129, 391–409.
  • [55] Johnson, R. W. (1993). A note on variance bounds for a function of a Pearson variate, Statistics & Risk Modeling 11, 273–278.
  • [56] Klaassen, C. A. J. (1985). On an inequality of Chernoff, Annals of Probability 13, 966–974.
  • [57] Korwar, R. M. (1991). On characterizations of distributions by mean absolute deviation and variance bounds, Annals of the Institute of Statistical Mathematics 43, 287–295.
  • [58] Kusuoka, S. and Tudor, C. A. (2012). Stein’s method for invariant measures of diffusions via Malliavin calculus, Stochastic Processes and Their Applications 122, 1627–1651.
  • [59] Kusuoka, S. and Tudor, C. A. (2013). Extension of the fourth moment theorem to invariant measures of diffusions, preprint arXiv:1310.3785.
  • [60] Ledoux, M., Nourdin, I. and Peccati, G. (2015). Stein’s method, logarithmic Sobolev and transport inequalities, Geometric and Functional Analysis 25, 256–306.
  • [61] Lefèvre, C., Papathanasiou, V. and Utev, S. (2002). Generalized Pearson distributions and related characterization problems, Annals of the Institute of Statistical Mathematics 54, 731–742.
  • [62] Ley, C., Reinert, G. and Swan, Y. (2016). Distances between nested densities and a measure of the impact of the prior in Bayesian statistics, Annals of Applied Probability, to appear.
  • [63] Ley, C. and Swan, Y. (2016). A general parametric Stein characterization, Statistics & Probability Letters 111, 67–71.
  • [64] Ley, C. and Swan, Y. (2013). Local Pinsker inequalities via Stein’s discrete density approach, IEEE Transactions on Information Theory 59, 5584–4491.
  • [65] Ley, C. and Swan, Y. (2013). Stein’s density approach and information inequalities, Electronic Communications in Probability 18, 1–14.
  • [66] Ley, C. and Swan, Y. (2016). Parametric Stein operators and variance bounds, Brazilian Journal of Probability and Statistics 30, 171–195.
  • [67] Loh, W.-L. (2004). On the characteristic function of Pearson type IV distributions, A Festschrift for Herman Rubin, Institute of Mathematical Statistics, 171–179.
  • [68] Luk, H. M. (1994). Stein’s method for the Gamma distribution and related statistical applications, Ph.D. thesis, University of Southern California.
  • [69] Mackey, L. and Gorham, J. (2016). Multivariate Stein factors for a class of strongly log-concave distributions. Electronic Communications in Probability 21 paper no. 56, 14 pp.
  • [70] Nourdin, I. and Peccati, G. (2009). Stein’s method on Wiener chaos, Probability Theory and Related Fields 145, 75–118.
  • [71] Nourdin, I. and Peccati, G. (2012). Normal approximations with Malliavin calculus: from Stein’s method to universality, Cambridge Tracts in Mathematics, Cambridge University Press.
  • [72] Nourdin, I., Peccati, G. and Reinert, G. (2009). Second order Poincaré inequalities and CLTs on Wiener space, Journal of Functional Analysis 257, 593–609.
  • [73] Nourdin, I., Peccati, G. and Swan, Y. (2014). Entropy and the fourth moment phenomenon, Journal of Functional Analysis 266, 3170–3207.
  • [74] Nourdin, I., Peccati, G. and Swan, Y. (2014). Integration by parts and representation of information functionals, IEEE International Symposium on Information Theory (ISIT), 2217–2221.
  • [75] Novak, S. Y. (2011). Extreme Value Methods with Applications to Finance, Chapman & Hall/CRC Press, Boca Raton.
  • [76] Ord, J. K. (1967). On a system of discrete distributions, Biometrika 54, 649–656.
  • [77] Papadatos, N. and Papathanasiou, V. (1995). Distance in variation between two arbitrary distributions via the associated w-functions, Theory of Probability & Its Applications 40, 567–575.
  • [78] Papathanasiou, V. (1995). A characterization of the Pearson system of distributions and the associated orthogonal polynomials, Annals of the Institute of Statistical Mathematics 47, 171–176.
  • [79] Peköz, E. and Röllin, A. (2011). New rates for exponential approximation and the theorems of Rényi and Yaglom, Annals of Probability 39, 587–608.
  • [80] Peköz, E. Röllin, A. and Ross, N. (2013). Degree asymptotics with rates for preferential attachment random graphs, Annals of Applied Probability 23, 1188–1218.
  • [81] Pickett, A. (2004). Rates of convergence of $\chi^{2}$ approximations via Stein’s method, Ph.D. thesis, Lincoln College, University of Oxford.
  • [82] Pike, J. and Ren, H. (2014). Stein’s method and the Laplace distribution, ALEA Latin American Journal of Probability and Mathematical Statistics 11, 571–587.
  • [83] Rachev, S. T. (1991). Probability Metrics and the Stability of Stochastic Models, vol. 334, Wiley New York.
  • [84] Reinert, G. (1998). Couplings for normal approximations with Stein’s method, DIMACS Ser. Discrete Math. Theoret. Comput. Sci., 41, 193–207.
  • [85] Reinert, G. and Röllin, A. (2009). Multivariate normal approximation with Stein’s method of exchangeable pairs under a general linearity condition, Annals of Probability 37, 2150–2173.
  • [86] Röllin, A. (2012). On magic factors and the construction of examples with sharp rates in Stein’s method. Probability Approximations and Beyond, Lecture Notes in Statistics 205, Springer.
  • [87] Ross, N. (2011). Fundamentals of Stein’s method, Probability Surveys 8, 210–293.
  • [88] Schoutens, W. (2001). Orthogonal polynomials in Stein’s method, Journal of Mathematical Analysis and Applications 253, 515–531.
  • [89] Shimizu, R. (1975). On Fisher’s amount of information for location family, A Modern Course on Statistical Distributions in Scientific Work, Springer, 305–312.
  • [90] Stein, C. (1972). A bound for the error in the normal approximation to the distribution of a sum of dependent random variables, Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (Univ. California, Berkeley, Calif., 1970/1971), Vol. II: Probability theory (Berkeley, Calif.), Univ. California Press, 583–602.
  • [91] Stein, C. (1986). Approximate computation of expectations, Institute of Mathematical Statistics Lecture Notes—Monograph Series, 7, Institute of Mathematical Statistics, Hayward, CA.
  • [92] Stein, C., Diaconis, P., Holmes, S. and Reinert, G. (2004). Use of exchangeable pairs in the analysis of simulations, Stein’s method: expository lectures and applications (Persi Diaconis and Susan Holmes, eds.), IMS Lecture Notes Monogr. Ser, vol. 46, Beachwood, Ohio, USA: Institute of Mathematical Statistics, 1–26.
  • [93] Upadhye, N. S., Cekanavicius, V. and Vellaisamy, P. (2016). On Stein operators for discrete approximations, Bernoulli, to appear.