Uniform rates of the Glivenko–Cantelli convergence and their use in approximating Bayesian inferences

Abstract

This paper deals with suitable quantifications in approximating a probability measure by an “empirical” random probability measure $\hat{\mathfrak{p}}_{n}$, depending on the first $n$ terms of a sequence $\{\tilde{\xi}_{i}\}_{i\geq1}$ of random elements. Section 2 studies the range of oscillation near zero of the Wasserstein distance $\mathrm{d}^{(p)}_{[\mathbb{S}]}$ between $\mathfrak{p}_{0}$ and $\hat{\mathfrak{p}}_{n}$, assuming the $\tilde{\xi}_{i}$’s i.i.d. from $\mathfrak{p}_{0}$. In Theorem 2.1 $\mathfrak{p}_{0}$ can be fixed in the space of all probability measures on $(\mathbb{R}^{d},\mathscr{B}(\mathbb{R}^{d}))$ and $\hat{\mathfrak{p}}_{n}$ coincides with the empirical measure $\tilde{\mathfrak{e}}_{n}:=\frac{1}{n}\sum_{i=1}^{n}\delta_{\tilde{\xi}_{i}}$. In Theorem 2.2 (Theorem 2.3, respectively), $\mathfrak{p}_{0}$ is a $d$-dimensional Gaussian distribution (an element of a distinguished statistical exponential family, respectively) and $\hat{\mathfrak{p}}_{n}$ is another $d$-dimensional Gaussian distribution with estimated mean and covariance matrix (another element of the same family with an estimated parameter, respectively). These new results improve on allied recent works by providing also uniform bounds with respect to $n$, meaning the finiteness of the $p$-moment of $\mathop{\mathrm{sup}}_{n\geq1}b_{n}\mathrm{d}^{(p)}_{[\mathbb{S}]}(\mathfrak{p}_{0},\hat{\mathfrak{p}}_{n})$ is proved for some diverging sequence $b_{n}$ of positive numbers. In Section 3, assuming the $\tilde{\xi}_{i}$’s exchangeable, one studies the range of oscillation near zero of the Wasserstein distance between the conditional distribution – also called posterior – of the directing measure of the sequence, given $\tilde{\xi}_{1},\ldots,\tilde{\xi}_{n}$, and the point mass at $\hat{\mathfrak{p}}_{n}$. Similarly, a bound for the approximation of predictive distributions is given. Finally, Theorems from 3.3 to 3.5 reconsider Theorems from 2.1 to 2.3, respectively, according to a Bayesian perspective.