November 2022 A unified performance analysis of likelihood-informed subspace methods
Tiangang Cui, Xin T. Tong
Author Affiliations +
Bernoulli 28(4): 2788-2815 (November 2022). DOI: 10.3150/21-BEJ1437


The likelihood-informed subspace (LIS) method offers a viable route to reducing the dimensionality of high-dimensional probability distributions arising in Bayesian inference. LIS identifies an intrinsic low-dimensional linear subspace where the target distribution differs the most from some tractable reference distribution. Such a subspace can be identified using the leading eigenvectors of a Gram matrix of the gradient of the log-likelihood function. Then, the original high-dimensional target distribution is approximated through various forms of marginalization of the likelihood function, in which the approximated likelihood only has support on the intrinsic low-dimensional subspace. This approximation enables the design of inference algorithms that can scale sub-linearly with the apparent dimensionality of the problem. Intuitively, the accuracy of the approximation, and hence the performance of the inference algorithms, are influenced by three factors—the dimension truncation error in identifying the subspace, Monte Carlo error in estimating the Gram matrices, and Monte Carlo error in constructing marginalizations. This work establishes a unified framework to analyze each of these three factors and their interplay. Under mild technical assumptions, we establish error bounds for a range of existing dimension reduction techniques based on the principle of LIS. Our error bounds also provide useful insights into the accuracy of these methods. In addition, we analyze the integration of LIS with sampling methods such as Markov Chain Monte Carlo (MCMC) and sequential Monte Carlo (SMC). We also demonstrate the applicability of our analysis on a linear inverse problem with Gaussian prior, which shows that all the estimates can be dimension-independent if the prior covariance is a trace-class operator. Finally, we demonstrate various aspects of our theoretical claims on two nonlinear inverse problems.

Funding Statement

X. T. Tong’s research is supported by MOE Academic Research Funds R-146-000-292-114. T. Cui acknowledges support from the Australian Research Council under the grant DP210103092.


The authors also thank the anonymous reviewer for all the suggestions made in the reviewing process.


Download Citation

Tiangang Cui. Xin T. Tong. "A unified performance analysis of likelihood-informed subspace methods." Bernoulli 28 (4) 2788 - 2815, November 2022.


Received: 1 January 2021; Published: November 2022
First available in Project Euclid: 17 August 2022

zbMATH: 1501.65002
MathSciNet: MR4474562
Digital Object Identifier: 10.3150/21-BEJ1437

Keywords: Approximation error , Dimension reduction , likelihood informed subspace , Monte Carlo estimation


This article is only available to subscribers.
It is not available for individual sale.

Vol.28 • No. 4 • November 2022
Back to Top