Abstract
We investigate the approximation of high-dimensional target measures as low-dimensional updates of a dominating reference measure. This approximation class replaces the associated density with the composition of: (i) a feature map that identifies the leading principal components or features of the target measure, relative to the reference, and (ii) a low-dimensional profile function. When the reference measure satisfies a subspace ϕ-Sobolev inequality, we construct a computationally tractable approximation that yields certifiable error guarantees with respect to the Amari α-divergences. Our construction proceeds in two stages. First, for any feature map and any α-divergence, we obtain an analytical expression for the optimal profile function. Second, for linear feature maps, the principal features are obtained from eigenvectors of a matrix involving gradients of the log-density. Neither step requires explicit access to normalizing constants. Notably, by leveraging the ϕ-Sobolev inequalities, we demonstrate that these features universally certify approximation errors across the range of α-divergences
Funding Statement
ML and YMM acknowledge support from the US Department of Energy, Office of Advanced Scientific Computing Research, under grants DE-SC0023187 and DE-SC0023188, and from the ExxonMobil Technology and Engineering Company. OZ acknowledges support from the ANR JCJC project MODENA (ANR-21-CE46-0006-01).
Acknowledgments
The authors would like to thank the anonymous referees, the associate editor, and the editor-in-chief for many insightful and constructive comments that improved the quality of this paper.
Citation
Matthew T.C. Li. Youssef Marzouk. Olivier Zahm. "Principal feature detection via ϕ-Sobolev inequalities." Bernoulli 30 (4) 2979 - 3003, November 2024. https://doi.org/10.3150/23-BEJ1702
Information