Translator Disclaimer
2018 Noise contrastive estimation: Asymptotic properties, formal comparison with MC-MLE
Lionel Riou-Durand, Nicolas Chopin
Electron. J. Statist. 12(2): 3473-3518 (2018). DOI: 10.1214/18-EJS1485

Abstract

A statistical model is said to be un-normalised when its likelihood function involves an intractable normalising constant. Two popular methods for parameter inference for these models are MC-MLE (Monte Carlo maximum likelihood estimation), and NCE (noise contrastive estimation); both methods rely on simulating artificial data-points to approximate the normalising constant. While the asymptotics of MC-MLE have been established under general hypotheses (Geyer, 1994), this is not so for NCE. We establish consistency and asymptotic normality of NCE estimators under mild assumptions. We compare NCE and MC-MLE under several asymptotic regimes. In particular, we show that, when $m\rightarrow \infty $ while $n$ is fixed ($m$ and $n$ being respectively the number of artificial data-points, and actual data-points), the two estimators are asymptotically equivalent. Conversely, we prove that, when the artificial data-points are IID, and when $n\rightarrow \infty $ while $m/n$ converges to a positive constant, the asymptotic variance of a NCE estimator is always smaller than the asymptotic variance of the corresponding MC-MLE estimator. We illustrate the variance reduction brought by NCE through a numerical study.

Citation

Download Citation

Lionel Riou-Durand. Nicolas Chopin. "Noise contrastive estimation: Asymptotic properties, formal comparison with MC-MLE." Electron. J. Statist. 12 (2) 3473 - 3518, 2018. https://doi.org/10.1214/18-EJS1485

Information

Received: 1 February 2018; Published: 2018
First available in Project Euclid: 17 October 2018

zbMATH: 06970010
MathSciNet: MR3865268
Digital Object Identifier: 10.1214/18-EJS1485

JOURNAL ARTICLE
46 PAGES


SHARE
Vol.12 • No. 2 • 2018
Back to Top