Open Access
September 2019 Incorporating conditional dependence in latent class models for probabilistic record linkage: Does it matter?
Huiping Xu, Xiaochun Li, Changyu Shen, Siu L. Hui, Shaun Grannis
Ann. Appl. Stat. 13(3): 1753-1790 (September 2019). DOI: 10.1214/19-AOAS1256


The conditional independence assumption of the Felligi and Sunter (FS) model in probabilistic record linkage is often violated when matching real-world data. Ignoring conditional dependence has been shown to seriously bias parameter estimates. However, in record linkage, the ultimate goal is to inform the match status of record pairs and therefore, record linkage algorithms should be evaluated in terms of matching accuracy. In the literature, more flexible models have been proposed to relax the conditional independence assumption, but few studies have assessed whether such accommodations improve matching accuracy. In this paper, we show that incorporating the conditional dependence appropriately yields comparable or improved matching accuracy than the FS model using three real-world data linkage examples. Through a simulation study, we further investigate when conditional dependence models provide improved matching accuracy. Our study shows that the FS model is generally robust to the conditional independence assumption and provides comparable matching accuracy as the more complex conditional dependence models. However, when the match prevalence approaches 0% or 100% and conditional dependence exists in the dominating class, it is necessary to address conditional dependence as the FS model produces suboptimal matching accuracy. The need to address conditional dependence becomes less important when highly discriminating fields are used. Our simulation study also shows that conditional dependence models with misspecified dependence structure could produce less accurate record matching than the FS model and therefore we caution against the blind use of conditional dependence models.


Download Citation

Huiping Xu. Xiaochun Li. Changyu Shen. Siu L. Hui. Shaun Grannis. "Incorporating conditional dependence in latent class models for probabilistic record linkage: Does it matter?." Ann. Appl. Stat. 13 (3) 1753 - 1790, September 2019.


Received: 1 May 2016; Revised: 1 January 2019; Published: September 2019
First available in Project Euclid: 17 October 2019

zbMATH: 07145975
MathSciNet: MR4019157
Digital Object Identifier: 10.1214/19-AOAS1256

Keywords: Conditional dependence , finite mixture , Gaussian random effects model , latent class analysis , Log-linear model , record linkage

Rights: Copyright © 2019 Institute of Mathematical Statistics

Vol.13 • No. 3 • September 2019
Back to Top