Annals of Statistics
- Ann. Statist.
- Volume 46, Number 4 (2018), 1481-1512.
Consistency and convergence rate of phylogenetic inference via regularization
Vu Dinh, Lam Si Tung Ho, Marc A. Suchard, and Frederick A. Matsen IV
Abstract
It is common in phylogenetics to have some, perhaps partial, information about the overall evolutionary tree of a group of organisms and wish to find an evolutionary tree of a specific gene for those organisms. There may not be enough information in the gene sequences alone to accurately reconstruct the correct “gene tree.” Although the gene tree may deviate from the “species tree” due to a variety of genetic processes, in the absence of evidence to the contrary it is parsimonious to assume that they agree. A common statistical approach in these situations is to develop a likelihood penalty to incorporate such additional information. Recent studies using simulation and empirical data suggest that a likelihood penalty quantifying concordance with a species tree can significantly improve the accuracy of gene tree reconstruction compared to using sequence data alone. However, the consistency of such an approach has not yet been established, nor have convergence rates been bounded. Because phylogenetics is a nonstandard inference problem, the standard theory does not apply. In this paper, we propose a penalized maximum likelihood estimator for gene tree reconstruction, where the penalty is the square of the Billera–Holmes–Vogtmann geodesic distance from the gene tree to the species tree. We prove that this method is consistent, and derive its convergence rate for estimating the discrete gene tree structure and continuous edge lengths (representing the amount of evolution that has occurred on that branch) simultaneously. We find that the regularized estimator is “adaptive fast converging,” meaning that it can reconstruct all edges of length greater than any given threshold from gene sequences of polynomial length. Our method does not require the species tree to be known exactly; in fact, our asymptotic theory holds for any such guide tree.
Article information
Source
Ann. Statist., Volume 46, Number 4 (2018), 1481-1512.
Dates
Received: June 2016
Revised: February 2017
First available in Project Euclid: 27 June 2018
Permanent link to this document
https://projecteuclid.org/euclid.aos/1530086423
Digital Object Identifier
doi:10.1214/17-AOS1592
Mathematical Reviews number (MathSciNet)
MR3819107
Zentralblatt MATH identifier
06936468
Subjects
Primary: 05C05: Trees 62F12: Asymptotic properties of estimators
Secondary: 92B10: Taxonomy, cladistics, statistics 92D15: Problems related to evolution
Keywords
Phylogenetics tree reconstruction gene tree species tree maximum likelihood estimator regularization
Citation
Dinh, Vu; Ho, Lam Si Tung; Suchard, Marc A.; Matsen IV, Frederick A. Consistency and convergence rate of phylogenetic inference via regularization. Ann. Statist. 46 (2018), no. 4, 1481--1512. doi:10.1214/17-AOS1592. https://projecteuclid.org/euclid.aos/1530086423