Open Access
2023 How Trustworthy Is Your Tree? Bayesian Phylogenetic Effective Sample Size Through the Lens of Monte Carlo Error
Andrew Magee, Michael Karcher, Frederick A. Matsen IV, Volodymyr M. Minin
Author Affiliations +
Bayesian Anal. Advance Publication 1-29 (2023). DOI: 10.1214/22-BA1339


Bayesian inference is a popular and widely-used approach to infer phylogenies (evolutionary trees). However, despite decades of widespread application, it remains difficult to judge how well a given Bayesian Markov chain Monte Carlo (MCMC) run explores the space of phylogenetic trees. In this paper, we investigate the Monte Carlo error of phylogenies, focusing on high-dimensional summaries of the posterior distribution, including variability in estimated edge/branch (known in phylogenetics as “split”) probabilities and tree probabilities, and variability in the estimated summary tree. Specifically, we ask if there is any measure of effective sample size (ESS) applicable to phylogenetic trees which is capable of capturing the Monte Carlo error of these three summary measures. We find that there are some ESS measures capable of capturing the error inherent in using MCMC samples to approximate the posterior distributions on phylogenies. We term these tree ESS measures, and identify a set of three which are useful in practice for assessing the Monte Carlo error. Lastly, we present visualization tools that can improve comparisons between multiple independent MCMC runs by accounting for the Monte Carlo error present in each chain. Our results indicate that common post-MCMC workflows are insufficient to capture the inherent Monte Carlo error of the tree, and highlight the need for both within-chain mixing and between-chain convergence assessments.

Funding Statement

This work was supported by NSF grants CISE-1561334, CISE-1564137 and DGE-1762114, as well as NIH grant R01 AI162611. Andrew Magee was supported by an ARCS Foundation Fellowship. The research of Frederick Matsen was supported in part by a Faculty Scholar grant from the Howard Hughes Medical Institute and the Simons Foundation. Dr. Matsen is an Investigator of the Howard Hughes Medical Institute. Scientific Computing Infrastructure at Fred Hutch funded by ORIP grant S10OD028685.


Download Citation

Andrew Magee. Michael Karcher. Frederick A. Matsen IV. Volodymyr M. Minin. "How Trustworthy Is Your Tree? Bayesian Phylogenetic Effective Sample Size Through the Lens of Monte Carlo Error." Bayesian Anal. Advance Publication 1 - 29, 2023.


Published: 2023
First available in Project Euclid: 20 January 2023

Digital Object Identifier: 10.1214/22-BA1339

Keywords: effective sample size , MCMC , Monte Carlo error , Phylogenetics

Advance Publication
Back to Top