Thomas Bayes, from whom Bayes theorem takes its name, was probably born in 1701, so the year 2001 marked the 300th anniversary of his birth. This biography was written to celebrate this anniversary. The current sketch of his life includes his family background and education, as well as his scientific and theological work. In contrast to some, but not all, biographies of Bayes, the current biography is an attempt to cover areas beyond Bayes’ scientific work. When commenting on the writing of scientific biography, Pearson [(1978). The History of Statistics in the 17th and 18th Centuries… . Charles Griffin and Company, London] stated, “it is impossible to understand a man’s work unless you understand something of his character and unless you understand something of his environment. And his environment means the state of affairs social and political of his own age.” The intention here is to follow this general approach to biography.

There is very little primary source material on Bayes and his work. For example, only three of his letters and a notebook containing some sketches of his own work, almost all unpublished, as well as notes on the work of others are known to have survived. Neither the letters nor the notebook is dated, and only one of the letters can be dated accurately from internal evidence. This biography contains new information about Bayes. In particular, among the papers of the 2nd Earl Stanhope, letters and papers of Bayes have been uncovered that previously were not known to exist. The letters indirectly confirm the centrality of Stanhope in Bayes’ election to the Royal Society. They also provide evidence that Bayes was part of a network of mathematicians initially centered on Stanhope. In addition, the letters shed light on Bayes’ work in infinite series.

This article expounds a philosophical approach to Probability and Causality: a synthesis of the personalist Bayesian views of de Finetti and Popper’s falsificationist programme. A falsification method for probabilistic or causal theories, based on “Borel criteria,” is described. It is argued that this minimalist approach, free of any distracting metaphysical inputs, provides the essential support required for the conduct and advance of Science.

Statistics has struggled for nearly a century over the issue of whether the Bayesian or frequentist paradigm is superior. This debate is far from over and, indeed, should continue, since there are fundamental philosophical and pedagogical issues at stake. At the methodological level, however, the debate has become considerably muted, with the recognition that each approach has a great deal to contribute to statistical practice and each is actually essential for full development of the other approach. In this article, we embark upon a rather idiosyncratic walk through some of these issues.

The evolution of Bayesian approaches for model uncertainty over the past decade has been remarkable. Catalyzed by advances in methods and technology for posterior computation, the scope of these methods has widened substantially. Major thrusts of these developments have included new methods for semiautomatic prior specification and posterior exploration. To illustrate key aspects of this evolution, the highlights of some of these developments are described.

We review the current state of nonparametric Bayesian inference. The discussion follows a list of important statistical inference problems, including density estimation, regression, survival analysis, hierarchical models and model validation. For each inference problem we review relevant nonparametric Bayesian models and approaches including Dirichlet process (DP) models and variations, Pólya trees, wavelet based models, neural network models, spline regression, CART, dependent DP models and model validation with DP and Pólya tree extensions of parametric models.

A survey of modern Bayesian asymptotics is given. Specific attention is paid to the Hellinger consistency of posterior distributions and the asymptotic study of Bayes factors.

The emergence in the past years of Bayesian analysis in many methodological and applied fields as the solution to the modeling of complex problems cannot be dissociated from major changes in its computational implementation. We show in this review how the advances in Bayesian analysis and statistical computation are intermingled.

Models such as feed-forward neural networks and certain other structures investigated in the computer science literature are not amenable to closed-form Bayesian analysis. The paper reviews the various approaches taken to overcome this difficulty, involving the use of Gaussian approximations, Markov chain Monte Carlo simulation routines and a class of non-Gaussian but “deterministic” approximations called variational approximations.

Statistical applications in fields such as bioinformatics, information retrieval, speech processing, image processing and communications often involve large-scale models in which thousands or millions of random variables are linked in complex ways. Graphical models provide a general methodology for approaching these problems, and indeed many of the models developed by researchers in these applied fields are instances of the general graphical model formalism. We review some of the basic ideas underlying graphical models, including the algorithmic ideas that allow graphical models to be deployed in large-scale data analysis problems. We also present examples of graphical models in bioinformatics, error-control coding and language processing.

We argue that the Bayesian approach is best seen as providing additional tools for those carrying out health-care evaluations, rather than replacing their traditional methods. A distinction is made between those features that arise from the basic Bayesian philosophy and those that come from the modern ability to make inferences using very complex models. Selected examples of the former include explicit recognition of the wide cast of stakeholders in any evaluation, simple use of Bayes theorem and use of a community of prior distributions. In the context of complex models, we selectively focus on the possible role of simple Monte Carlo methods, alternative structural models for incorporating historical data and making inferences on complex functions of indirectly estimated parameters. These selected issues are illustrated by two worked examples presented in a standardized format. The emphasis throughout is on inference rather than decision-making.

The Bayesian approach is being used increasingly in medical research. The flexibility of the Bayesian approach allows for building designs of clinical trials that have good properties of any desired sort. Examples include maximizing effective treatment of patients in the trial, maximizing information about the slope of a dose–response curve, minimizing costs, minimizing the number of patients treated, minimizing the length of the trial and combinations of these desiderata. They also include standard frequentist operating characteristics when these are important considerations. Posterior probabilities are updated via Bayes’ theorem on the basis of accumulating data. These are used to effect modifications of the trial’s course, including stopping accrual, extending accrual beyond that originally planned, dropping treatment arms, adding arms, etc. An important aspect of the approach I advocate is modeling the relationship between a trial’s primary endpoint and early indications of patient performance—auxiliary endpoints. This has several highly desirable consequences. One is that it improves the efficiency of adaptive trials because information is available sooner than otherwise.

The Bayesian approach together with Markov chain Monte Carlo techniques has provided an attractive solution to many important bioinformatics problems such as multiple sequence alignment, microarray analysis and the discovery of gene regulatory binding motifs. The employment of such methods and, more broadly, explicit statistical modeling, has revolutionized the field of computational biology. After reviewing several heuristics-based computational methods, this article presents a systematic account of Bayesian formulations and solutions to the motif discovery problem. Generalizations are made to further enhance the Bayesian approach. Motivated by the need of a speedy algorithm, we also provide a perspective of the problem from the viewpoint of optimizing a scoring function. We observe that scoring functions resulting from proper posterior distributions, or approximations to such distributions, showed the best performance and can be used to improve upon existing motif-finding programs. Simulation analyses and a real-data example are used to support our observation.

James O. Berger was born on April 6, 1950, in Minneapolis, Minnesota, to Orvis and Thelma Berger. He earned his AB, MA and Ph.D. degrees in mathematics from Cornell University in 1971, 1973 and 1974, respectively. He served at Purdue University 1974–1997, as the Richard M. Brumfield Distinguished Professor of Statistics 1986–1997, and at Duke University as the Arts and Sciences Professor of Statistics in the Institute of Statistics and Decision Sciences (ISDS) from 1997 to the present. He is also the Director of the Statistical and Applied Mathematical Sciences Institute (SAMSI), one of the U.S. National Science Foundation’s national institutes in the mathematical and statistical sciences.

Berger served as President of the Institute of Mathematical Statistics (IMS) 1995–1996 and Chair of the Section on Bayesian Statistical Science (SBSS) of the American Statistical Association (ASA) in 1995; he is President of the International Society for Bayesian Analysis (ISBA) in 2004. He has served as Co-Editor of The Annals of Statistics and on the editorial boards of the Springer Series in Statistics, of the Journal of Statistical Planning and Inference, of Statistics and Decisions, of the International Statistical Review and of Test. He has organized or been on the organizing committees of twenty-eight conferences, including five of the Purdue Symposia on Statistical Decision Theory and Related Topics and four of the Valencia International Meetings on Bayesian Statistics.

Berger is the recipient of a host of honors. He is an elected fellow of the AAAS, the ASA and the IMS and an Elected Member of the International Statistical Institute (ISI). He has earned Guggenheim and Sloan Fellowships and was the 1985 winner of the COPSS Presidents’ Award ( joint from IMS, ASA, ENAR, WNAR and CSS) given to an outstanding statistician below forty years of age (it is particularly noteworthy that he won that award at the tender age of 35), and was selected as the COPSS Fisher Lecturer in 2001. He was elected as a foreign member of the Spanish Real Academia de Ciencias in 2002, and to membership in the U.S. National Academy of Sciences in 2003.

Berger has been the advisor of 30 Ph.D. students, has written or edited thirteen books and has numerous other statistical publications. A listing of these publications can be found at his Web site, http://www.stat.duke.edu/~berger/.

Berger married Ann Louise Duer (whom he first met when they were in the seventh grade together) in 1970, and they have two children, Jill Berger, who is married to Sascha Hallstein and works as an optical scientist in Silicon Valley, and Julie Gish, who is married to Ryan Gish and works as a consultant in Chicago.