A large number of statistical models are “doubly-intractable”: the likelihood normalising term, which is a function of the model parameters, is intractable, as well as the marginal likelihood (model evidence). This means that standard inference techniques to sample from the posterior, such as Markov chain Monte Carlo (MCMC), cannot be used. Examples include, but are not confined to, massive Gaussian Markov random fields, autologistic models and Exponential random graph models. A number of approximate schemes based on MCMC techniques, Approximate Bayesian computation (ABC) or analytic approximations to the posterior have been suggested, and these are reviewed here. Exact MCMC schemes, which can be applied to a subset of doubly-intractable distributions, have also been developed and are described in this paper. As yet, no general method exists which can be applied to all classes of models with doubly-intractable posteriors.

In addition, taking inspiration from the Physics literature, we study an alternative method based on representing the intractable likelihood as an infinite series. Unbiased estimates of the likelihood can then be obtained by finite time stochastic truncation of the series via Russian Roulette sampling, although the estimates are not necessarily positive. Results from the Quantum Chromodynamics literature are exploited to allow the use of possibly negative estimates in a pseudo-marginal MCMC scheme such that expectations with respect to the posterior distribution are preserved. The methodology is reviewed on well-known examples such as the parameters in Ising models, the posterior for Fisher–Bingham distributions on the $d$-Sphere and a large-scale Gaussian Markov Random Field model describing the Ozone Column data. This leads to a critical assessment of the strengths and weaknesses of the methodology with pointers to ongoing research.

The abundance of functional observations in scientific endeavors has led to a significant development in tools for functional data analysis (FDA). This kind of data comes with several challenges: infinite-dimensionality of function spaces, observation noise, and so on. However, there is another interesting phenomena that creates problems in FDA. The functional data often comes with lateral displacements/deformations in curves, a phenomenon which is different from the height or amplitude variability and is termed phase variation. The presence of phase variability artificially often inflates data variance, blurs underlying data structures, and distorts principal components. While the separation and/or removal of phase from amplitude data is desirable, this is a difficult problem. In particular, a commonly used alignment procedure, based on minimizing the $\mathbb{L}^{2}$ norm between functions, does not provide satisfactory results. In this paper we motivate the importance of dealing with the phase variability and summarize several current ideas for separating phase and amplitude components. These approaches differ in the following: (1) the definition and mathematical representation of phase variability, (2) the objective functions that are used in functional data alignment, and (3) the algorithmic tools for solving estimation/optimization problems. We use simple examples to illustrate various approaches and to provide useful contrast between them.

The probabilistic characterization of the relationship between two or more random variables calls for a notion of dependence. Dependence modeling leads to mathematical and statistical challenges, and recent developments in extremal dependence concepts have drawn a lot of attention to probability and its applications in several disciplines. The aim of this paper is to review various concepts of extremal positive and negative dependence, including several recently established results, reconstruct their history, link them to probabilistic optimization problems, and provide a list of open questions in this area. While the concept of extremal positive dependence is agreed upon for random vectors of arbitrary dimensions, various notions of extremal negative dependence arise when more than two random variables are involved. We review existing popular concepts of extremal negative dependence given in literature and introduce a novel notion, which in a general sense includes the existing ones as particular cases. Even if much of the literature on dependence is focused on positive dependence, we show that negative dependence plays an equally important role in the solution of many optimization problems. While the most popular tool used nowadays to model dependence is that of a copula function, in this paper we use the equivalent concept of a set of rearrangements. This is not only for historical reasons. Rearrangement functions describe the relationship between random variables in a completely deterministic way, allow a deeper understanding of dependence itself, and have several advantages on the approximation of solutions in a broad class of optimization problems.

Despite its popularity, it is widely recognized that the investigation of some theoretical aspects of clustering has been relatively sparse. One of the main reasons for this lack of theoretical results is surely the fact that, whereas for other statistical problems the theoretical population goal is clearly defined (as in regression or classification), for some of the clustering methodologies it is difficult to specify the population goal to which the data-based clustering algorithms should try to get close. This paper aims to provide some insight into the theoretical foundations of clustering by focusing on two main objectives: to provide an explicit formulation for the ideal population goal of the modal clustering methodology, which understands clusters as regions of high density; and to present two new loss functions, applicable in fact to any clustering methodology, to evaluate the performance of a data-based clustering algorithm with respect to the ideal population goal. In particular, it is shown that only mild conditions on a sequence of density estimators are needed to ensure that the sequence of modal clusterings that they induce is consistent.

We present a (selective) review of recent frequentist high-dimensional inference methods for constructing $p$-values and confidence intervals in linear and generalized linear models. We include a broad, comparative empirical study which complements the viewpoint from statistical methodology and theory. Furthermore, we introduce and illustrate the R-package hdi which easily allows the use of different methods and supports reproducibility.

Proximal algorithms are useful for obtaining solutions to difficult optimization problems, especially those involving nonsmooth or composite objective functions. A proximal algorithm is one whose basic iterations involve the proximal operator of some function, whose evaluation requires solving a specific optimization problem that is typically easier than the original problem. Many familiar algorithms can be cast in this form, and this “proximal view” turns out to provide a set of broad organizing principles for many algorithms useful in statistics and machine learning. In this paper, we show how a number of recent advances in this area can inform modern statistical practice. We focus on several main themes: (1) variable splitting strategies and the augmented Lagrangian; (2) the broad utility of envelope (or variational) representations of objective functions; (3) proximal algorithms for composite objective functions; and (4) the surprisingly large number of functions for which there are closed-form solutions of proximal operators. We illustrate our methodology with regularized Logistic and Poisson regression incorporating a nonconvex bridge penalty and a fused lasso penalty. We also discuss several related issues, including the convergence of nondescent algorithms, acceleration and optimization for nonconvex functions. Finally, we provide directions for future research in this exciting area at the intersection of statistics and optimization.

Nan McKenzie Laird is the Harvey V. Fineberg Professor of Biostatistics at the Harvard T. H. Chan School of Public Health. She has made fundamental contributions to statistical methods for longitudinal data analysis, missing data and meta-analysis. In addition, she is widely known for her work in statistical genetics and in statistical methods for psychiatric epidemiology. Her 1977 paper with Dempster and Rubin on the EM algorithm is among the top 100 most highly cited papers in science [Nature 524 (2014) 550–553]. Her applied work on medical practice errors is widely cited among the medical malpractice community.

Nan was born in Gainesville, Florida, in 1943. Shortly thereafter, her parents Angus McKenzie Laird and Myra Adelia Doyle, moved to Tallahassee, Florida, with Nan and her sister Victoria Mell. Nan started college at Rice University in 1961, but then transferred to the University of Georgia where she received a B.S. in Statistics in 1969 and was elected to Phi Beta Kappa. After graduation Nan worked at the Massachusetts Institute of Technology Draper Laboratories where she worked on Kalman filtering for the Apollo Man to the Moon Program. She enrolled in the Statistics Department at Harvard University in 1971 and received her Ph.D. in 1975. She joined the faculty of Harvard School of Public Health upon receiving her Ph.D., and remains there as research professor, after her retirement in 2015.

In the 40 years that Nan M. Laird spent at Harvard she authored or co-authored over 300 papers and three books, and mentored numerous graduate students, postdoctoral fellows and junior faculty. According to Google scholar, her work has been cited over 111,000 times. She has received many awards for her varied contributions to statistical science. Some highlights include the Samuel S. Wilks Award, her election as Fellow of the American Association for the Advancement of Science, her election as Fellow of the American Statistical Association, the Janet Norwood Prize, the F. N. David Award and, most recently, the Marvin Zelen Leadership Award in Statistical Science. Professor Laird has served on many panels and editorial boards including a National Academy of Science Panel on Airliner Cabin Environment, which led to the elimination of smoking on airplanes. Professor Laird chaired the Department of Biostatistics at the Harvard School of Public Health from 1990–1999 where she was the Henry Pickering Walcott Professor of Biostatistics. In this interview she talks about her career, including her passion for mentoring students. She offers some helpful advice about balancing work and family life, acknowledging the powerful encouragement and support that her own family has given her over the years.

The interview was conducted in Boston, Massachusetts, in July 2014. A link to Nan’s full CV can be found at www.hsph.harvard.edu/nan-laird/.