We review inference under models with nonparametric Bayesian (BNP) priors. The discussion follows a set of examples for some common inference problems. The examples are chosen to highlight problems that are challenging for standard parametric inference. We discuss inference for density estimation, clustering, regression and for mixed effects models with random effects distributions. While we focus on arguing for the need for the flexibility of BNP models, we also review some of the more commonly used BNP models, thus hopefully answering a bit of both questions, why and how to use BNP.

This review was sponsored by the Bayesian Nonparametrics Section of ISBA (ISBA/BNP). The authors thank the section officers for the support and encouragement.

Due to their great flexibility, nonparametric Bayes methods have proven to be a valuable tool for discovering complicated patterns in data. The term “nonparametric Bayes” suggests that these methods inherit model-free operating characteristics of classical nonparametric methods, as well as coherent uncertainty assessments provided by Bayesian procedures. However, as the authors say in the conclusion to their article, nonparametric Bayesian methods may be more aptly described as “massively parametric.” Furthermore, I argue that many of the default nonparametric Bayes procedures are only Bayesian in the weakest sense of the term, and cannot be assumed to provide honest assessments of uncertainty merely because they carry the Bayesian label. However useful such procedures may be, we should be cautious about advertising default nonparametric Bayes procedures as either being “assumption free” or providing descriptions of our uncertainty. If we want our nonparametric Bayes procedures to have a Bayesian interpretation, we should modify default NP Bayes methods to accommodate real prior information, or at the very least, carefully evaluate the effects of hyperparameters on posterior quantities of interest.

In Bayesian model selection when the prior information on the parameters of the models is vague default priors should be used. Unfortunately, these priors are usually improper yielding indeterminate Bayes factors that preclude the comparison of the models. To calibrate the initial default priors Cano et al. (2008) proposed integral priors as prior distributions for Bayesian model selection. These priors were defined as the solution of a system of two integral equations that under some general assumptions has a unique solution associated with a recurrent Markov chain. Later, in Cano et al. (2012b) integral priors were successfully applied in some situations where they are known and they are unique, being proper or not, and it was pointed out how to deal with other situations. Here, we present some new situations to illustrate how this new methodology works in the cases where we are not able to explicitly find the integral priors but we know they are proper and unique (one-sided testing for the exponential distribution) and in the cases where recurrence of the associated Markov chains is difficult to check. To deal with this latter scenario we impose a technical constraint on the imaginary training samples space that virtually implies the existence and the uniqueness of integral priors which are proper distributions. The improvement over other existing methodologies comes from the fact that this method is more automatic since we only need to simulate from the involved models and their posteriors to compute very well behaved Bayes factors.

This paper aims at providing the prior and posterior interpretations for the parameters in the logistic regression model with random or cluster-level intercept when univariate and multivariate classes of skew normal distributions are assumed to model the random effects behavior. We obtain the prior distributions for the odds ratio and their medians under skew normality for the random effects. Original results related to linear combinations of skew-normal distributions are obtained as a by-product and, in the univariate case, a new class of log-skew-normal distribution is introduced. Robust results are obtained whenever a class of multivariate skew-normal distribution is assumed. We also evaluate the effect of the misspecification of the random effects distributions in the odds ratio estimation. We consider both simulated and the Teratogenic activity experiment datasets. The latter was previously analysed in the literature. We concluded that the misspecification of the random effects distribution yields poor odds ratios estimates and that the median odds ratio is not necessarily the best measure of heterogeneity among the clusters as suggested in the literature.

Sequential Monte Carlo (SMC) methods are not only a popular tool in the analysis of state–space models, but offer an alternative to Markov chain Monte Carlo (MCMC) in situations where Bayesian inference must proceed via simulation. This paper introduces a new SMC method that uses adaptive MCMC kernels for particle dynamics. The proposed algorithm features an online stochastic optimization procedure to select the best MCMC kernel and simultaneously learn optimal tuning parameters. Theoretical results are presented that justify the approach and give guidance on how it should be implemented. Empirical results, based on analysing data from mixture models, show that the new adaptive SMC algorithm (ASMC) can both choose the best MCMC kernel, and learn an appropriate scaling for it. ASMC with a choice between kernels outperformed the adaptive MCMC algorithm of Haario et al. (1998) in 5 out of the 6 cases considered.

A family of prior distributions for covariance matrices is studied. Members of the family possess the attractive property of all standard deviation and correlation parameters being marginally noninformative for particular hyperparameter choices. Moreover, the family is quite simple and, for approximate Bayesian inference techniques such as Markov chain Monte Carlo and mean field variational Bayes, has tractability on par with the Inverse-Wishart conjugate family of prior distributions. A simulation study shows that the new prior distributions can lead to more accurate sparse covariance matrix estimation.

Multinomial outcomes with many levels can be challenging to model. Information typically accrues slowly with increasing sample size, yet the parameter space expands rapidly with additional covariates. Shrinking all regression parameters towards zero, as often done in models of continuous or binary response variables, is unsatisfactory, since setting parameters equal to zero in multinomial models does not necessarily imply “no effect.” We propose an approach to modeling multinomial outcomes with many levels based on a Bayesian multinomial probit (MNP) model and a multiple shrinkage prior distribution for the regression parameters. The prior distribution encourages the MNP regression parameters to shrink toward a number of learned locations, thereby substantially reducing the dimension of the parameter space. Using simulated data, we compare the predictive performance of this model against two other recently-proposed methods for big multinomial models. The results suggest that the fully Bayesian, multiple shrinkage approach can outperform these other methods. We apply the multiple shrinkage MNP to simulating replacement values for areal identifiers, e.g., census tract indicators, in order to protect data confidentiality in public use datasets.

We explore an asymptotic justification for the widely used and empirically verified approach of assuming an asymmetric Laplace distribution (ALD) for the response in Bayesian Quantile Regression. Based on empirical findings, Yu and Moyeed (2001) argued that the use of ALD is satisfactory even if it is not the true underlying distribution. We provide a justification to this claim by establishing posterior consistency and deriving the rate of convergence under the ALD misspecification. Related literature on misspecified models focuses mostly on i.i.d. models which in the regression context amounts to considering i.i.d. random covariates with i.i.d. errors. We study the behavior of the posterior for the misspecified ALD model with independent but non identically distributed response in the presence of non-random covariates. Exploiting the specific form of ALD helps us derive conditions that are more intuitive and easily seen to be satisfied by a wide range of potential true underlying probability distributions for the response. Through simulations, we demonstrate our result and also find that the robustness of the posterior that holds for ALD fails for a Gaussian formulation, thus providing further support for the use of ALD models in quantile regression.