Bayesian Analysis

Markov switching Dirichlet process mixture regression

Athanasios Kottas and Matthew A. Taddy

Full-text: Open access

PDF File (766 KB)

Abstract

Markov switching models can be used to study heterogeneous populations that are observed over time. This paper explores modeling the group characteristics nonparametrically, under both homogeneous and nonhomogeneous Markov switching for group probabilities. The model formulation involves a finite mixture of conditionally independent Dirichlet process mixtures, with a Markov chain defining the mixing distribution. The proposed methodology focuses on settings where the number of subpopulations is small and can be assumed to be known, and flexible modeling is required for group regressions. We develop Dirichlet process mixture prior probability models for the joint distribution of individual group responses and covariates. The implied conditional distribution of the response given the covariates is then used for inference. The modeling framework allows for both non-linearities in the resulting regression functions and non-standard shapes in the response distributions. We design a simulation-based model fitting method for full posterior inference. Furthermore, we propose a general approach for inclusion of external covariates dependent on the Markov chain but conditionally independent from the response. The methodology is applied to a problem from fisheries research involving analysis of stock-recruitment data under shifts in the ecosystem state.

Article information

Source
Bayesian Anal., Volume 4, Number 4 (2009), 793-816.

Dates
First available in Project Euclid: 22 June 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1340369825

Digital Object Identifier
doi:10.1214/09-BA430

Mathematical Reviews number (MathSciNet)
MR2570089

Zentralblatt MATH identifier
1330.62141

Keywords
Dirichlet process prior hidden Markov model Markov chain Monte Carlo multivariate normal mixture stock-recruitment relationship

Citation

Taddy, Matthew A.; Kottas, Athanasios. Markov switching Dirichlet process mixture regression. Bayesian Anal. 4 (2009), no. 4, 793--816. doi:10.1214/09-BA430. https://projecteuclid.org/euclid.ba/1340369825


Export citation

References

  • Antoniak, C. (1974). "Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems." Annals of Statistics, 2: 1152 – 1174.
    Mathematical Reviews (MathSciNet): MR365969
    Zentralblatt MATH: 0335.60034
    Digital Object Identifier: doi:10.1214/aos/1176342871
    Project Euclid: euclid.aos/1176342871
  • Berliner, L. M. and Lu, Z.-Q. (1999). "Markov switching time series models with application to a daily runoff series." Water Resources Research, 35: 523–534.
  • Billio, M., Monfort, A., and Robert, C. P. (1999). "Bayesian estimation of switching ARMA" models. Journal of Econometrics, 93: 229–255.
    Mathematical Reviews (MathSciNet): MR1721099
    Zentralblatt MATH: 1070.62515
    Digital Object Identifier: doi:10.1016/S0304-4076(99)00010-X
  • Blackwell, D. and MacQueen, J. (1973). "Ferguson distributions via Pólya urn schemes." Annals of Statistics, 1: 353–355.
    Mathematical Reviews (MathSciNet): MR362614
    Digital Object Identifier: doi:10.1214/aos/1176342372
    Project Euclid: euclid.aos/1176342372
  • Chib, S. (1996). "Calculating posterior distributions and modal estimates in Markov mixture models." Journal of Econometrics, 75: 79–97.
    Mathematical Reviews (MathSciNet): MR1414504
    Zentralblatt MATH: 0864.62010
    Digital Object Identifier: doi:10.1016/0304-4076(95)01770-4
  • Connor, R. and Mosimann, J. (1969). "Concepts of independence for proportions with a generalization of the Dirichlet distribution." Journal of the American Statistical Association, 64: 194–206.
    Mathematical Reviews (MathSciNet): MR240895
    Zentralblatt MATH: 0179.24101
  • Ferguson, T. (1973). "A Bayesian analysis of some nonparametric problems." Annals of Statistics, 1: 209–230.
    Mathematical Reviews (MathSciNet): MR350949
    Zentralblatt MATH: 0255.62037
    Digital Object Identifier: doi:10.1214/aos/1176342360
    Project Euclid: euclid.aos/1176342360
  • Frühwirth-Schnatter, S. (2006). Finite Mixture and Markov Switching Models. Springer.
    Mathematical Reviews (MathSciNet): MR2265601
  • Gelfand, A. E. and Kottas, A. (2002). "A Computational Approach for Full Nonparametric Bayesian Inference under Dirichlet Process Mixture Models." Journal of Computational and Graphical Statistics, 11: 289–305.
    Mathematical Reviews (MathSciNet): MR1938136
    Digital Object Identifier: doi:10.1198/106186002760180518
  • Goldfeld, S. M. and Quandt, R. E. (1973). "A Markov model for switching regression." Journal of Econometrics, 1: 3–16.
    Zentralblatt MATH: 0294.62087
  • Hamilton, J. D. (1989). "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle." Econometrica, 57: 357–384.
    Mathematical Reviews (MathSciNet): MR996941
    Digital Object Identifier: doi:10.2307/1912559
    Zentralblatt MATH: 0685.62092
  • Hare, S. R. and Mantua, N. J. (2000). "Empirical evidence for North Pacific regime shifts in 1977 and 1989." Progress in Oceanography, 47: 103–145.
  • Hughes, J. P. and Guttorp, P. (1994). "A class of stochastic model for relating synoptic atmospheric patters to regional hydrologic phenomena." Water Resources Research, 30: 1535–1546.
  • Hurn, M., Justel, A., and Robert, C. P. (2003). "Estimating mixtures of regressions." Journal of Computational and Graphical Statistics, 12: 55–79.
    Mathematical Reviews (MathSciNet): MR1977206
    Digital Object Identifier: doi:10.1198/1061860031329
  • Ishwaran, H. and James, L. (2001). "Gibbs sampling methods for stick-breaking priors." Journal of the American Statistical Association, 96: 161–173.
    Mathematical Reviews (MathSciNet): MR1952729
    Zentralblatt MATH: 1014.62006
    Digital Object Identifier: doi:10.1198/016214501750332758
  • Ishwaran, H. and Zarepour, M. (2000). "Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models." Biometrika, 87: 371–390.
    Mathematical Reviews (MathSciNet): MR1782485
    Zentralblatt MATH: 0949.62037
    Digital Object Identifier: doi:10.1093/biomet/87.2.371
  • Jacobson, L. D., Bograd, S. J., Parrish, R. H., Mendelssohn, R., and Schwing, F. B. (2005). "An ecosystem-based hypothesis for climatic effects on surplus production in California sardine and environmentally dependent surplus production models." Canadian Journal of Fisheries and Aquatic Sciences, 62: 1782–1796.
  • MacCall, A. D. (2002). "An hypothesis explaining biological regimes in sardine-producing Pacific boundary current systems." In Climate and fisheries: interacting paradigms, scales, and policy approaches: the IRI-IPRC Pacific Climate-Fisheries Workshop, 39–42. International Research Institute for Climate Prediction, Columbia University.
  • McCulloch, R. E. and Tsay, R. S. (1994). "Statistical Analysis of Economic Time Series via Markov Switching Models." Journal of Time Series Analysis, 15: 523–539.
    Mathematical Reviews (MathSciNet): MR1263893
    Zentralblatt MATH: 0800.62549
    Digital Object Identifier: doi:10.1111/j.1467-9892.1994.tb00188.x
  • McGowan, J. A., Cayan, D. R., and Dorman, L. M. (1998). "Climate-ocean variability and ecosystem response in the Northeast Pacific." Science, 281: 210–217.
  • Müller, P., Erkanli, A., and West, M. (1996). "Bayesian curve fitting using multivariate Normal mixtures." Biometrika, 83: 67–79.
    Mathematical Reviews (MathSciNet): MR1399156
    Zentralblatt MATH: 0865.62029
    Digital Object Identifier: doi:10.1093/biomet/83.1.67
  • Munch, S. B., Kottas, A., and Mangel, M. (2005). "Bayesian nonparametric analysis of stock-recruitment relationships." Canadian Journal of Fisheries and Aquatic Sciences, 62: 1808–1821.
  • Neal, R. (2000). "Markov Chain Sampling Methods for Dirichlet Process Mixture Models." Journal of Computational and Graphical Statistics, 9: 249–265.
    Mathematical Reviews (MathSciNet): MR1823804
  • Quandt, R. E. and Ramsey, J. B. (1978). "Estimating Mixtures of Normal Distributions and switching regressions (with discussion)." Journal of the American Statistical Association, 73: 730–752.
    Mathematical Reviews (MathSciNet): MR521324
    Zentralblatt MATH: 0401.62024
  • Quinn, T. J. I. and Derisio, R. B. (1999). Quantitative Fish Dynamics. Oxford University Press.
  • Robert, C., Celeux, G., and Diebolt, J. (1993). "Bayesian estimation of hidden Markov chains: a stochastic implementation." Statistics & Probability Letters, 16: 77–83.
    Mathematical Reviews (MathSciNet): MR1208503
    Zentralblatt MATH: 0783.62062
  • Scott, S. (2002). "Bayesian methods for hidden Markov models: recursive computing for the 21st century." Journal of the American Statistical Association, 97: 337–351.
    Mathematical Reviews (MathSciNet): MR1963393
    Zentralblatt MATH: 1073.65503
    Digital Object Identifier: doi:10.1198/016214502753479464
  • Sethuraman, J. (1994). "A constructive definition of Dirichlet priors." Statistica Sinica, 4: 639–6650.
    Mathematical Reviews (MathSciNet): MR1309433
    Zentralblatt MATH: 0823.62007
  • Shi, J. Q., Murray-Smith, R., and Titterington, D. M. (2005). "Hierarchical Gaussian process mixtures for regression." Statistics and Computing, 15: 31–41.
    Mathematical Reviews (MathSciNet): MR2137215
    Digital Object Identifier: doi:10.1007/s11222-005-4787-7
  • Shumway, R. H. and Stoffer, D. S. (1991). "Dynamic linear models with switching." Journal of the American Statistical Association, 86: 763–769.
    Mathematical Reviews (MathSciNet): MR1147103
  • Taddy, M. and Kottas, A. (2009). "A Bayesian Nonparametric Approach to Inference for Quantile Regression." Journal of Business and Economic Statistics, to appear.
    Mathematical Reviews (MathSciNet): MR2570089
    Digital Object Identifier: doi:10.1214/09-BA430
    Zentralblatt MATH: 1330.62141
  • Wada, T. and Jacobson, L. D. (1998). "Regimes and stock-recruitment relationships in Japanese sardine, 1951-1995." Canadian Journal of Fisheries and Aquatic Sciences, 55: 2455–2463.

International Society for Bayesian Analysis

Bayesian Analysis

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more