Bayesian Analysis

Latent Marked Poisson Process with Applications to Object Segmentation

Sindhu Ghanta, Jennifer G. Dy, Donglin Niu, and Michael I. Jordan

Full-text: Open access


In difficult object segmentation tasks, utilizing image information alone is not sufficient; incorporation of object shape prior models is necessary to obtain competitive segmentation performance. Most formulations that incorporate both shape and image information are in the form of energy functional optimization problems. This paper introduces a Bayesian latent marked Poisson process for segmenting multiple objects in an image. The model takes both shape and image feature/appearance into account—it generates object locations from a spatial Poisson process, then generates shape parameters from a shape prior model as the latent marks. Inferentially, this partitions the image: pixels inside objects are assumed to be generated from an object observation/appearance model and pixels outside objects come from a background model. The Poisson process provides (non-homogeneous) spatial priors for object locations and the marks allow the incorporation of shape priors. We develop a hybrid Gibbs sampler that addresses the variation in model order and nonconjugacy that arise in this setting and we present experimental results on synthetic images and two diverse domains in real images: cell segmentation in biological images and pedestrian and car detection in traffic images.

Article information

Bayesian Anal., Volume 13, Number 1 (2018), 85-113.

First available in Project Euclid: 26 November 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

spatial Poisson process segmentation Bayesian nonparametrics object detection

Creative Commons Attribution 4.0 International License.


Ghanta, Sindhu; Dy, Jennifer G.; Niu, Donglin; Jordan, Michael I. Latent Marked Poisson Process with Applications to Object Segmentation. Bayesian Anal. 13 (2018), no. 1, 85--113. doi:10.1214/16-BA1035.

Export citation


  • Adams, R., Murray, I., and MacKay, D. (2009). “Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities.” In Proceedings of the 26th Annual International Conference on Machine Learning, 9–16. NY, USA: ACM.
  • Baddeley, A. (2007). Stochastic Geometry: Lectures given at the C.I.M.E. summer school held in Martina Franca, Italy, September 13–18, 2004, chapter Spatial point processes and their applications, 1–75. Springer.
  • Baddeley, A. J. and Van Lieshout, M. N. M. (1992). “Object recognition using Markov spatial processes.” In International Conference on Pattern Recognition, 136–139. The Hague, The Netherlands.
  • Baddeley, A. J. and Van Lieshout, M. N. M (1993). “Stochastic geometry models in high-level vision.” Journal of Applied Statistics, 20(5–6): 231–256.
  • Bhattacharya, A. and Dunson, D. B. (2010). “Nonparametric Bayesian density estimation on manifolds with applications to planar shapes.” Biometrika, 97: 851–865.
  • Blei, D. M. and Frazier, P. I. (2011). “Distance dependent Chinese restaurant processes.” Journal of Machine Learning Research, 12: 2461–2488.
  • Chan, T. and Zhu, W. (2005). “Level set based shape prior segmentation.” In IEEE Conference on Computer Vision and Pattern Recognition, 1164–1170. San Diego, CA.
  • Coelho, L. P., Shariff, A., and Murphy, R. F. (2009). “Nuclear segmentation in microscope cell images: A hand-segmented dataset and comparison of algorithms.” In IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 518–521. Boston, MA.
  • Cootes, T. F., Taylor, C. J., Cooper, D. H., and Graham, J. (1995). “Active shape models-their training and application.” Computer Vision and Image Understanding, 61(1): 38–59.
  • Cremers, D., Rousson, M., and Deriche, R. (2007). “A Review of statistical approaches to level set segmentation: Integrating color, texture, motion and shape.” International Journal of Computer Vision, 72(2): 195–215.
  • Dalal, N. and Triggs, B. (2005). “Histograms of oriented gradients for human detection.” In IEEE Conference on Computer Vision and Pattern Recognition, volume 1, 886–893. San Diego, CA.
  • Descombes, X. and Zerubia, J. (2002). “Marked point processes in image analysis.” IEEE Signal Processing Magazine, 19(5): 77–84.
  • Dufour, A., Shinin, V., Tajbakhsh, S., Guillen-Aghion, N., Olivo-Marin, J.-C., and Zimmer, C. (2005). “Segmenting and tracking fluorescent cells in dynamic 3-D microscopy with coupled active surfaces.” IEEE Transactions on Image Processing, 14(9): 1396–1410.
  • Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. (2010). “The pascal visual object classes (VOC) challenge.” International Journal of Computer Vision, 88(2): 303–338.
  • Felzenszwalb, P. F. and Huttenlocher, D. P. (2004). “Efficient graph-based image segmentation.” International Journal of Computer Vision, 59(2): 167–181.
  • Forgy, E. (1965). “Cluster analysis of multivariate data: Efficiency vs. interpretability of classifications.” Biometrics, 21: 768.
  • Ge, W. and Collins, R. T. (2009). “Marked point processes for crowd counting.” In IEEE Conference on Computer Vision and Pattern Recognition, 2913–2920. Miami, FL.
  • Gelfand, A. E., Le, S., and Carlin, B. (2009). “Analysis of marked point patterns with spatial and nonspatial covariate information.” Annals of Applied Statistics, 3: 943–962.
  • Geman, S. and Geman, D. (1984). “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6): 721–741.
  • Gonzalez, J., Low, Y., Gretton, A., and Guestrin, C. (2011). “Parallel Gibbs sampling: From colored fields to thin junction trees.” In Artificial Intelligence and Statistics. Ft. Lauderdale, FL.
  • Green, P. J. and Mira, A. (2001). “Delayed rejection in reversible jump Metropolis–Hastings.” Biometrika, 88(4): 1035–1053.
  • Griffin, G., Holub, A., and Perona, P. (2007). “Caltech-256 object category dataset.” Technical Report 7694, California Institute of Technology.
  • He, X., Zemel, R., and Carreira-Perpinan, M. (2004). “Multiscale conditional random fields for image labeling.” In IEEE Conference on Computer Vision and Pattern Recognition, volume 2, II-695–II-702. Washington, DC, USA.
  • Huang, X. and Metaxas, D. N. (2008). “Metamorphs: Deformable shape and appearance models.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(8): 1444–1459.
  • Kumar, M. P., Torr, P. H. S., and Zisserman, A. (2005). “OBJ CUT.” In IEEE Conference on Computer Vision and Pattern Recognition, 18–25. San Diego, CA.
  • Lafarge, F., Gimel’farb, G., and Descombes, X. (2010). “Geometric feature extraction by a multi-marked point process.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9): 1597–1609.
  • Leventon, M. E., Grimson, W. E. L., and Faugeras, O. (2000). “Statistical shape influence in geodesic active contours.” In IEEE Conference on Computer Vision and Pattern Recognition, 316–323. Hilton Head, SC.
  • Lou, X., Kang, M., Xenopoulos, P., Muñoz Descalzo, S., and Hadjantonakis, A.-K. (2014). “A rapid and efficient 2D/3D nuclear segmentation method for analysis of early mouse embryo and stem cell image data.” Stem Cell Reports, 2(3): 382–397.
  • Macqueen, J. (1967). “Some methods for classifications and analysis of multivariate observations.” IN Proc. Symp. Mathematical Statistics and Probability, 5th, volume 1, 281–297, Berkeley.
  • Marr, D. (1982). Vision: A Computational Investigation Into the Human Representation and Processing of Visual Information. San Francisco, CA: Freeman and Co.
  • Moller, J., Pettitt, A. N., Reeves, R. W., and Berthelsen, K. K. (2006). “An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants.” Biometrika, 93(2): 451–458.
  • Moller, J., Syversveen, A. R., and Waagepetersen, R. P. (1998). “Log Gaussian Cox processes.” Scandinavian Journal of Statistics, 25(3): 451–482.
  • Ng, A. Y., Jordan, M. I., and Weiss, Y. (2002). “On Spectral clustering: Analysis and an algorithm.” In Dietterich, T., Becker, S., and Ghahramani, Z. (eds.), Advances in Neural Information Processing Systems 14, 849–856. MIT Press.
  • Orbanz, P. and Buhmann, J. M. (2008). “Nonparametric Bayesian image segmentation.” International Journal of Computer Vision, 77(1–3): 25–45.
  • Pollard, J. (1971). “On distance estimators of density in randomly distributed forests.” Biometrics, 27: 991–1002.
  • Rand, W. M. (1971). “Objective criteria for the evaluation of clustering methods.” Journal of the American Statistical Association, 66(336): 846–850.
  • Ren, L., Du, L., Carin, L., and Dunson, D. B. (2011). “Logistic stick-breaking process.” Journal of Machine Learning Research (JMLR), 12: 203–239.
  • Rotondi, R. and Varini, E. (2003). “Bayesian analysis of a marked point process: Application in seismic hazard assessment.” Statistical Methods and Applications, 12(1): 79–92.
  • Rue, H. and Hurn, M. A. (1999). “Bayesian object identification.” Biometrika, 86(3): 649–660.
  • Shotton, J., Winn, J., Rother, C., and Criminisi, A. (2006). “Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation.” In 9th European Conference on Computer Vision, 1–15. Graz, Austria: Springer.
  • Skellam, J. (1952). “Studies in statistical ecology: I. Spatial pattern.” Biometrika, 39: 346–362.
  • Smith, A. F. M. and Gelfand, A. E. (1992). “Bayesian statistics without tears: A sampling–resampling perspective.” The American Statistician, 46(2): 84–88.
  • Sudderth, E. and Jordan, M. I. (2009). “Shared segmentation of natural scenes using dependent Pitman–Yor processes.” In Koller, D., Schuurmans, D., Bengio, Y., and Bottou, L. (eds.), Advances in Neural Information Processing Systems 21, 1585–1592. Curran Associates, Inc.
  • Taddy, M. A. (2010). “Autoregressive mixture models for dynamic spatial Poisson processes: Application to tracking intensity of violent crime.” Journal of the American Statistical Association, 105(492): 1403–1417.
  • Terenin, A., Simpson, D., and Draper, D. (2015). “Asynchronous distributed Gibbs sampling.” arXiv:1509.08999.
  • Vese, L. A. and Chan, T. F. (2002). “A multiphase level set framework for image segmentation using the Mumford and Shah model.” International Journal of Computer Vision, 50(3): 271–293.
  • Vu, N. and Manjunath, B. S. (2008). “Shape prior segmentation of multiple objects with graph cuts.” In IEEE Conference on Computer Vision and Pattern Recognition, 1–8. Anchorage, AK.
  • Wang, X., Wang, M., and Li, W. (2013). “Scene-specific pedestrian detection for static video surveillance.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 99: 1.
  • Xiao, S., Kottas, A., and Sansó, B. (2015). “Modeling for seasonal marked point processes: An analysis of evolving hurricane occurrences.” Annals of Applied Statistics, 9(1): 353–382.
  • Zhou, Z., Matteson, D. S., Woodard, D. B., Henderson, S. G., and Micheas, A. C. (2014). “A spatio-temporal point process model for ambulance demand.” arXiv:1401.5547.
  • Zhu, S., Wu, Y., and Mumford, D. (1998). “Filters, random fields and maximum entropy (FRAME): Towards a unified theory for texture modeling.” International Journal of Computer Vision, 27(2): 107–126.