The Annals of Applied Statistics

Small area estimation of the homeless in Los Angeles: An application of cost-sensitive stochastic gradient boosting

Brian Kriegler and Richard Berk

Full-text: Open access


In many metropolitan areas efforts are made to count the homeless to ensure proper provision of social services. Some areas are very large, which makes spatial sampling a viable alternative to an enumeration of the entire terrain. Counts are observed in sampled regions but must be imputed in unvisited areas. Along with the imputation process, the costs of underestimating and overestimating may be different. For example, if precise estimation in areas with large homeless c ounts is critical, then underestimation should be penalized more than overestimation in the loss function. We analyze data from the 2004–2005 Los Angeles County homeless study using an augmentation of L1 stochastic gradient boosting that can weight overestimates and underestimates asymmetrically. We discuss our choice to utilize stochastic gradient boosting over other function estimation procedures. In-sample fitted and out-of-sample imputed values, as well as relationships between the response and predictors, are analyzed for various cost functions. Practical usage and policy implications of these results are discussed briefly.

Article information

Ann. Appl. Stat., Volume 4, Number 3 (2010), 1234-1255.

First available in Project Euclid: 18 October 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Homeless boosting statistical learning costs imputation quantile estimation small area estimation


Kriegler, Brian; Berk, Richard. Small area estimation of the homeless in Los Angeles: An application of cost-sensitive stochastic gradient boosting. Ann. Appl. Stat. 4 (2010), no. 3, 1234--1255. doi:10.1214/10-AOAS328.

Export citation


  • Berk, R. A. (2008). Statistical Learning From a Regression Perspective. Springer, New York.
  • Berk, R. A. and MacDonald, J. (2010). Policing the homeless: An evaluation of efforts to reduce homeless-related crime. Criminology & Public Policy. To appear.
  • Berk, R. A., Brown, L. and Zhao, L. (2010). Statistical inference after model selection. Journal of Quantitative Criminology. To appear. Available at
  • Berk, R. A., Kriegler, B. and Baek, J. H. (2006). Forecasting dangerous inmate misconduct: An application of ensemble statistical procedures. Journal of Quantitative Criminology 22 131–145.
  • Berk, R. A., Kriegler, B. and Ylvisaker, D. (2008). Counting the homeless in Los Angeles County. In Probability and Statistics: Essays in Honor of David A. Freedman. Inst. Math. Statist. Collect. 2 127–141. IMS, Beachwood, OH.
  • Binder, H. (2009). GAMBoost: Generalized additive models by likelihood based boosting. R version 1.1. Available at
  • Bratton, W. and Knobler, P. (1998). The Turnaround: How America’s Top Cop Reversed the Crime Epidemic. Random House, New York.
  • Breiman, L. (1996). Bagging predictors. Machine Learning 26 123–140.
  • Breiman, L. (2001). Random forests. Machine Learning 45 5–32.
  • Breiman, L., Friedman, J. H., Olshen, R. A. and Stone, C. J. (1984). Classification and Regression Trees. Wadsworth, Monterey, CA.
  • Bühlmann, P. and Hothorn, T. (2007). Boosting algorithms: Regularization, prediction and model fitting. Statist. Sci. 22 477–505.
  • Bühlmann, P. and Yu, B. (2003). Boosting with the L2 loss: Regression and classification. J. Amer. Statist. Assoc. 98 324–340.
  • Cordray, D. S. and Pion, G. M. (1991). What’s behind the numbers? Definitional issues in counting the homeless. Housing Policy Debates 2 587–616.
  • Cowen, D. D. (1991). Estimating census and survey undercounts through multiple service contacts. Housing Policy Debates 2 869–882.
  • Culp, M. (2006). ada: Performs boosting algorithms for a binary response. R package 2.0-1. Available at
  • Culp, M., Michailidis, G. and Johnson, K. (2006). ada: An R package for stochastic boosting. Journal of Statistical Software 17.
  • Fan, W., Stolfo, S. J., Zhang, J. and Chan, P. K. (1999). Adacost: Misclassification cost-sensitive boosting. In Machine Learning: Proceedings of the Sixteenth International Conference 97–105. Bled, Slovenia.
  • Freund, Y. and Schapire, R. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55 119–139.
  • Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Ann. Statist. 29 1189–1232.
  • Friedman, J. H. (2002). Stochastic gradient boosting. Comput. Statist. Data Anal. 38 367–378.
  • Friedman, J. H., Hastie, T. and Tibshirani, R. (2000). Additive logistic regression: A statistical view of boosting. Ann. Statist. 28 337–374.
  • Harcourt, B. E. (2005). Policing L.A.’s skid row: Crime and real estate development in downtown Los Angeles (an experiment in real time). Univ. Chicago Legal Forum 2005 325–404.
  • Hastie, T., Tibshirani, R. and Friedman J. (2001). The Elements of Statistical Learning. Springer, New York.
  • Hothorn, T. (2009). mboost: Model-based boosting. R package 1.1-2. Available at
  • Hyndman, R. J. and Fan, Y. (1996). Sample quantiles in statistical packages. Amer. Statist. 50 361–365.
  • Koegel, P., Burnam, A. and Farr, R. K. (1988). The prevalence of specific psychiatric disorders among homeless individuals in the inner-city of Los Angeles. Archives of General Psychiatry 45 1085–1092.
  • Koenker, R. (2005). Quantile Regression. Cambridge Univ. Press, New York.
  • Koenker, R. (2009). quantreg: Quantile regression. R package 4.38. Available at
  • Kushel, M. B., Evans, J. L., Perry, S., Robertson, M. J. and Moss, A. R. (2003). No door to lock: Victimization among homeless and marginally housed persons. Arch. Intern. Med. 163 2492–2499.
  • Leeb, H. and Pötscher, B. M. (2005). Model selection and inference: Facts and fiction. Econom. Theory 21 21–59.
  • Leeb, H. and Pötscher, B. M. (2006). Can one estimate the conditional distribution of post-model-selection estimators? Ann. Statist. 34 2554–2591.
  • Lopez, S. (2005). Demons are winning on Skid Row. Los Angeles Times, October 16.
  • Madigan, D. and Ridgeway, G. (2004). Discussion of “Least angle regression” by B. Efron et al. Ann. Statist. 32 465–469.
  • Magnano, P. F. and Blasi, G. (2007). Stuck on Skid Row. Los Angeles Times (Opinion Section), October 29.
  • Mease, D. and Wyner, A. (2008). Evidence contrary to the statistical view of boosting. J. Mach. Learn. Res. 9 131–156.
  • Mease, D., Wyner, A. and Buja, A. (2007). Cost-weighted boosting with jittering and over/under-sampling: JOUS-boost. J. Mach. Learn. Res. 8 409–439.
  • Meinshausen, N. (2006). Quantile regression forests. J. Mach. Learn. Res. 7 983–999.
  • Rao, J. N. K. (2003). Small Area Estimation. Wiley, Hoboken, NJ.
  • Ridgeway, G. (2007). gbm: Generalized boosted regression models. R package 1.6-3. Available at
  • Rossi, P. H. (1989). Down and Out in America: The Origins of Homelessness. Univ. Chicago Press, Chicago.
  • Rossi, P. H. (1991). Strategies for homeless research in the 1990s. Housing Policy Debates 2 1029–1055.
  • Schapire, R. E. (1999). A brief introduction to boosting. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence 2 1401–1406. Morgan Kaufmann, San Francisco, CA.
  • Sexton, J. (2009) gbev: Gradient boosted regression trees with errors-in-variables. R package 0.1.1. Available at
  • Takeuchi, I., Le, Q. V., Sears, T. D. and Smola, A. J. (2006). Nonparametric quantile regression. J. Mach. Learn. Res. 7 1231–1264.
  • Ting, K. M. (2000). A comparative study of cost-sensitive boosting algorithms. In Proceedings of the Seventeenth International Conference on Machine Learning 983–990. Morgan Kaufmann, San Francisco, CA.
  • Wilson, J. Q. and Kelling, G. L. (1982). Broken windows: The police and neighborhood safety. Atlantic Monthly March 29–38.
  • Wright, J. D. and Devine, J. A. (1992). Counting the homeless: The Census Bureau’s ‘S-Night’ in five U.S. cities. Evaluation Review 16 355–364.
  • Zhang, T. and Yu, B. (2005). Boosting with early stopping: Convergence and consistency. Ann. Statist. 33 1538–1579.