The Annals of Applied Statistics

Intervention analysis with state-space models to estimate discontinuities due to a survey redesign

Jan van den Brakel and Joeri Roels

Full-text: Open access

Abstract

An important quality aspect of official statistics produced by national statistical institutes is comparability over time. To maintain uninterrupted time series, surveys conducted by national statistical institutes are often kept unchanged as long as possible. To improve the quality or efficiency of a survey process, however, it remains inevitable to adjust methods or redesign this process from time to time. Adjustments in the survey process generally affect survey characteristics such as response bias and therefore have a systematic effect on the parameter estimates of a sample survey. Therefore, it is important that the effects of a survey redesign on the estimated series are explained and quantified. In this paper a structural time series model is applied to estimate discontinuities in series of the Dutch survey on social participation and environmental consciousness due to a redesign of the underlying survey process.

Article information

Source
Ann. Appl. Stat., Volume 4, Number 2 (2010), 1105-1138.

Dates
First available in Project Euclid: 3 August 2010

Permanent link to this document
https://projecteuclid.org/euclid.aoas/1280842154

Digital Object Identifier
doi:10.1214/09-AOAS305

Mathematical Reviews number (MathSciNet)
MR2758435

Zentralblatt MATH identifier
1194.62013

Keywords
Intervention analysis response bias structural time series models survey sampling

Citation

van den Brakel, Jan; Roels, Joeri. Intervention analysis with state-space models to estimate discontinuities due to a survey redesign. Ann. Appl. Stat. 4 (2010), no. 2, 1105--1138. doi:10.1214/09-AOAS305. https://projecteuclid.org/euclid.aoas/1280842154


Export citation

References

  • Abraham, B. and Vijayan, K. (1992). Time series analysis for repeated surveys. Comm. Statist. Simulation Comput. 21 893–908.
  • Aitchison, J. (1986). The Statistical Analysis of Compositional Data. Chapman & Hall, London.
  • Akinbami, L. J. and Schoendorf, K. C. (2002). Trends in childhood asthma: Prevalence, health, care utilization and mortality. Pediatrics 110 315–322.
  • Akinbami, L. J., Schoendorf, K. C. and Parker, J. (2003). US childhood asthma prevalence estimates: The impact of the 1997 National Health Interview Survey Redesign. American Journal of Epidemiology 158 99–104.
  • Bailar, B. A. (1975). The effects of rotation group bias on estimates from panel surveys. J. Amer. Statist. Assoc. 70 23–30.
  • Bell, W. R. and Hillmer, S. C. (1990). The time series approach to estimation of periodic surveys. Survey Methodology 16 195–215.
  • Binder, D. A. and Dick, J. P. (1989). Modeling and estimation for repeated surveys. Survey Methodology 15 29–45.
  • Binder, D. A. and Dick, J. P. (1990). A method for the analysis of seasonal ARIMA models. Survey Methodology 16 239–253.
  • Blight, B. J. N. and Scott, A. J. (1973). A stochastic model for repeated surveys. J. Roy. Statist. Soc. Ser. B 35 61–66.
  • Brackstone, G. (1999). Managing data quality in a statistical agency. Survey Methodology 25 139–149.
  • Brunsdon, T. M. and Smith, T. M. F. (1998). The time series analysis of compositional data. Journal of Official Statistics 14 237–253.
  • Caban, A. J., Lee, D. J., Fleming, L. E., Gómez-Marin, O., LeBlanc, W. and Pitman, T. (2005). Obesity in US workers: The National Health Interview Survey, 1986–2002. American Journal of Public Health 95 1614–1622.
  • Cochran, W. G. (1977). Sampling Techniques, 3rd ed. Wiley, New York.
  • De Leeuw, E. (2005). To mix or not to mix data collection modes in surveys. Journal of Official Statistics 21 233–255.
  • Dillman, D. A. and Christian, L. M. (2005). Survey mode as a source of instability in responses across surveys. Field Methods 17 30–52.
  • Dippo, C. S., Kostanich, D. L. and Polivka, A. E. (1994). Effects of methodological change in the Current Population Survey. In Proceedings of the Section on Survey Research Methods 260–262. Amer. Statist. Assoc., Alexandria.
  • Doornik, J. A. (1998). Object-Oriented Matrix Programming Using Ox 2.0. Timberlake Consultants Press, London.
  • Durbin, J. and Koopman, S. J. (2000). Time series analysis of non-Gaussian observations based on state-space models from both classical and Bayesian perspectives (with discussion). J. Roy. Statist. Soc. Ser. B 62 3–56.
  • Durbin, J. and Koopman, S. J. (2001). Time Series Analysis by State Space Methods. Oxford Univ. Press, Oxford.
  • Feder, M. (2001). Time series analysis of repeated surveys: The state-space approach. Statist. Neerlandica 55 182–199.
  • Fowler, F. J. (1996). The redesign of the National Health Interview Survey. Public Health Reports 111 508–511.
  • Harvey, A. C. (1989). Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge Univ. Press, Cambridge.
  • Harvey, A. C. and Chung, C. H. (2000). Estimating the underlying change in unemployment in the UK. J. Roy. Statist. Soc. Ser. A 163 303–339.
  • Harvey, A. C. and Durbin, J. (1986). The effects of seat belt legislation on British road casualties: A case study in structural time series modelling. J. Roy. Statist. Soc. Ser. A 149 187–227.
  • Holbrook, A. L., Green, M. C. and Krosnick, J. A. (2003). Telephone versus face-to-face interviewing of national probability samples with long questionnaires. Public Opinion Quarterly 67 79–125.
  • Kalton, G. and Schuman, H. (1982). The effect of the question on survey responses: A review. J. Roy. Statist. Soc. Ser. A 145 42–73.
  • Kindermann, C. and Lynch, J. (1997). Effects of the redesign on vistimization estimates. Technical report, US Dept. of Justice, Bureau of Justice Statistics. Available at http://www.ojp.usdoj.gov/bjs/abstract/erve.htm.
  • Koopman, S. J. (1997). Exact initial Kalman filtering and smoothing for non-stationary time series models. J. Amer. Statist. Assoc. 92 1630–1638.
  • Koopman, S. J., Shephard, N. and Doornik, J. A. (1999). Statistical algorithms for models in state space using SsfPack 2.2. Econom. J. 2 113–166.
  • Koopman, S. J., Shephard, N. and Doornik, J. A. (2008). SsfPack 3.0: Statistical Algorithms for Models in State Space Form. Timberlake Consultants Press, London.
  • Lind, J. T. (2005). Repeated surveys and the Kalman filter. Econom. J. 8 418–427.
  • Pfeffermann, D. (1991). Estimation and seasonal adjustment of population means using data from repeated surveys. J. Bus. Econom. Statist. 9 163–175.
  • Pfeffermann, D. and Bleuer, S. R. (1993). Robust joint modelling of labour force series of small areas. Survey Methodology 19 149–163.
  • Pfeffermann, D. and Burck, L. (1990). Robust small area estimation combining time series and cross-sectional data. Survey Methodology 16 217–237.
  • Pfeffermann, D., Feder, M. and Signorelli, D. (1998). Estimation of autocorrelations of survey errors with application to trend estimation in small areas. J. Bus. Econom. Statist. 16 339–348.
  • Pfeffermann, D. and Tiller, R. (2006). Small area estimation with state space models subject to benchmark constraints. J. Amer. Statist. Assoc. 101 1387–1397.
  • Rao, J. N. K. and Yu, M. (1994). Small area estimation by combining time series and cross-sectional data. Canad. J. Statist. 22 511–528.
  • Roberts, C. (2007). Mixing modes of data collection in surveys: A methodological review. Review paper NCRM/008, National Centre for Research Methods, City Univ. London.
  • Särndal, C.-E., Swensson, B. and Wretman, J. (1992). Model Assisted Survey Sampling. Springer, New York.
  • Scott, A. J. and Smith, T. M. F. (1974). Analysis of repeated surveys using time series methods. J. Amer. Statist. Assoc. 69 674–678.
  • Scott, A. J., Smith, T. M. F. and Jones, R. G. (1977). The application of time series methods to the analysis of repeated surveys. Internat. Statist. Rev. 45 13–28.
  • Silva, D. B. N. and Smith, T. M. F. (2001). Modelling compositional time series from repeated surveys. Survey Methodology 27 205–215.
  • Tam, S. M. (1987). Analysis of repeated surveys using a dynamic linear model. Internat. Statist. Rev. 55 63–73.
  • Tiller, R. B. (1992). Time series modelling of sample survey data from the U.S. current population survey. Journal of Official Statistics 8 149–166.
  • van den Brakel, J. A. (2008). Design-based analysis of embedded experiments with applications in the Dutch Labour Force Survey. J. Roy. Statist. Soc. Ser. A 171 581–613.
  • van den Brakel, J. A. and Krieg, S. (2009). Estimation of the monthly unemployment rate through structural time series modelling in a rotating panel design. Survey Methodology. 35 117–190.
  • van den Brakel, J. A. and Renssen, R. H. (2005). Analysis of experiments embedded in complex sampling designs. Survey Methodology 31 23–40.
  • van den Brakel, J. A. and Roels, J. (2010). Supplement to “Intervention analysis with state-space models to estimate discontinuities due to a survey redesign.” DOI:10.1214/09-AOAS305SUPP.
  • van den Brakel, J. A., Smith, P. A. and Compton, S. (2008). Quality procedures for survey transitions, experiments, time series and discontinuities. Journal for Survey Research Methods 2 123–141.

Supplemental materials

  • Supplementary material: Software and article supplement to: Intervention analysis with state-space models to estimate discontinuities due to a survey redesign. The supplementary article contains additional information about discontinuities in the target variables about social participation and environmental consciousness that occurred due to the changeover from the PSLC to the SSPEC. It contains a description of the target variables about social participation and environmental consciousness as well as an overview of the observed differences that occurred during the year of the changeover from the PSLC in 2004 to the SSPEC in 2005. Finally, the analysis results using the time series model selected in Section 5.3 are presented for these variables. As an example, the estimated series and the corrected series for three variables are provided. This supplement also contains the Ox-program, used to conduct the intervention analysis with the state-space models developed in this paper. Input files (time series of “contact frequency with neighbors” and “separating chemical waste” and a series with the sample sizes of the surveys for the different time points) are also provided to illustrate the use of the program.