Annals of Applied Statistics
- Ann. Appl. Stat.
- Volume 14, Number 3 (2020), 1182-1206.
Active matrix factorization for surveys
Amid historically low response rates, survey researchers seek ways to reduce respondent burden while measuring desired concepts with precision. We propose to ask fewer questions of respondents and impute missing responses via probabilistic matrix factorization. A variance-minimizing active learning criterion chooses the most informative questions per respondent. In simulations of our matrix sampling procedure on real-world surveys as well as a Facebook survey experiment, we find active question selection achieves efficiency gains over baselines. The reduction in imputation error is heterogeneous across questions and depends on the latent concepts they capture. Modeling responses with the ordered logit likelihood improves imputations and yields an adaptive question order. We find for the Facebook survey that potential biases from order effects are likely to be small. With our method, survey researchers obtain principled suggestions of questions to retain and, if desired, can automate the design of shorter instruments.
Ann. Appl. Stat., Volume 14, Number 3 (2020), 1182-1206.
Received: June 2019
Revised: December 2019
First available in Project Euclid: 18 September 2020
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zhang, Chelsea; Taylor, Sean J.; Cobb, Curtiss; Sekhon, Jasjeet. Active matrix factorization for surveys. Ann. Appl. Stat. 14 (2020), no. 3, 1182--1206. doi:10.1214/20-AOAS1322. https://projecteuclid.org/euclid.aoas/1600454862
- Supplementary material. We provide proofs, additional simulation and experimental results, robustness checks for the active ordering, a two-dimensional illustration of the active strategy, and exploratory work incorporating side information.
- Replication code. This archive contains the CCES datasets, simulation code and analysis notebooks. It includes the implementation of the ordered logit model.