Source: Braz. J. Probab. Stat. Volume 24, Number 3
(2010), 415-433.
The nominal response model (NRM) was proposed by Bock [Psychometrika 37 (1972) 29–51] in order to improve the latent trait (ability) estimation in multiple choice tests with nominal items. When the item parameters are known, expectation a posteriori or maximum a posteriori methods are commonly employed to estimate the latent traits, considering a standard symmetric normal distribution as the latent traits prior density. However, when this item set is presented to a new group of examinees, it is not only necessary to estimate their latent traits but also the population parameters of this group. This article has two main purposes: first, to develop a Monte Carlo Markov Chain algorithm to estimate both latent traits and population parameters concurrently. This algorithm comprises the Metropolis–Hastings within Gibbs sampling algorithm (MHWGS) proposed by Patz and Junker [Journal of Educational and Behavioral Statistics 24 (1999b) 346–366]. Second, to compare, in the latent trait recovering, the performance of this method with three other methods: maximum likelihood, expectation a posteriori and maximum a posteriori. The comparisons were performed by varying the total number of items (NI), the number of categories and the values of the mean and the variance of the latent trait distribution. The results showed that MHWGS outperforms the other methods concerning the latent traits estimation as well as it recoveries properly the population parameters. Furthermore, we found that NI accounts for the highest percentage of the variability in the accuracy of latent trait estimation.
References
Andrade, D. F. and Tavares, H. R. (2005). Item response theory for longitudinal data: Population parameter estimation., Journal of Multivariate Analysis 95 1–22.
Azevedo, C. L. N. (2003). Métodos de estimação na Teoria da Resposta ao Item. Master’s Dissertation, University of São Paulo, in, Portuguese.
Baker, F. B. and Kim, S.-H. (2004)., Item Response Theory: Parameter Estimation Techniques, 2nd ed. Marcel Dekker, New York.
Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories., Psychometrika 37 29–51.
Bock, R. D and Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm., Psychometrika 46 443–459.
Mathematical Reviews (MathSciNet):
MR668311
Bolt, D. M., Cohen, A. S. and Wollack, J. A. (2001). A mixture item response model for multiple choice data., Journal of Educational and Behavorial Statistics 26 381–409.
De Ayala, R. J. (1989). A comparison of the nominal response model and three-parameter logistic model in computerized adaptive testing., Educational and Psychological Measurement 49 789–805.
De Ayala, R. J. (1992). The nominal response model in computerized adaptive testing., Applied Psychological Measurement 16 327–343.
De Ayala, R. J. and Sava-Bolesta, M. (1999). Item Parameter Recovery for the Nominal Response Model., Applied Psychological Measurement 23 3–19.
DeMars, C. E. (2003). Sample Size and the Recovery of Nominal Response Model Item Parameters., Applied psychological measurement 27 275–288.
Gamerman, D. and Lopes, H. F. (2006)., Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. Chapman and Hall, London.
Gelman, A. (2006). Prior distribution for variance parameters in hierarchical models., Bayesian Analysis 1 515–533.
Gelman, A. and Carlin, J. B. and Stern, H. S. and Rubin, D. B. (2004)., Bayesian Data Analysis, 2nd ed. Chapman and Hall, London.
Ghosh, M., Ghosh, A., Chen, M.-H. and Agresti, A. (2000). Noninformative priors for one-parameter item response models., Journal of Statistical Planning and Inference 88 99–115.
Harwell, M., Stone, C. A., Hsu, T.-C. and Kirisci, L. (1996). Monte Carlo studies in item response theory., Applied Psychological Measurement 20 101–125.
Lunn, D. J., Thomas, A., Best, N. and Spiegelhalter, D. (2000). WinBUGS—a Bayesian modelling framework: Concepts, structure, and extensibility., Statistics and Computing 10 325–337.
Patz, R. J. and Junker, B. W. (1999a). A straightforward approach to Markov chain Monte Carlo methods for item response models., Journal of Educational and Behavioral Statistics 24 146–178.
Patz, R. J. and Junker, B. W. (1999b). Applications and extensions of MCMC in IRT: Multiple item types, missing data and rated responses., Journal of Educational and Behavioral Statistics 24 346–366.
Rao, C. R. (2002)., Linear Statistical Inference and Its Applications. Wiley, New York.
Stroud, A. H. and Secrest, D. (1980)., Gaussian Quadrature Formulas. Prentice-Hall, Englewood, NJ.
Mathematical Reviews (MathSciNet):
MR202312
Thissen, D., Chen, W. H. and Bock, D. (2003). In, IRT from SSI (M. Du Toiti, ed.). Scientific Software International, Chicago, IL.
Wollack, J. A., Bolt, D. M., Cohen, A. S. and Lee, Y.-S. (2002). Recovery of item parameters in the nominal response model: A comparison of marginal maximum likelihood estimation and Markov chain Monte Carlo estimation., Applied Psychological Measurement 26 339–352.
