Bernoulli

  • Bernoulli
  • Volume 19, Number 5A (2013), 1790-1817.

Theory of self-learning $Q$-matrix

Jingchen Liu, Gongjun Xu, and Zhiliang Ying

Full-text: Open access

Abstract

Cognitive assessment is a growing area in psychological and educational measurement, where tests are given to assess mastery/deficiency of attributes or skills. A key issue is the correct identification of attributes associated with items in a test. In this paper, we set up a mathematical framework under which theoretical properties may be discussed. We establish sufficient conditions to ensure that the attributes required by each item are learnable from the data.

Article information

Source
Bernoulli Volume 19, Number 5A (2013), 1790-1817.

Dates
First available in Project Euclid: 5 November 2013

Permanent link to this document
https://projecteuclid.org/euclid.bj/1383661203

Digital Object Identifier
doi:10.3150/12-BEJ430

Mathematical Reviews number (MathSciNet)
MR3129034

Zentralblatt MATH identifier
1294.68118

Keywords
classification model cognitive assessment consistency diagnostic $Q$-matrix self-learning

Citation

Liu, Jingchen; Xu, Gongjun; Ying, Zhiliang. Theory of self-learning $Q$-matrix. Bernoulli 19 (2013), no. 5A, 1790--1817. doi:10.3150/12-BEJ430. https://projecteuclid.org/euclid.bj/1383661203


Export citation

References

  • [1] Chiu, C.Y., Douglas, J.A. and Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika 74 633–665.
  • [2] de la Torre, J. (2008). An empirically-based method of $Q$-matrix balidation for the DINA model: Development and applications. Journal of Educational Measurement 45 343–362.
  • [3] de la Torre, J. (2008). The generalized DINA model. In International Meeting of the Psychometric Society, Durham, NH.
  • [4] de la Torre, J. and Douglas, J.A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika 69 333–353.
  • [5] DiBello, L.V., Stout, W.F. and Roussos, L. (1995). Unified cognitive psychometric assessment likelihood-based classification techniques. In Cognitively Diagnostic Assessment 361–390. Hillsdale, NJ: Erlbaum.
  • [6] Gelman, A., Jakulin, A., Pittau, M.G. and Su, Y.S. (2008). A weakly informative default prior distribution for logistic and other regression models. Ann. Appl. Stat. 2 1360–1383.
  • [7] Hartz, S.M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Ph.D. dissertation, Univ. Illinois, Urbana–Champaign.
  • [8] Henson, R. and Douglas, J. (2005). Test construction for cognitive diagnosis. Appl. Psychol. Meas. 29 262–277.
  • [9] Henson, R., Roussos, L., Douglas, J. and He, X. (2008). Cognitive diagnostic attribute-level discrimination indices. Appl. Psychol. Meas. 32 275–288.
  • [10] Henson, R.A. and Templin, J.L. (2005). Hierarchical log-linear modeling of the skill joint distribution. Technical report, External Diagnostic Research Group.
  • [11] Junker, B.W. (1999). Some statistical models and computational methods that may be useful for cognitively-relevant assessment. Technical report. Available at http://www.stat.cmu.edu/~brian/nrc/cfa/documents/final.pdf.
  • [12] Junker, B.W. and Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Appl. Psychol. Meas. 25 258–272.
  • [13] Leighton, J.P., Gierl, M.J. and Hunka, S.M. (2004). The attribute hierarchy model for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement 41 205–237.
  • [14] Liu, Y., Douglas, J.A. and Henson, R. (2007). Testing person fit in cognitive diagnosis. In The Annual Meeting of the National Council on Measurement in Education (NCME), Chicago, IL.
  • [15] Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika 64 187–212.
  • [16] Reckase, M.D. (1990). Unidimensional data from multidimensional tests and multidimensional data from unidimensional tests. In The Annual Meeting of the American Educational Research Association, Boston, MA, April 16–20.
  • [17] Reckase, M.D. (2009). Multidimensional Item Response Theory. New York: Springer.
  • [18] Roussos, L., DiBello, L.V., Stout, W., Hartz, S., Henson, R. and Templin, J. (2007). The fusion model skills diagnosis system. In Cognitively Diagnostic Assessment for Education: Theory and Practice (J. P. Leighton and M. J. Gierl, eds.) 275–318. New York: Cambridge Univ. Press.
  • [19] Roussos, L.A., Templin, J.L. and Henson, R.A. (2007). Skills diagnosis using IRT-based latent class models. Journal of Educational Measurement 44 293–311.
  • [20] Rupp, A.A. (2002). Feature selection for choosing and assembling measurement models: A building-block-based organization. Psychometrika 2 311–360.
  • [21] Rupp, A.A. and Templin, J.L. (2008). Effects of $Q$-matrix misspecification on parameter estimates and misclassification rates in the DINA model. Educational and Psychological Measurement 68 78–98.
  • [22] Rupp, A.A. and Templin, J.L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement: Interdisciplinary Research and Perspective 6 219–262.
  • [23] Rupp, A.A., Templin, J.L. and Henson, R.A. (2010). Diagnostic Measurement: Theory, Methods, and Applications. New York: Guilford Press.
  • [24] Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464.
  • [25] Stout, W. (2007). Skills diagnosis using IRT-based continuous latent trait models. Journal of Educational Measurement 44 313–324.
  • [26] Tatsuoka, C. (2002). Data analytic methods for latent partially ordered classification models. J. Roy. Statist. Soc. Ser. C 51 337–350.
  • [27] Tatsuoka, K.K. (1983). Rule space: An approch for dealing with misconceptions based on item response theory. Journal of Educational Measurement 20 345–354.
  • [28] Tatsuoka, K.K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach. Journal of Educational Statistics 12 55–73.
  • [29] Tatsuoka, K.K. (2009). Cognitive Assessment: An Introduction to the Rule Space Method. Florence, KY: Routledge.
  • [30] Templin, J., He, X., Roussos, L. and Stout, W. (2003). The pseudo-item method: A simple technique for analysis of polytomous data with the fusion model. Technical report, External Diagnostic Research Group.
  • [31] Templin, J.L. (2006). CDM: Cognitive diagnosis modeling with Mplus. Computer software. Available at http://jtemplin.coe.uga.edu/research/.
  • [32] Templin, J.L. and Henson, R.A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychol. Methods 11 287–305.
  • [33] von Davier, M. (2005). A general diagnosis model applied to language testing data. Research report, Educational Testing Service.