Probabilistic Knowledge Space and Item-Response-Theories

Current psychological test theory has its origin in the GUTTMAN-model, developed more than 50 years ago. This model allows the linear ranking of persons (regarding their abilities) and test items (regarding their difficulties). Since then, test theory parted into two different directions. On the one hand, based on the RASCH-model and generalized by MOKKEN`s monotone homogeneity model, a family of linear-scale probabilistic models (Item- Response-Theories) arose, on the other hand, starting with AIRASIAN, BART, and KRUS, a family of non-linear deterministic models (Knowledge Space Theory) has been developed. Although there exist scattered approaches to create specific probabilistic non-linear deterministic models, respectively, to create "de-linearized" (linear-scale) probabilistic models, those activities did not arise from the idea of conflating the splitted directions of psychological test theories. Thus, the explicit aim of the current project is the fusion of mentioned splitted directions of theories in order to develop a superior probabilistic test theory that includes the existing models as special cases. The related work is basically of psychological-mathematical nature. In addition, this theoretical work must be accompanied by the development of related software that supports the theoretical model evolution and that enables the application to practical requirements. Software development, therefore, will be based on current technologies and the integration of existing software components. There is a tremendous practical impact of a comprehensive test theory in addition with the related software products: It allows (a) an exact acquisition of personal profiles and item difficulties even in highly complex structures and noisy datasets. Additional, despite the mentioned conditions, it allows (b) an adaptive testing and gathering a person`s exact knowledge and capability profile with comparably low effort. This is important, for example, for a further stage of intercultural knowledge profiling, e.g. by the International Mathematics and Science Studies, that recently attained wide popularity with the results of the last PISA study. Exact measures of knowledge and capability, furthermore, are from importance for (c) personalized interventions, e.g. individual training. Modeled and empirically validated complex structures of knowledge and capability may also be used to optimize (d) the internal structure of single courses up to (e) complete curricula.

Current psychological test theory has its origin in the GUTTMAN-model, developed more than 50 years ago. This model allows the linear ranking of persons (regarding their abilities) and test items (regarding their difficulties). Since then, test theory has parted into two directions. On the one hand, based on the RASCH-model and generalized by MOKKEN`s monotone homogeneity model, a family of linear-scale probabilistic models (Item Response Theory) has emerged. On the other hand, starting with AIRASIAN, BART, and KRUS, a family of non- linear deterministic models (Knowledge Space Theory) has been developed. This project has provided a first important step toward a fusion of the mentioned split directions of test theories. On the one hand, a generalization of Latent Class Analysis combining both discrete and continuous latent variables was considered and exhaustively investigated, which led to first results in a unification of Knowledge Space Theory (with its discrete knowledge states) and Item Response Theory (with its continuous ability parameters). On the other hand, the scope of the linear Mokken axioms and the resulting measurement properties were comprehensively analyzed in the nonlinear context of Knowledge Space Theory; statements about the validity, respectively, invalidity of nonparametric Item Response Theory axioms and measurement properties for established Knowledge Space Theory models were obtained, and conditions were specified under which such axioms and properties can be guaranteed to hold. Closely related is the evaluation of existing software components and their adaptation and expansion according to the needs of mathematical modelling, as well as the development of new software. This project has thus created a basis for a superior general probabilistic test theory that would include the existing two model families as special cases. Such a comprehensive test theory would have a tremendous practical impact. It would, just to name two important implications, allow for (a) an exact acquisition of personal profiles and item difficulties even in complex structures, and (b) an adaptive testing and gathering a person`s exact knowledge and capability profile with comparably low effort. This is important, for example, for a further stage of intercultural knowledge profiling, e.g. by the International Mathematics and Science Studies, that recently attained wide popularity with the results of the last PISA study. More information can be retrieved from http://css.uni-graz.at/projects/pksirt/pksirt.html.