TY - GEN
T1 - β3-IRT
T2 - A New Item Response Model and its Applications
AU - Chen, Yu
AU - Filho, Telmo M Silva
AU - Prudêncio, Ricardo B. C.
AU - Diethe, Tom
AU - Flach, Peter
PY - 2019/3/10
Y1 - 2019/3/10
N2 - Item Response Theory (IRT) aims to assess latent abilities of respondents based on the correctness of their answers in aptitude test items with different difficulty levels. In this paper, we propose the β3-IRT model, which models continuous responses and can generate a much enriched family of Item Characteristic Curves. In experiments we applied the proposed model to data from an online exam platform, and show our model outperforms a more standard 2PL-ND model on all datasets. Furthermore, we show how to apply β3-IRT to assess the ability of machine learning classifiers.This novel application results in a new metric for evaluating the quality of the classifier’s probability estimates, based on the inferred difficulty and discrimination of data instances.
AB - Item Response Theory (IRT) aims to assess latent abilities of respondents based on the correctness of their answers in aptitude test items with different difficulty levels. In this paper, we propose the β3-IRT model, which models continuous responses and can generate a much enriched family of Item Characteristic Curves. In experiments we applied the proposed model to data from an online exam platform, and show our model outperforms a more standard 2PL-ND model on all datasets. Furthermore, we show how to apply β3-IRT to assess the ability of machine learning classifiers.This novel application results in a new metric for evaluating the quality of the classifier’s probability estimates, based on the inferred difficulty and discrimination of data instances.
M3 - Conference contribution
T3 - Proceedings of Machine Learning Research
SP - 1013
EP - 1021
BT - Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS 2019)
A2 - Chaudhuri, Kamalika
A2 - Sugiyama , Masashi
PB - Proceedings of Machine Learning Research
ER -