Patricia Santos PhD Defense

The last 20th of December, Patricia Santos, a member of the Technology-Enhanced Learning section of the GTI, presented her PhD thesis entitled “Computing-Based Testing: conceptual model, implementations and experiments extending IMS QTI”, she obtained the qualification of Suma cum Laude. It has been co-directed by Davinia Hernández-Leo and Josep Blat.

Her work is framed in the research domain of Technology-Enhanced Learning (TEL), and more specifically of Computing-Based Testing (CBT), in which Information and Communication Technologies (ICT) are applied to enhance the assessment of higher-order skills. In particular, the activities and results of her thesis contribute to the design and implementation of advanced scenarios and tools, extending the scope of the IMS Question and Test Interoperability specification.

A number of publications related to the work presented in this dissertation has been reviewed and accepted by the research community: two international JCR journal papers published and one conditionally accepted, two book chapters and five international conference papers, a nomination to the BDigital for Digital Innovation 2011 award, and collaborations with international research projects such as the TENCompetence project (where an IMS award was received in the category of “Most Innovative New Realization of Standards” by the contribution done in the TENCompetence Learning Design Toolkit) and Learn3. Moreover, the QTIMaps case was selected by the ICOPER project as one of its top best practices for assessment and evaluation specifications and standards. These are indicators of the relevance of this research and show the importance of the future work derived from this research.

The abstract of the thesis:

The use of objective tests in Technology Enhanced Learning (TEL) is based on the application of computers to support automatic assessment. Current research in this domain is mainly focused on the design of new question-items, being IMS Question and Test Interoperability (QTI) the recognized de-facto standard. This thesis claims that the domain can be extended with the design of advanced test-scenarios that integrate new interactive contexts for the visualization of question-items and tests, and that consider different types of devices and technologies that enable diverse activity settings. In this context, the dissertation proposes to term the domain as Computing-Based Testing (CBT) instead of Computer-Based Testing because it captures better the new technological support possibilities for testing. Advanced CBT scenarios can increase teachers’ choices in the design of more appropriate tests for their subject areas, enabling the assessment of higher-order skills.

With the aim of modelling an advanced CBT domain that extends the current possibilities of QTI and related work, this thesis provides a set of contributions around three objectives. The first objective deals with proposing a Conceptual Model for the CBT domain considering three main dimensions: the Question-item, the Test and the Activity. To tackle this objective, the thesis presents, on the one hand, a framework to assist in the categorization and design of advanced CBT scenarios and, on the other hand, two models that suggest elements for technologically representing the Test and Question-item dimensions. The models are platform-independent models (PIM) that extend QTI in order to support advanced CBT. Besides, the use of patterns is proposed to complement the modelling of the domain. The second objective seeks to show the relevance, value and applicability of the CBT Conceptual Model through exemplary challenging scenarios and case studies in authentic settings. To this end, the dissertation evaluates the design and implementation of a set of CBT systems and experiments. All the experiments use the proposed CBT Conceptual Model for designing an advanced CBT scenario. For each case the CBT-PIMs serve as the basis for developing a particular CBT-PSM and system. The evaluation results show that the implementations foster educational benefits, enable the assessment of higher-order skills and enhance the students’ motivation. Finally, the third objective is devoted to propose extension paths for QTI. The collection of models proposed in the thesis suggests different extension directions for QTI so as to enable the implementation of advanced questions, tests and activities. The proposed systems and scenarios also represent reference implementation and good practices of the proposed extension paths.