User Acceptance of a Proposed Self-Evaluation and Continuous Assessment System

Article excerpt

Introduction

New issues related to the learning process in the field of higher education, especially on-line or distance evaluation, are currently a hot topic of research, and it is of a great interest due to the growing use of Internet (Ryan et al., 2000; O'Reilly, 2007; Dagger et al., 2007; Munoz-Organero et al., 2010). Basically, the evaluation process consists of determining whether the educational objectives are actually achieved. This fact includes not only the student, but all the elements involved in an educational program (given educational resources, faculty, etc.). Faculty must be able to adapt the learning process to students, strengthening or expanding it as appropriate within the context of the European Higher Education Area (EHEA). In addition to this, if we focus on a distance methodology (Carswell et al., 2000; Morgan et al., 2002;

Schrum et al., 2007), the evaluation process becomes an even more important and indispensable instrument within the learning process. By using the available evaluation elements, faculty determines the chosen learning outcomes and, at the same time, the subject content can be adapted dynamically to students. Since interaction between faculty and students within a distance methodology increases during the evaluation process, performing this in an efficient and correct way is essential in order to achieve the previously proposed objectives, as already stated in (Robles-Gomez et al., 2011; Agudo et al., 2011).

This work describes a solution, based on the service-oriented paradigm, to follow the progress of students and, automatically, assess the students' practical activities. In order to analyze its acceptance, the Technology Acceptance Model (TAM) is adapted for our purposes, since there is a lack of references related to self-evaluation and continuous evaluation service oriented systems. This model is based on the perception of utility and ease of use as key factors when deciding on the use of Information Technologies (ITs). In this sense, TAM questionnaire results for each item will be presented for both students and faculty members.

In order to center this work, the system will only use activities related to the configuration of network services. Its scope is much broader, since this system has been designed and implemented as a modular system, which is independent of the design and implementation of specific activities. In this regard, we focus on the "Network Services Management in Operating Systems" (NetServicesOS) course belonging to the "Communications, Networks, and Content Management" post-graduate program at Spanish University for Distance Education (in Spanish, Universidad Nacional de Educacion a Distancia). In our traditional evaluation approach, once a student finished an activity, he/she proceeded to write a solution document detailing how he/she had solved the requested work, and uploaded it in the virtual platform for its evaluation. After each deadline, faculty could assess the students' activity from this document, although it was difficult to check the correctness of the activity, since the activities of the subject are very practical which involve starting some network services and, also, creating and modifying a set configuration files, among others (Robles-Gomez et al., 2011; Agudo et al., 2011).

Thanks to the use of the service-based system, faculty can track the progress of a large number of students and adapt dynamically the learning/teaching process. In particular, the system is able to automatically assess the practical activities of the subject, detect possible failures within the network services and configuration files, and help faculty to understand the main weaknesses of students during the process of learning from the evaluation reports that the system generates. These features are not possible in traditional Learning Management Systems (LMSs), since they only focus on multiple choice, questions, short answer questions, or similar, but not practical activities, as our case is. …