The Electronic Journal of Information Systems Evaluation provides critical perspectives on topics relevant to Information Systems Evaluation, with an emphasis on the organisational and management implications

Abstract

In today's business world, Small‑to‑Medium enterprises (SMEs) increasingly join their larger counterparts in regarding use of Information Technology (IT) and Information Systems (IS) as fundamental to business operations. For SMEs, investment in packaged software that has not been customized to individual enterprise needs, allows ready access to much of the IT function enjoyed by their larger counterparts. However, given these systems are not exclusively tailored to the enterprise and further given the collective nature of the work‑place in these enterprises, the likelihood increases for work‑arounds and unexpected usage to occur to manage enterprise needs. Studies that explore system use typically focus on individual use. Using an interpretive case study approach, this study considers users of a common system in individually owned SMEs to explore evidence of collective, innovative, configural (CIC) use, the causes of this and its impact on fellow workers. Results provide insight into the role of systems as dynamic business tools and show that despite impacts on financial and operational reporting, CIC use occurs for reasons of operational efficiency and also out of frustration with system functionality. This provides some insight into attitudes concerning Use and Net Benefits in the IS Success Model, which in turn informs system evolution.

Abstract

This study aimed at evaluating the applicability of the Theory of Planned Behaviour (TPB) model in predicting user acceptance of online video services. Few studies have applied the TPB model within this context, even though the model has proven to be effective in predicting technology adoption. Validating the TPB model would improve the understanding of both academics and practitioners of the most influential antecedents of user acceptance. Past studies have demonstrated the importance of integrating user needs and behaviour as a requirement for building successful user‑centric online services. Structural equation modelling was used as the main statistical procedure for data analysis. The results of the study confirmed that the TPB model was viable in predicting user acceptance of online video services. The findings also revealed that perceived behavioural control was the highest contributor to predicting intention to use online video services. Attitude toward use and subjective norm were found to have moderate predictive power, mostly because online video services present obvious benefits to users and are consumed privately.

Abstract

The objective of this investigation was to determine whether the analytical hierarchy process algorithm is suitable for the evaluation of software by evaluators with little Information Technology experience. The scope of the research was the evaluation of two free and open source e‑learning systems at the Open University of Tanzania using 33 stakeholders with diverse levels of Information Technology experience. Both quantitative and qualitative research methods were used. The qualitative methods comprised participative observation and interviews. Questionnaires and the analytical hierarchy process, a multiple‑criteria decision‑ making algorithm, represented the quantitative methods. The results showed that of the two e‑learning systems evaluated, Moodle was preferred over ATutor. Furthermore it was found that the analytical hierarchy process algorithm is appropriate for the evaluation of software in a situation where Information Technology experience is limited. It is anticipated that the paper contributes to the theory and practise of decision making in developing countries such as Tanzania.

Abstract

In evaluating the performance of electronic marketplaces, research has focussed on the impact of factors such as IT, marketplace process design and competition. However, such research has neglected the impact of ownership upon electronic marketplace performance. This paper explores the issue of electronic marketplace ownership and contributes to the literature by revealing four distinct aspects of ownership of electronic marketplaces; investor structure, investor objectives, investor commitment and governance efficiency. Using a multiple case approach, the paper evaluates the performance of seven electronic marketplaces in order to determine the relationship between marketplace performance and ownership. The study reveals a multitude of investor objectives for their marketplace investment, broadly categorised as; transactional, financial or fear. The analysis reveals that investor objectives impact upon investor commitment levels with those investors interested in the transactional benefits of electronic marketplaces being most committed. Analysis also reveals that investor objectives impact upon how efficiently a marketplace is governed. Both investor commitment levels and governance efficiency in turn impact upon electronic marketplace performance in terms of the volumes traded on the electronic marketplace, electronic marketplace adoption levels and electronic marketplace revenues and profitability.

Abstract

Attempts to resolve the problems in software development have concentrated on the tools and methodologies used, despite an acceptance by many that it is a sociological problem. An example of this is the procedures and processes surrounding evaluations within projects, yet ultimately it depends on individuals more than process. This paper examines one of the sociological factors inherent in a software development team to determine its impact on evaluation within a project. Social loafing occurs where individual members of a team demonstrate a tendency not to work as hard as they could or should. This "slacking off" occurs because the team provides a degree of anonymity – the individual feels their lack of work will be hidden from evaluation within the overall output of the team. Some authors purport that Agile Software development teams have low incidences of social loafing (though these are opinions rather than research findings); the contrary can also be argued. An examination of the philosophy behind Agile Software Development, demonstrated by the Agile Manifesto, highlighted the possibility of occurrences of social loafing brought about by the Agile values. Agile espouses the importance of cohesive teams, the empowerment of these teams, and the collective ownership and self‑ evaluation of work by the team. These values map onto factors which are described as affecting social loafing. An investigation of two teams over an eight month period examined if the Agile values could lead to incidences of social loafing, specifically when their work is being evaluated The investigation determined that the opposite was actually the case. This paper then goes on to determine why the findings go against the initial hypothesis and to show the impact this can have on those evaluating software development projects.

Abstract

The Information and Communications Technology (ICT) evaluation literature now spans several decades. Nonetheless, evidence continues to suggest that there remains a lack of formal ICT evaluation practices within organisations. Several challenges exist, not least the social and political contexts within which evaluation takes place and limitations in existing evaluation techniques. However, while ICT evaluation exercises have spanned many fields of study, an in‑depth review of the ICT evaluation literature revealed that there is a paucity of ICT evaluation studies within the Higher Education sector. The 14 Irish Institutes of Technology (IoTs) have recently undergone an extensive transformation of their ICT systems. A national project launched by the Department of Education and Science and the Council of Directors of the IoTs performed a nationwide implementation of a suite of integrated Information Systems for library, human resources, finance and student management functions in order to standardise the ICT systems of the IoT sector. Yet, at the time of research, no formal evaluation of this project had been completed. This paper advances the body of ICT evaluation knowledge in the tertiary education sector through evaluating the impact of the Student MIS implementation within the IoTs. The research study was interpretive in nature; case studies based on multiple evidence sources were conducted in five IoTs. Analysis of the evidence led to the distillation of 15 findings on the Student MIS implementation which were centred on five key project areas – system selection, system development in the Irish IoTs, system commissioning, ex‑post performance at system start‑up and at the time of research. The 15 findings uncovered either support existing research in the ICT evaluation field or further advance the body of ICT evaluation theoretical knowledge. This paper makes a number of valuable contributions. It enhances understanding of ICT evaluation in tertiary education. It discusses the difficulties involved in operationalising a standard ICT system in multiple diverse organisations and provides lessons with respect to managing the difficulties experienced in large‑scale government projects.