The Electronic Journal of Information Systems Evaluation provides critical perspectives on topics relevant to Information Systems Evaluation, with an emphasis on the organisational and management implications

Abstract

Abstract

Abstract: Understanding the value derived from IT investments and IT enabled operational improvements is difficult, and has been a subject of research and debate among ICT practitioners and academics for many years. This is particularly so because innovat
ive technological developments have supported transformative changes in organizational operational activities. Research continues to investigate approaches to not only understanding the value derived by IT but also to optimizing this value. One of the key
aspects of optimizing IT‑driven value is the requirement to effectively manage risk. The continual evolution of the IT risk landscape requires effective Risk Management (RM) practices for all IT risk areas, such as, but not limited to security, investm
ents, service contracts, data protection and information privacy. Effectively managing these risk areas pose specific concerns from the perspective of Chief Information Officers (CIOs) and Chief Risk Officers (CROs). Hence, significant considerations
should be given to not only the processes involved in assessing, prioritizing, handling and monitoring these risks but also to ensuring the development of an appropriate risk culture and the establishment of effective RM governance structures, to support
effective RM. This paper examines the maturity model/framework approach to improving an organizations IT capabilities, with specific reference to effectively managing IT‑related risks, and increasing value derived over time. A new IT Risk Management mat
urity model is presented; this framework is part of the IT Capability Maturity Framework (IT CMF) which supports value‑driven IT management practices. It was developed by the Innovation Value Institute at the National University of Ireland Maynooth, fol
lowing a design science and open innovation research approach. The IT CMF, consisting of 33 Critical Capabilities, focuses on maturing key activities of the IT organization. The Risk Management Critical Capability presented in this paper enables organizat
ions to determine their IT RM maturity and identify key recommendations in specific areas to improve maturity overtime. Thereafter the paper presents an analysis of the maturity model approach to managing risk, to improving an organizations IT capabiliti
es, and to deriving enterprise‑wide value from more mature IT practices.

Abstract

Abstract: Many small and medium‑sized enterprises (SMEs) turn to consultants for assistance with IS projects, for example, to help the firm select and implement a new system. Prior studies have shown that consultants have a major influence on IS success
for SMEs. However, despite its importance to IS success, relatively little research has focused on the relationship between SMEs and IS consultants and whether this relationship has any impact or influence on the development of IS competences in SMEs. T
his study investigates whether consultants compensate for or enhance an SMEs IS competences during a major IS project. A multiple case study approach was adopted involving SMEs who implemented an Accounting Information System (AIS). The case firms prov
ide evidence that SMEs lack many IS skills and abilities. The study identified a number of competences that are compensated or enhanced by consultants. The major finding of this study is that IS consultants help SMEs overcome a lack of IS competences, rat
her than help develop IS competences within an SME. Managing the relationship between consultant and SME is crucial for SMEs lacking many IS competences.

Abstract

Abstract: In the anticipation of the knowledge economy and the organisational pursuit of knowing what we know modern organisations have endeavoured to achieve varying levels of KM. It has typically been larger organisations that have possessed the econ
omies of scale i.e. the financial resources to pursue this strategy, where they perceive they will lose their market share if they do not follow the trend. Smaller organisations have not had the same luxury. Ironically however, it is smaller organisations
that have successfully managed knowledge for centuries. However there remains an absence of empirical evidence that highlights how SMEs operationalise their approach to KM, particularly in the high‑technology sectors. In view of the current financial ins
tability, never has it been more important to focus on the knowledge capabilities of software SMEs where managing organisational knowledge is essential to the continued success of an SME. Pursuing a qualitative analysis approach using multiple case studie
s in four Irish software SMEs, this study identifies sources of knowledge and occurrences of knowledge activities (KAs) as a means of understanding the firms approach to knowledge management (KM) and how this may be closely aligned to the organisatio
ns greater strategic objectives thus providing them with greater flexibility to deal with environmental uncertainty. At the level of the cases, it was evident that software SMEs leverage KAs to serve their knowledge transfer needs. Unexpectedly, the find
ings from this study indicate that these software SMEs were not good at knowledge creation activity. This may be attributed to the nature of the SME where a small number of key players i.e. founder/manager/head of development assumed responsibility for th
is type of activity. Fundamentally, these software SMEs choose to leverage knowledge and KAs in order to serve the greater needs of the firm such as the need to develop a new software product, improve their customer relationships or ensure their position
as an important cog in a larger organisation.

Abstract

Abstract: The development and expansion of evaluation theory and practice is at the core of several different disciplines. There exist different traditional Information System (IS) evaluation approaches, like experimental, pragmatic, constructivist, plu
ralist and realist IS evaluation. IS evaluation approaches are influenced by the way they address to technology. Recently actor network theory (ANT) and sociomateriality are two influential information systems (IS) entanglement perspectives. Additiona
lly, El Sawy identified three faces of IS views: connection, immersion, and fusion. In terms of IS evaluation approaches, connection and immersion view are the dominant views in which these approaches are positioned. We believe the IS fusion view calls fo
r IS evaluation approaches to be revised. This paper uses the relational emergence theory, based on the philosophy of critical realism to theorize and operationalize the fusion view, as it lacks a theoretical grounding and as well to push forward the trad
itional IS evaluation research approaches. At the core of relational emergence theory is the emergence concept, in which parts are structured by the relations among each other to create an entity as a whole. Based on this, we present and discuss the im
plications for IS evaluation in terms of how to evaluate a process as well as the output of the process. The discussion on IS evaluation is illustrated through an empirical example, drawn on a longitudinal research study within a police organization. This
paper concludes that in the fusion view, the evaluation process shall embrace a holistic perspective. The focus of the evaluation process shall be the emergent entity consisting of IS, users, task and processes structured by means of relationships among
each other. The properties exhibited by this emergent entity shall be evaluated.

Abstract

Abstract: As researchers evaluate organisations, projects, and teams, there is a desire for a consensus from those within the organisations who are participating in the research. A common consensual perspective from a team appears to reflect an optimal st
ate where those being evaluated have a common understanding of the current state of events within the context of their environment. The question arises, though, whether an evaluation finding consensus reflects the reality: there are a variety of reasons w
hy a common understanding may be false consensus. Hidden behind this false consensus may be a variety of unaddressed issues which are actually the core of the problem. This paper proposes an evaluation method incorporating the principles of sensemaking an
d devils advocate, where a consensus of perspectives is challenged before they are considered valid. This is achieved in a workshop where participants reflect on their own perception of reality and represent this reality in a matrix of influencing and re
levant factors. The individual matrices are then combined and used to highlight disparities in the participants perspectives through a single matrix visualisation. Discussion in the workshop then focusses on the areas, highlighted by the matrix, where di
fferences of perspectives are identified. In effect, the consensus presented by those being evaluated will be challenged, and a new common understanding will have to be created. Problems such as groupthink can create a false consensus, and it is proposed
herein that the workshop provides a mechanism for challenging this. The objective of the research herein was to determine the feasibility and potential benefits of the proposed workshop. The workshop itself is evaluated in this paper, to determine if it
has value. The benefits of such a workshop are described, showing how an organisation went from a false consensus concerning problems within the organisation, to the start of a process to address the real underlying issues.

Abstract

Abstract: An academic group and discussion forum were established on Facebook for a cohort of postgraduate students studying Concepts and Principles of eLearning. The Forum had a constructivist, student‑centric ethos, in which students initiated discuss
ion topics, while the course leader and administrator facilitated. In previous research, content analysis was undertaken of the discussions, but the present study evaluates the collaborative learning environment on Facebook, investigating social relations
hips, study‑related pursuits and the balance between them, as well as considering whether the Group could be viewed as a Web 2.0 application. A literature review shows how social networking by students, initially social, began to overlap with academia, l
eading to groups and forums for academic purposes. In mixed‑methods research, qualitative analysis was done on free‑text data to extract themes from students reflective essays and from an exam question, while heuristic evaluation was conducted by expert
evaluators, who analysed forum discourse in line with contemporary learning theory and considered the social culture of participation. Findings of the qualitative analysis and results of the heuristic evaluation of forum participation confirmed each other
, indicating a good social climate and a conducive, well‑facilitated environment. Inter‑personal relationships were fostered between distance learners, and academic value arose from independent research, peer‑learning and social negotiation. Facebook serv
ed well as an environment for collaborative learning, but did not provide a full Web 2.0 environment for the collaborative generation of artifacts or projects.

Abstract

Abstract: Agent based modelling (ABM) is a new modelling paradigm and one of the most advanced practical developments in modelling. ABM promises to have far‑reaching effects on the way that business practitioners and academic researchers use information
communication technologies to support decision making at different levels of management. Modern design models and architectural structures are opening up new possibilities and new application areas are coming to the foreground. Multi‑agent systems as sys
tems of distributed artificial intelligence are now having a significant influence on information systems design, simulation and analysis. This paper focuses on the various modelling methods and technologies that are employed in the development of intelli
gent decision support systems. Its goal is to evaluate the role of the agent based modelling in the design of management decision processes. The paper considers the main features of intellectual agent modelling methodology, and discusses the different ty
pes modelling categorization. It does so from research base that draws from theoretical underpinnings as well as international and domestic industry practices. The basic principles of agent‑based modelling are first introduced and areas of application are
then discussed from perspective of real‑world applications: flow simulation, organizational simulation, market simulation, and diffusion simulation. The classification of modelling types is discussed, together with and business application simulation fra
meworks.