Outline

Introduction

Quality Management (QM) is an integral part of process models for software engineering. Due to the neglect of quality assurance (QA) measures, such as reviews, software projects face a high risk to fail or be postponed [Ref.Â 1], [Ref.Â 2]. Software-engineering teams and companies look for effective and efficient QA measures that can easily be integrated in their projects. Inspections (technical reviews) are used for crucial modules and documents and are performed by IT experts. Other less formalized review types, that integrate domain experts and users, can and should also be used for the QM in a software engineering project. Health informatics with its multidisciplinary approach in software-engineering and maintenance can facilitate the development and use of alternative review methods.

Background

Reviews are time consuming procedures and require experience in moderation and in product centred software assessment. Common review types are inspections (technical reviews), team reviews, walkthroughs, pair programming, peer desk check, pass around and ad hoc review. The level of formalization follows a descending order: from inspections, that are well defined and formalized, down to ad hoc reviews, which are casual reflections and discussions during the development process [Ref.Â 1], [Ref.Â 2]. A GREME Review (Group discussion and Email-based Evaluation) combines team review and pass around. The purpose is to facilitate structured, comprehensible and economical quality assurance during software engineering for non-crucial modules and documents.

Method

A research team developed and used the GREME Review during its work on a web-based information system on hospitals [Ref.Â 3]. GREME can be applied in the different phases of software engineering: analysis, design, and implementation. The possible products to be reviewed may comprise user interfaces, documentations, manuals, user guides and working processes. The review team includes moderator, author (developer) and peers, that is IT and domain experts (health professionals, economists, others) and users.

GREME follows a defined workflow, which can be tailored to the need of a particular software engineering project: prototype releases alternate with pass arounds and moderated group discussions (team review). The discussions are integrated in routine meetings, for instance weekly department meetings, and do not exceed 30 minutes, to reduce the workload. The team discusses the current prototype, identifies errors and open requirements and proposes solutions. From the emails, submitted by the peers in the pass around cycles, comprehensible insights are gained through qualitative text analysis performed by the moderator. The author uses the results of the analysis for the identification of must, should and can measures.

The work in a group, the review of a product, and the use of qualitative research methods require both professional expertise and several social skills from the group members. They have to be open minded for the approach and new insights. Furthermore, they have to collaborate reliably, give feedback and take part in the discussions in a polite, open and supportive way. The persons involved have to bear in mind, that the subject of the review is the product and not the developer’s performance [Ref.Â 1], [Ref.Â 2], [Ref.Â 4].

The moderator has to be able to guide the group discussions neutrally and goal-oriented. Reflections on the group dynamics and the ability to focus on the contributions of the group members support this [Ref.Â 1], [Ref.Â 2], [Ref.Â 5]. To perform the text analysis of the email feedback the moderator needs basic knowledge in the collection of qualitative data and in qualitative text analysis. Examples on introductory literature and training programs are for instance [Ref.Â 6], [Ref.Â 7]. The text analysis can be performed computer supported (for instance with [Ref.Â 8]).

Results

GREME was used for the work on several modules that were required for the online release of a web-based information system on hospitals [Ref.Â 3] (1) re-engineering of the web frontend for the presentation of hospital data in manner of tables, (2) development of the questionnaire content for the web-based evaluation, (3) implementation of the homepage for the web-based information system on hospital data. Review costs are generally based upon working time. Table 1 [Tab.Â 1] summarizes the GREME steps. Table 2 [Tab.Â 2] gives an overview of the effort (working hours) on the development, the GREME steps and the ratio of GREME effort to development time. Accounting all modules the overall fraction of GREME lies between 16 to 29 percent. This corresponds to the values of 10 to 50 percent reported on standardized reviews [Ref.Â 1], [Ref.Â 2].

Discussion

GREME is intended as a further method for quality assurance in software engineering and user-centred development in order to meet the need for adaptable and cost effective QA methods [Ref.Â 1], [Ref.Â 2]. The examples show that GREME can be used to review non-crucial products economically, efficiently and effectively. GREME was easy to integrate in an ongoing software engineering project. The efficient and effective communication fostered the fast recovery of new insights. The team detected errors and aberrations at an early stage, designed solutions and assessed the quality of a product. The effort for the participants was foreseeable. This and the across-the-board communication of IT and domain experts led to a high acceptance of GREME. The adherence to delivery dates was high. Performed by a multidisciplinary team GREME facilitates the integration of IT knowledge and user perspective [Ref.Â 1]. The work beyond established development teams and professional areas can strengthen the identity of an organization and foster organizational learning [Ref.Â 1], [Ref.Â 4].

Acknowledgement

The work presented here was done at the Department of Medical Informatics at RWTH Aachen University. In the meantime the main author of this paper, who was head of the ISG Research Group from 2002 until 2007, started a sabbatical. The authors thank the other members of the ISG Research Group for their work on and with GREME, and the head of the department, Prof. Dr. Dr. Spitzer, for his encouragement.