4
14/01/20044 Usability definition The environment and equipment: For each user group, which web browsers will be used, what type of machines, with what type of screens and what speed of web access? What access will users have to assistance if they encounter problems? User: What are the skills, motivations and previous experience of each anticipated user group? The tasks: For what identifiable purposes will each user group be visiting the web site? What will be their tasks? How motivated will they be to persist in achieving their task objectives?

5
14/01/20045 ISO (1995) for traditional software user interface Bevan (1997) for web user interface EffectivenessCan be measured by the extent (accuracy and completeness) to which the intended goals have been satisfied. How many of the goals of the intended users can the web site support (e.g. how much of the information required by a potential tourist is available on the site?) Will a typical user accessing the site easily locate all the information relevant to their goal? EfficiencyCan be measured by the resources that have to be expended to achieve the intended goals. How much time and effort will be required to locate the required information? SatisfactionCan be measured by the extent to which the users find the overall system acceptable. How satisfied will the user be, and how much will they enjoy using the site?

6
14/01/20046 Usability Evaluation “…concerned with the collection of data about the usability of a design or product by a specified group of users for a particular activity within a specified environment or work context” (Preece et al., 1994: #602).

7
14/01/20047 Usability Evaluation Expert vs Users Time - in the development lifecycle Performance Measures Produced

8
14/01/20048 Time formative evaluation that takes place before the implementation in order to influence the development of the product summative evaluation that takes place after the implementation in order to test the proper functioning of the system.

9
14/01/20049 Performance measures objective performance measures, which are objective measurements or observations of user behaviour and are focused on task performance - that is, how well the users can achieve a specific task and subjective user preference measures, that measure the users’ opinions of working with the system - that is, how much they like to use the system (Nielsen and Lavy, 1994; and Lewis, 1995).

10
14/01/200410 Heuristic Evaluation (1) Heuristic evaluation is an inspection technique. Usability inspection:“the generic name for a set of methods based on having evaluators inspect or examine usability-related aspects of a user interface” (Mack and Nielsen, 1994: #1). Selection of evaluators and principles.

11
14/01/200411 Heuristic Evaluation (2) Set of evaluators produces lists of usability problems in a user interface by going through it and noting deviations from accepted usability principles. Prior to the evaluation the evaluators need to obtain: a description of the objectives, target audiences, expected usage patterns of the system being tested, list of heuristics.

12
14/01/200412 Sessions Briefing sessions: evaluators are told what to do Evaluation period: 1-2 hours independently inspecting the system Debriefing session: experts come together to discuss their findings

13
14/01/200413 Heuristics (Evaluation Table) Degree of system conformance with each particular rule [1..5] Severity Rating [0..1] Comment Visibility of System Status Match between system and the real world - Consistency User control and freedom Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design

14
14/01/200414 Heuristics Aesthetic and minimalist design Is color used in a form of coding? Is color used to make the screen bright? Match between system and the real world – Consistency Are the same buttons, used across the system? Are they used in the same way? Visibility of System Status Does the system respond to user’s actions

15
14/01/200415 Degree of Conformance Please rate your degree of conformance with each particular rule. The range of assigned values is [1..5]. Severity rating - importance weight - is also given to each rule, indicating the relevance of the general principle to the system according to the experts opinion. The range of assigned values is [0..1]. Degree of conformance 1 = Not at all conformance 2 = … 3 = … 4 = … 5 = Absolute conformance with the rule

16
14/01/200416 Severity Rating [weight assignment] Please rate the severity rating for each particular rule. 0 = this is not a usability problem 0.25 = cosmetic problem only – need not fixed unless extra time is available on project 0.50 = minor usability problem – fixing this should be given low priority 0.75 = major usability problem – important to fix, so should be given high priority 1 = usability catastrophe – imperative to fix this before product can be released

17
14/01/200417 Formula e = Σ w i r i r i = average score of rule i (heuristic) - degree of conformance with each particular rule. w i = the relative weight of this rule according to ALL experts opinions - relevance of the general principle to the system

18
14/01/200418 Interpretation of Results High importance weight (w i ) and low conformance of the system with particular rule (r i ) = necessitates corrective actions

19
14/01/200419 Card Sorting “The purpose of card sorting is to better understand the user’s concept of how information on the web site should be organised” (Fuccella, 1997: #71). Provides feedback on global questions regarding the organisation and structure of a web site (Levi and Conrad, 1997).

20
14/01/200420 Card Sorting (2) A group of participants (users of the system) and a stack of standard-sized index cards are required. Two Categories: Affinity clustering: participants are instructed to sort the index cards into groups that look similar to them (Fuccella, 1997) and then to provide a description of each group. Pre-defined categories: sorting of cards based on a predefined set of categories or specific criteria, for example, relative importance, and expected frequency (Constantine and Lockwood, 1999).

21
14/01/200421 When can it be used This technique can be used throughout the design lifecycle. During the prototype stage, the results of the technique can suggest structures to design menu trees. At a later stage it can be used to compare the card sort results to a draft web structural design. The results will identify specific areas where the underlying hierarchy can be improved so that users can more easily find the information they are looking for (Levi and Conrad, 1997).

22
14/01/200422 Questionnaires The questionnaire and survey type of usability evaluation technique is one way of measuring the user’s opinion and attitude. Questionnaires that can be used to measure user satisfaction the Questionnaire for User Satisfaction (QUIS) the Purdue Usability Testing Questionnaire (PUTQ) the Software Usability Measurement Inventory and the Website Analysis and MeasureMent Inventory (WAMMI).

25
14/01/200425 Software Usability Measurement Inventory Tool (SUMI) SUMI uses a five-point scale to rate 50 system attributes to measure users, perceived usability. Affect: The degree to which the user feels that the system is enjoyable and stress-free to use; Learnability: The degree to which the user feels he can learn new operations with the system; Control: The degree to which the user feels in control of the system, rather than vice versa; Efficiency: The degree to which the user feels he gets his work done well with the system; and Helpfulness: The degree to which the user feels the system helps him along.

26
14/01/200426 Website Analysis and MeasureMent Inventory (WAMMI) WAMMI uses a five-point scale to rate 20 web site usability characteristics. Attractiveness: The degree to which users like the site, whether they find the site pleasant to use. Examples of items are: This web site is presented in an attractive way, and You can learn a lot on this web site. Control: The degree to which users feel ‘in charge’, whether the site allows them to navigate through it with ease, and whether the site communicates with them about what it is doing, for example: Going from one part to another is easy on this web site, and I feel in control when I'm using this web site.

27
14/01/200427 Website Analysis and MeasureMent Inventory (WAMMI) Learnability: The degree to which users feel they can get to use the site if they come into it for the first time, and the degree to which they feel they can learn to use other facilities or access other information once they have started using it, for example: All the material is written in a way that is easy to understand, and It will be easy to forget how to use this web site. Helpfulness: The degree to which users feel that the site enables them to solve their problems with finding information and navigating, for example: This web site has not been designed to suit its users, and All the parts of this web site are clearly labeled. Efficiency: The degree to which users feel that the site has the information they are looking for, whether it works at a reasonable speed and is adapted to their browser, for example: You can find what you want on this web site right away, and This web site works exactly how I would expect it to work.

30
14/01/200430 Item Sampling Psychometric and Survey type of Questionnaires (Kirakowski, 1994). Set criteria that will allow for the selection of questions. “A range of questions, adjectives, or other prompts is developed that the researcher hopes might have some bearing on the stated requirements of the study.” (Kirakowski and Corbett, 1990: #212). Sources: Guidelines Heuristics Questionnaires Checklists Relevant Surveys

31
14/01/200431 Development of Questionnaire A Questionnaire usually consists of several sections, based on the objectives. ONE SECTION: Representation of socio- demographic variables. Use of Filtering Questions: split the population into groups.

32
14/01/200432 Sample Size and Scale Sample size: 5 times the number of questions or variables. NOT NECESSARY!!! Rating scales: should have betwen 5 and 7 categories (Kirakowski and Corbett, 1990; Tull and Hawkins, 1990; and Lewis, 1995). The larger the number of scale steps used in a scale the higher the reliability of the scale, but with rapidly diminishing returns. “As the number of scale steps is increased from 2 up through 20, the increase in reliability is very rapid at first. It tends to level off at about 7, and after about 11 steps, there is little gain in reliability from increasing the number of steps” (Nunnally, 1967: #521).

33
14/01/200433 Reliability To assess the reliability of the factors of the usability test, Cronbach’s Alpha (Cronbach, 1970) statistic is usually utilised. When performing reliability analysis, some variables may be excluded from the factors or moved to other factors in order to refine: a) the interpretability and b) the internal consistency of that factor.

37
14/01/200437 Plan the Usability Test Define Purpose and audience of site Set the Usability goals Define Tasks Users will perform to observe users’ compliance with the system Define criteria, metrics and a method of collection of user’s response, and Design Questionnaire that is used to assess user’s subjective preferences to the system Test Users, Instructors, scheduling, payment Material Do not allow the developer of the site to be in the same room, especially if s/he has a bad temper…

39
14/01/200439 Report And Follow-up Tabulate Data According to the Listed Tasks Report Results Time needed to complete the test tasks Need for help during the test How frequently the instructor had to help the test persons solve a problem? What kind of user problems needed to be solved? Type of errors Pressing the wrong button on the interface Wrong numerical value entered Mode error: The correct name or number was entered in the wrong mode Provide a List with Design Recommendations Suggest Specific Actions for the Designer(s)