Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

HCI 3e - Ch 9: Evaluation techniques

1.
chapter 9 evaluation techniques

2.
Evaluation Techniques <ul><li>Evaluation </li></ul><ul><ul><li>tests usability and functionality of system </li></ul></ul><ul><ul><li>occurs in laboratory, field and/or in collaboration with users </li></ul></ul><ul><ul><li>evaluates both design and implementation </li></ul></ul><ul><ul><li>should be considered at all stages in the design life cycle </li></ul></ul>

8.
Review-based evaluation <ul><li>Results from the literature used to support or refute parts of design. </li></ul><ul><li>Care needed to ensure results are transferable to new design. </li></ul><ul><li>Model-based evaluation </li></ul><ul><li>Cognitive models used to filter design options </li></ul><ul><ul><li>e.g. GOMS prediction of user performance. </li></ul></ul><ul><li>Design rationale can also provide useful evaluation information </li></ul>

13.
Experimental evaluation <ul><li>controlled evaluation of specific aspects of interactive behaviour </li></ul><ul><li>evaluator chooses hypothesis to be tested </li></ul><ul><li>a number of experimental conditions are considered which differ only in the value of some controlled variable. </li></ul><ul><li>changes in behavioural measure are attributed to different conditions </li></ul>

14.
Experimental factors <ul><li>Subjects </li></ul><ul><ul><li>who – representative, sufficient sample </li></ul></ul><ul><li>Variables </li></ul><ul><ul><li>things to modify and measure </li></ul></ul><ul><li>Hypothesis </li></ul><ul><ul><li>what you’d like to show </li></ul></ul><ul><li>Experimental design </li></ul><ul><ul><li>how you are going to do it </li></ul></ul>

18.
Analysis of data <ul><li>Before you start to do any statistics: </li></ul><ul><ul><li>look at data </li></ul></ul><ul><ul><li>save original data </li></ul></ul><ul><li>Choice of statistical technique depends on </li></ul><ul><ul><li>type of data </li></ul></ul><ul><ul><li>information required </li></ul></ul><ul><li>Type of data </li></ul><ul><ul><li>discrete - finite number of values </li></ul></ul><ul><ul><li>continuous - any value </li></ul></ul>

20.
Analysis of data (cont.) <ul><li>What information is required? </li></ul><ul><ul><li>is there a difference? </li></ul></ul><ul><ul><li>how big is the difference? </li></ul></ul><ul><ul><li>how accurate is the estimate? </li></ul></ul><ul><li>Parametric and non-parametric tests mainly address first of these </li></ul>

22.
Subject groups <ul><li>larger number of subjects  more expensive </li></ul><ul><li>longer time to `settle down’ … even more variation! </li></ul><ul><li>difficult to timetable </li></ul><ul><li>so … often only three or four groups </li></ul>

29.
Cooperative evaluation <ul><li>variation on think aloud </li></ul><ul><li>user collaborates in evaluation </li></ul><ul><li>both user and evaluator can ask each other questions throughout </li></ul><ul><li>Additional advantages </li></ul><ul><ul><li>less constrained and easier to use </li></ul></ul><ul><ul><li>user is encouraged to criticize system </li></ul></ul><ul><ul><li>clarification possible </li></ul></ul>

32.
post-task walkthroughs <ul><li>transcript played back to participant for comment </li></ul><ul><ul><li>immediately  fresh in mind </li></ul></ul><ul><ul><li>delayed  evaluator has time to identify questions </li></ul></ul><ul><li>useful to identify reasons for actions and alternatives considered </li></ul><ul><li>necessary in cases where think aloud is not possible </li></ul>

38.
eye tracking <ul><li>head or desk mounted equipment tracks the position of the eye </li></ul><ul><li>eye movement reflects the amount of cognitive processing a display requires </li></ul><ul><li>measurements include </li></ul><ul><ul><li>fixations: eye maintains stable position. Number and duration indicate level of difficulty with display </li></ul></ul><ul><ul><li>saccades: rapid eye movement from one point of interest to another </li></ul></ul><ul><ul><li>scan paths: moving straight to a target with a short fixation at the target is optimal </li></ul></ul>