Methods for evaluating functional prototypes

In this post I will take a look at three methods on how to evaluate Functional prototypes. 

The methods are:
1) System usability scale;
2) AttrakDiff; and
3) Think aloud protocol.

General classes of usability should be covered if usability is measured. They are effectiveness, efficiency and satisfaction. Vastly different metrics can  also be used while measuring these attributes. Context-specificity makes comparing different systems usability very difficult. It also means that one design which is very good in one system and with one set of users/use cases, might not work well in other context. Copying usable solutions is then very risky.

The usability of an artefact is defined by the context it is used in. Thus every usability study should require different and detailed approach. To make usability testing easier and more universally comparable, the System Usability Scale (SUS) was created. System usability scale is a reliable and low-cost usability scale which can be universally used for assessment of systems usability.  SUS is a Likert scale and the overall outcome will be a score between 0 and 100. Alltogether it is a valuable, reliable and robust evaluation tool that allows usability evaluation to be performed effectively.

AttrakDiff is used to assess users feelings about a system with a queastionnaire. The questionnaire studies both hedonic and pragmatic dimensions with semantic differentials. The data aquired is quantitative and also comparable, much like in the case of System Usability Scale, but the weakness of AttrakDiff is that it uses the reflections of the users rather than the real experiences themselves. The approach is also used both in lab and field studies.

Think aloud protocol incorporates another dimension to the studies. While doing usability testing, users are asked to talk aloud and say what comes to their minds while performing the set tasks. The thoughts are often not even related to the task but rather something that comes to mind while performing them. This might include what they are looking at, thinking, doing, and feeling. Also a set of questionnaires are used. The questionnaires can be used before or after performing the tasks, as necessary.