Methods for evaluating early prototypes (Reading assignment 2)

This evaluation and discussion is to learn and select from three methods for evaluating early prototypes. The methods are:

(1) Multiple Sorting
(2) Contextual laddering
(3) Wizard of OZ

Multiple Sorting
The first method addressed is multiple sorting. It relies on te assumption that People’s conceptualising and understanding of their world, and therefore their knowledge, is based on categorisation. A simple linear scale method (semantic differential – awkward vs. easy to use) was used to collect information on whow we see the world. Since a person is constantly updating their understanding of their surroundings and the world, Kelly (Kelly, G. A. The Psychology of Personal Constructs, Volume One: Theory and Personality. (1). 1955. New York, Norton.) developed a technique for eliciting personal constructs in an interview context which would be more multidimentional. This technique is known as the ‘Repertory Grid Test.
In the Repertory Grid Test, participants are asked to think aboud tiads of items. They need to think and describe, what is similar between two of them and why the third one is different. This method brings out contrastive dimentions and participants are then asked to rate those contrastive items.  The data collected then forms a grid and helps to explain how people interpret the items and connections genetrated by other people on the same items.
Both of these methods rely on the assumption that our constructs are polar. Categorization can often be multidimentional. The Repertory Grid Test method is quite timeconsuming also since interviewing the participants is also necessary. Verbalization of the answers is also often difficult. To address the linear approach of the earlier tests, new variations have been devised. A new version of the test provides the participants with a wide variety of objects and allows them to select as many to a group as they like. After each sorting, participants are asked for the reasons they made the decisions. Multidimensional Scalogram Analysis (MSA) is then performed on the resulting sort data, to yield spatial maps of constructs for interpretation alongside interview discussions.
Contextual laddering 
Laddering is especially a well-known interview technique where customers are asked multiple times to explain why an attribute, that has been given to a product by the user, is important. Many times after explaining the background of an attribute, the question why, is asked again. E.g. Why is that important to you? By probing into the reasons why, the interviewee will ‘climb up the ladder’. This way, the reasons (consequences) why certain attributes are important will first be revealed, followed by an expression of how
these consequences serve personal values. UX Laddering is useful to design for attributes that offer value and meaningful user experiences to users. The goal of laddering, is to identify and understand the links between key perceptual elements across the range of
attributes, consequences and values.
Wizard of OZ (WOz)
The WOz method helps designers avoid getting locked into a particular design or working under an incorrect set of assumptions about user preferences, because it lets them explore and evaluate designs before investing the considerable development time needed to build a complete prototype. The method is particularly useful in exploring user interfaces for pervasive, ubiquitous, or mixed-reality systems that combine complex sensing and intelligent control logic. The problem with the method is that it requires the engineering of an interface and integrating it with an incomplete system. Now when the system under investigation is developed further, the WOz interface is usually not and is thus a once (or twice) time used interface.
In WOz a person acts as a wizard and performs the steps that have not been automated or implemented by the system itself. Thus manually closing the gaps that the current state of development has within the system. As development goes onward the gaps, the wizard has to fill when testing, shrink.
In our case, the last method seems a good one to use since we need to test the overall system on potential users without having certain very important functionality developed. In order to see if it makes sense to go forward with the same plan, we need to test the overall perception of the system.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s