Evaluation Strategies

How can we measure success? How do we know?
Why evaluate design by people?

Taxonomy of methods - different methods achieve different goals
the scientific process of classifying things (= arranging them into groups)


Field strategies

• Field Study - You give your thing out there  and users use it over a longer period of time.
• Field Experiment -Like clinical drug trial.Half the people get one, the other half gets the other one. Users use it over a longer period of time.

Experimental strategies

• Experimental simulation - 
• Laboratory Experiment - You bring people into the lab and they try out one or two different interfaces.

Respondent strategies

• Judgement Study - how much do you like X or Y
• Sample Survey - different polls, you can do them before or after the user has used the interface.
• Surveys - you are not getting behavioral feedback but more like self reports. People may not be telling the truth what actually happened.

Theoretical strategies

• Formal Theory - 
• Computer Simulation - computer simulates what a user could do - this kind of method is not really widely used

Measure - dependent variables
task completion time
accuracy 
recall
emotional responses

Manipulations - independent variables
You are always manipulating some aspect of the user interface. Should we use a graph for representing info or textual. Should we put a navigation bar on the top or to the right of the page. These kinds of manipulations we see all the time in the field of HCI.

Precision - Internal validity
If we would run the tests again would we get the same results? Weather the results are useful etc.

Generalizability- External validity
If the results generalize to real world situation we ale going to say that UI has high external validity.