Wisdom of the crowds

Group of people and a bowl of jellybeans.

How many jellybeans are in the bowl?

Individually people do not get so good answers. Add all those answers and get a mathematical average and you´ll get really good answer.

This to work:

• individual answers
• people should guess well not socially appropriate way

Designing Software for Collaboration

Collaboration: Same time / same place

• Face to face interaction
• Roomware
• Shared tables, wall displays
• Group Decision Support Systems (GDSS)
• Single display groupware

Collaboration: Different time / same place

• Message Boards
• Dedicated Team rooms
• Large displays

Collaboration: Same time / different place: 

• Remote interaction
• Video-Conferencing, 
• Real-time groupware 
• Messaging (Instant messaging, Email) 
• Virtual worlds 
• Multi-User editors 
• Shared Screen (vnc)

Collaboration: Different time / different place:

• E-mail
• Wiki
• Blogs
• Workflow
• Version control

It is hard to synchronize things, much more easier asynchronous. 

Introducing Controlled Web Experiments

User tests online not in lab.

Two column layout can be better then one.


Ways design makes a difference

• The position and color of the primary call to action
• Position on the page of testimonials, if used
• Whether linked elements are in text or as images
• The amount of “white space” on a page, giving the content space to “breathe”
• The position and prominence of the main heading
• The number of columns used on the page
• The number of visual elements competing for attention
• The age, sex and appearance of someone in a photo

Setting Up, Debriefing, and Summarizing Results

Greeting participants
Explain the task
Warm up tasks
Give the questions
Sign the release form.
Ask if they have any questions.
Don´t give away what you want to measure or them to learn.

Instructions to Participants
We test UI not you!
They can quit at any time
Explain that you are not providing any help to them

Collecting data
What they did and think.
How many errors and where.
etc.

Debriefing
Show the critical incidences and what they think you could do better there.
Ask about design alternatives!

Summarize the results
Things that actually matter. Your brain is going to forget all the details and summarizes can be really valuable. Even just one sentence.

Prioritize the different results - then you know what you have to do next.

Capturing Results and Recruiting

Video recording - its god to narrow it down to 20 sec snippets where you presents successes and problems of your design.

Screen recording is also an options.

Note down all the critical mistakes. Use these to get feedback again from the user when he ends the user test. Also you use them to help you to improve your design.

The "Thinking aloud" method:

Need to know what users are thinking, not just what they are doing.
Ask users to talk while performing tasks

• tell us what they are thinking
• tell us what they are trying to do
• tell us questions  that arise as they work
• tell us things they read

Thinking out aloud can change things also their actions can be different compared to silent task performing.

Another good practice is to let users perform the test individually and afterwards pair them up and show them videos about their critical problems while they tried to use the UI. They can then discuss the problems and maybe give you even more valuable information.

Setting up the test session: You can already prepare notes for noting. For example you already know what kinds of areas are interesting for you or just fill in some blanks etc. It can be better than just blank notebook.

Goals, Tasks, and Details

How to lower down the stress on people who are measured?

You are testing the site not the user! Communicate to the user that he is a partner for you who helps you to find errors and bugs in your system.

Articulate the Goals of your study. Write down the parts of your system you think you could improve and where you need feedback.

Think about when and where this test is going to take place.

Create concrete tasks - written down on a paper on computer - then you can be sure that you are consistent during many study sessions with different users.

Plan out the steps  - what are you going to ask, that you plan time for introduction and also feedback.

You can have a built in timer in you app / webpage that collects data about the time user is on one page.

Roles in a team of three: a facilitator (all the interaction with the user), note taker, video maker/ photographer.

Experimental details - do you want that users have some kind of training or preparation for the test.

Before you bring any real users into lab make shore you have had a "pilot"- he is purely there for debugging your user test process.

Participants and tasks

Choosing participants

Persuade participants if needed. For example if you are building an interface for teachers and they don´t want to participate then you can say that in the end when your product goes to production then they can use it in their work and make their life easier.

Selecting tasks

Real tasks that people would perform while using your interface in everyday life.
Don´t bend the tasks in a way that your preferred design would stand out. That´s cheating!

Designing user studies

Between subjects design - one group uses only one interface. - use it if you want to avoid ordering effect.

• Half of the users try out one interface.
• The other half tries out the other  interface .

Within subjects design - both groups use both  interfaces - this one avoids all the individual differences problem

Users already know a bit more about interfaces by the time they start using the second interface. To get more objective results in the end, both groups should start with different interfaces.

Evaluation Strategies

How can we measure success? How do we know?
Why evaluate design by people?

Taxonomy of methods - different methods achieve different goals
the scientific process of classifying things (= arranging them into groups)


Field strategies

• Field Study - You give your thing out there  and users use it over a longer period of time.
• Field Experiment -Like clinical drug trial.Half the people get one, the other half gets the other one. Users use it over a longer period of time.

Experimental strategies

• Experimental simulation - 
• Laboratory Experiment - You bring people into the lab and they try out one or two different interfaces.

Respondent strategies

• Judgement Study - how much do you like X or Y
• Sample Survey - different polls, you can do them before or after the user has used the interface.
• Surveys - you are not getting behavioral feedback but more like self reports. People may not be telling the truth what actually happened.

Theoretical strategies

• Formal Theory - 
• Computer Simulation - computer simulates what a user could do - this kind of method is not really widely used

Measure - dependent variables
task completion time
accuracy 
recall
emotional responses

Manipulations - independent variables
You are always manipulating some aspect of the user interface. Should we use a graph for representing info or textual. Should we put a navigation bar on the top or to the right of the page. These kinds of manipulations we see all the time in the field of HCI.

Precision - Internal validity
If we would run the tests again would we get the same results? Weather the results are useful etc.

Generalizability- External validity
If the results generalize to real world situation we ale going to say that UI has high external validity.

Rating Scales and Graphs

Use scales that people can relate to.

This image of scale values can be really hard to understand. For example you see a post where you notice a random rating used. At first you have no clue does it represent good or bad value ..

Color

How to get color right?

• Design in grayscale first (black and white).
• Keep luminance (heledus, erksus) values from grayscale when moving to color. 

Design for Glanceability

glance - to look quickly at something/somebody; to read something quickly and not thoroughly


It is much easier to read the top half of the letters than the bottom half.

68 %of the time  the eye is in motion when reading, 32% of the time the eye is fixated on one spot. When the eye is in motion it cant actually see anything. It can only read when it is down on the fixations.

With an "heat tracker" you can see how much time is spent on different parts of a webpage. 

Things that are top left are likely seen at first. Less important you can drift out.

Interlaced browsing and navigation

interlaced - põimitud.

A lot of different tabs opened in browser. Reading a lot of stuff at the same time.
On a mobile you don't have maybe seven different sites open but your attention is disturbed by other people.
Users attention is scattered and we as designers have to deal with that.

Menu labels that don´t say you anything -> Transactions - what the hell is that?
Users start to "try" different things and going back and forth to find the action they are looking for.

If icons are not supporting the text, come up with better icons !

If you are looking for one perfect word that would be a heading for  bunch of different stuff, then you should understand - there isn´t one perfect word!

Speaking block navigation - describes if you would click there what will you get to.

http://www.smashingmagazine.com/2008/02/26/navigation-menus-trends-and-examples/

Matching semantics & visual layout

Some starting points:

• Gather materials you find successful 
• Could be from very different domain
• Good artists borrow, great artists steal - Picasso
• Include visual design professionals in the iterative design cycle

Icon design: Differences that make a difference. Contrast like black vs white. Green is GO and red is STOP.

The example with the pill bottles - they are full of information but if you have 4 different bottles you cannot differentiate  them only by looking. 

Type matters

Typeface: Gill Sans (sans-serif typeface designed by Eric Gill.)

Dynamism through asymmetric layout in a text. Do not center stuff!

Sans serif is for short texts or headings. Where serif is for more longer texts.

Use real quotation marks - easier to understand where a quote starts and ends.

Also numbers have upper case and low case. Low case numbers bled in to text more easily.

Books that are cheap don´t but so much white space around text. But with white space it is easier for the eye to read.

Representational shifts

Instead of remembering what have you copied on a clipboard, the thing you did cut or copy could be circled in the same document.

As a designer your task should be:design in a way that there would be no chance for an user to make an error. "Your text is outside of the printable area, would you like to proceed?" - Don´t ask me but show me where it is and show it already by doing not afterwards.

Good Representation

Representation: kujutis

Problem solving as Representation

"Solving a problem simply means representing it so as to make the solution transparent" - Herbert Simon

Design by 0 working memory - meaning that don´t make a user to memorize stuff.

A good representation shows all the relevant information, and nothing else. Should enable: comparison, exploration, problem solving.

If designing a system to replace a manual work, pay attention not only at the process itself but also the objects used. for example by doctor your record tells more then only information written to it. If it is really "used" then you know a lot of action has been going on. If your record is still on doctors table then you were recently there etc.

Command line vs GUI

GUI

• self disclosing (ise nn avalduvad)
• you know what you can do all options are on the screen
• you know what you did, feedback

Command line

• Perform abstract actions
• Move many things at once

Buttons often have to abstract names and users actually don´t know what they are doing or what does it mean. For example 'Save'. where or what do I save etc.

Map Interface controls

Controls should mirror real world.

Direct manipulation provides

• Good idea of how each object works and how to control it
• Interface itself discloses how it is used
• The art in design is ti translate users cognitive capabilities and existing mental models into interfaces that work.

  Computers bring flexibility and if we concentrate only on making interfaces more like real world then we cant have that kind of flexibility  .

"If technology is to provide an advantage, the correspondence to the real world must break down at some point."- Jonathan Crudin

The Gulf of Execution and The Gulf of Evaluation

The Gulf of Execution
Primary question of users when meeting UI-s:  How do I do what I want to do ?

The Gulf of Evaluation 
Second question: What happened.


Can the user do what they would like to?
What representations are used for communication?

Solution what many UIs come across : Direct manipulation user interfaces

• Immediate feedback on actions
• Continuous representation of objects
• Leverage metaphor  - we are more and more familiar with digital environment and we don´t need so many real world metaphors to understand an user interface. 

Good design reduces gulfs.

How easily can we:

• Determine a function of the device?
• Tell what actions are possible
• Determine mapping from intention to physical movement
• Perform an action?
• Tell if system is in desired state?
• Determine mapping from system state to interpretation?
• Tell what state the system is in?

One strategy is really effective - Mental models.

Mental model in users head does not do what real life is really like.

There are many kinds of mental models:

• My own behavior
• Someone else´s behavior
• A software application
• .. or any information process that´s mediated

Metaphores: Learning Mental Models
"A text processor is a typewriter"
Mental models are inconsistent in time

Users and designers communicate through their mental model

Designer´s model = mental / conceptual model of the system.
User´s model = mental model developed through interaction with the system
Designer expects user´s model to be the same as the designer´s model. But often it isn´t!

Conceptual Model Mismatch
Mismatch between designer´s and user´s conceptual models lead to: slow performance, errors, frustration. You see this by walking around and noticing notes on devices etc where people write what to do or how to do smth.

Heuristics

heuristics - a method of solving problems by finding practical ways of dealing with them, learning from past experience

Ten Usability Heuristics by J. Nielsen

Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Recognition rather than recall

Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

Flexibility and efficiency of use

Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

Wizard of Oz

Wizard of OZ was actually a regular guy who was using paraphernalia (equipment) to present himself as another character. 









Interpreted into design: We can be the ones who bring a prototype into life.

The guy behind the scenes - Designer
Paraphernalia used - Prototype (for example made out of paper)
Mysterious wizard of Oz - Experience  for the user

Paper prototype
No one gives you low level feedback. The user is not going to say that your pen line is to thick or too crooked. People understand that they should be focused on high level feedback.

If you have a quick prototype using already real code example and you say to people that they should look through the fonts and colors used then they still give you comments like 'This logo should be 3px to the right.'

AND you can go nuts yourself to fix all the pixel problems that annoy you. You don´t have the problem with paper.

Brainstorming

Brainstorming Guidelines

• Be Visual. 
• Defer judgment. 
• Encourage Wild Ideas. 
• Build on the Ideas of Others. 
• Go for Quantity. 
• One Conversation at a Time. 
• Stay Focused on the Topic.

How might we.. question.

• .. help people find the right train.
• .. help people to park their cars.
• .. help people to spend time while waiting on the line.
• ...

Ideas on postits to board, everyone can vote => good idea is picked.

Conformity v. Novelty

conformity - behaviour or actions that follow the accepted rules of societynovelty -  the quality of being new, different and interesting
If you do not have an idea and you search for inspiration, then you can end up borrowing ideas you saw. This happens only when you don´t come up with your own excellent idea.
Seeing examples dos not produce novelty.

Functional fixation and Parallel prototyping

Functional  fixedness :  "Mental block against using an object in a new way that is required to solve a problem" Duncker, 1945

Little change in the experiment produces drastic change in the results.
















In the design a lot of people experience  fixedness. 

"How can we structure design process in such way that designers are less likely to be fixated on a particular solution?"

For example a task to create a design for an ad. Designers had 2 possibilities: 
1. Create 6 designs in serial in a way that you´ll get feedback after every design (linear design process).
2. Create 3 designs, get feedback, make new 2 designs , get feedback and end up with the sixth design (parallel design process).

Parallel design process brings a better solution. In my opinion its because of the fact that if you present 3 different designs then the person(s) who give you feedback have information for comparison (read earlier post where this idea is presented). 


If you design in parallel way then you will respond to more positively to critique, because not THE design is criticised but ONE of many. Your ego is invested in your design and with only one design your ego is hurt.

Prototyping as Communication

Prototyping is to present the idea to different stakeholders:

Colleagues

Clients

Users

Ourselves

Prototypes are disposable!

The rights of a prototype:

• žShould not be required to be complete
• žShould be easy to change
• žGets to retire

Choose the right tools for presenting your ideas. For UI functionality you can use paper prototypes, for effects you already need a simple code example etc. Choose what is appropriate for the idea you want to present.

Also the best designers use prototypes. Its not only a tool for newbies.

Good artists borrow, bad artists steal. - Pablo Picasso

You can take inspiration from others but you cant copy them.

Goals and Objective functions

"Prototyping is a strategy for efficiently dealing with things that are hard to predict."

Different types of unknown stuff:

  known unknown   - I am aware that x is unknown variable, I know it exists.
  unknown unknown   - I wasn´t even aware that x is a variable

Prototyping helps you to reveal the unknown unknowns.

Often people have  AN  idea and only  ONE  deign. When they need to argue for their design they end up losing the fight because they do not have so many argument speaking for them.

In one way or another you have to push your design through challenge and therefore it is more convenient to think about objective functions. What will this design achieve? Why is it so good? What makes it valuable?

Users do not know which alternatives a design could have. Its the same if you ask a user what kind of tools he would like to have - he doen´t know. Therefore you as a designer need to present more than one design that the user would have the information and could answer your quest
ions through comparison. Is A better than B ? Rather then asking is A good?

User then can answer you: I like part of A and part of B because ..

-----------------------------------------------------------

Simulated annealing

“The best way to have a good idea is to have lots of ideas.”

-Linus Pauling

Simulated annealing - for example this technique is used to reduce the amount of defects in the material (metal).  Atoms begin to move while the metal is annealed. Afterwards the metal is cooled down slowly. This way it is more likely that atoms find a place more suitable for them.

Go broad with different ideas and solutions -- have a lot of different prototypes. Afterwards during the design process you can focus more on the best prototypes. Have different designs - because of the statistics you will end up then with a better design.

What is the cost of making changes over time?
In the web you dont have actual costs like money but you still have to ask yourself some questions before making any changes:

1. Are my users already used to the design right now?
2. Will I loose customers after an update?
3. Have they built stuff on top of my software?

Design for people

People’s tasks, goals, and values drive development

Work with users throughout the process
 Assess decisions from the vantage point of users, their work, and their environment

Pay attention to people’s abilities and situation.

Talk to the actual experts.

Finding defaults when an action or behavior  is showing up again and again. Defaults can make life easier for the users. Prevents some extra clicks or decisions.

"Prototyping is a strategy for efficiently dealing with things that are hard to predict."

"Make multiple prototypes simultaneously to get most value."

Other strategies

Be a prentice. Actually learn about that field you are asked to design in.


Find lead users -innovations they create. Lead users have seen different tools and they have also strong opinions how things could be better.


Diary studies - ask participants to do smth each day. Be there when they are doing it because their memories are faulty. We often think we are "better" "wiser" etc.


   Every idea does not have to come up  from those strategies.  
   Wherever the idea comes it can be good.   

Interviewing

GOOD PARTS FROM THE LECTURE

» Going out there to interview - one of the scaring things to do - but it is incredibly valuable.

  Before  

» Don't ask those fancy general questions - "Do you exercise regularly?" - my response would be - "What do you mean my regularly?"
» Before the actual interview set some goals and questions. Take a trial on your friends / colleagues .

» "What kind of tools he would like to have ?" - I don't carry around an index card of all the tools i would like to have someday. But if you would ask it in my environment where I perform a certain task or process then I would be able to tell you where my need lie.

» Plan is bad, planning however is valuable. - If you have a plan with all the questions you want to ask, then don´t be anal about this. You don´t need to answer all of them and you can mix the order of them ;).

  During  

» Bring along: notebook and a camera (documenting action)

» Objects tell a story-ask people about their artifacts that they carry around - kick off some excellent stories.

» Think from the perspective of the person who answers your questions.

» Let the silence in - give about 5 sec silence to process the question and form the answer

» Out there:

• Introduse youeself, explain your purpose
• The interview is about them, not you!
• Ask the question and let them answer
• Aollow up
  • Adjsut your questions to their precious answers
  • Ask questions in language they understand
  • Pick up on and ask for examples
• Be Flexible

  After  

» Should you record audio or video? - yes! Use short snippets afterwards to convince your boss or client about smth. 30 seconds of video is a great strategy.

» Synthesize and draw your conclusions.

Participant observation

PROCESS AND PRACTICE
You are in a foreign country that´s culture is far more different then yours and you are trying hard to do things according to the "culture´s user guide" but somehow its not working out for you. Why? There are things you can write down that other people can learn  by processing the information but there are also so called unwritten laws or just ways of behaving that you cant´t learn my reading. Those things can only be learned through observation and practice.

The best is the one who has has done it the longest. Second best is who sat next to the first one.

HACKS
Use hacks that people use to complete their tasks. For example someone has put duck tape on a button that is being used for the wrong reasons. You can use those hacks to redesign. So keep your eyes open.

ERRORS
Scott mentions two kinds of errors: slips and mistakes.

A slip: you know exactly how to perform a task but you just slipped by accident. For example you try to open a window but you somehow miss grabbing the handle. You slipped.

A mistake: you have totally other mental model for solving a task and its not the "right" one. Users need extra signs to get them through  stuff.

HCI History

Vannevar Bush & Memex (1945)

Bush was the man who introduced the idea of storing information (books, records and communications) into a system that can be accessed afterwards for further reading. In his vision this kind of system would have been in some way linkable. You could also add your comments and notes to different microfilm frames in the sequence.

Ivan Sutherland & Sketchpad (1963)

Inspired from Memex Ivan Sutherland to build worlds first graphical user interface.

Douglas Engelbart and the first computer mouse (1968)

Invented the first computer mouse.

Alan Kay and Dynabook (1968)

"“The best way to predict the future is to invent it."  

Dynabook was the forefather for laptops and tablets. At first it´s idea was to give children access to digital media

He was part of creating the theory about model human processor. This theory gave the scientists a chance to actually predict and design user interfaces for people.

Those success stories bring out also the core values of the given course.

You design for the people.

Prototyping is essential and tractable, also for future technologies.                              Compare.

Design improves through iterations.

Theories (principles) help to inspire designs.

Table of context

I. INTRODUCTION

 History of HCI, Part 1
 History of HCI, Part 2

II. DISCOVERY

 Participant Observation
 Interviewing
 Other Strategies

III. USER-CENTERED DESIGN

 Design for People
 Design as Simulated Annealing
 Goals and Objective Functions
 Prototyping as Communication

IV. PROTOTYPING

 Functional Fixation and Parallel Prototyping
 Conformity v. Novelty
 Brainstorming
 Wizard of Oz

V. HEURISTIC EVALUATION

 Some Heuristics
 How and Why

VI. DIRECT MANIPULATION

 The Gulf of Execution and The Gulf of Evaluation
 Map Interface Controls
 Command Line versus GUI

VII. REPRESENTATION MATTERS

 Good Representation
 Representational Shifts
 Attention, Stress, and Risk

VIII. VISUAL DESIGN

 Type Matters, Part 1
 Type Matters, Part 2
 Matching Semantics and Visual Layout

IX. INFORMATION DESIGN

 Interlaced Browsing and Navigation
 Design for Glanceability
 Color
 Rating Scales and Graphs

X. EVALUATING DESIGNS

 Evaluation Strategies
 Designing User Studies
 Participants and Tasks
 Chi-Square Test and Probability

XI. CONDUCTING USER STUDY

 Goals, Tasks, and Details
 Capturing Results and Recruiting
 Setting Up, Debriefing, and Summarizing Results
 Introducing Controlled Web Experiments

XII. COLLABORATION ON THE WEB

 Designing Software for Collaboration
 Collocation versus Remote, Part 1
 Collocation versus Remote, Part 2
 Wisdom of the Crowds
 Challenges

The Beginning

Some weeks ago I got an e-mail which reminded me about a course in which I had enrolled myself earlier this year. At first  I didn´t remember anything about that enrollment but after visiting the page I had the "ahhaa" moment. "Coursera" is a cool environment where professors all over the world lecture you in their free online courses. So within 10 minutes  I surfed the page and found an excellent new course for myself :

Human-Computer Interaction

Scott Klemmer

During this year I have been interested in that topic and now I have the opportunity to attend a course which is held by Stanford University professor. Yeah STANFORD :). I got SO excited. At that moment I knew I want to give everything from myself to perform very good. I started to think about preparation for the cousre but I didn´t really know how or where to begin.

A week later I decided to go through all my bookmarks and really build some structure into them. During the cleaning session I found another webpage full of lectures: Introduction to Human Computer Interaction Design and for my biggest surprise again the author is Scott Klammer.

Now I have a plan.

• » First I´m going to watch all the introduction lectures.
• » Second I´m going to write down all important facts I learn there.
• » Third I´m gonna be awesome at the real course which starts at 25.09.2012.