Q6 - Design
Rubinís claim surprises me in as much as the book was published in 1994. It must have been many years in the making to be taking this stance at this late a date! In all honesty I realize that evaluation techniques must come after the things that they evaluate are developed and for this reason there is a significant time lag. Not having read the book I give him the benefit of a doubt that he is addressing WIMP GUI designs.
Rubinís statement, however, flies in the face of the ecological psychology movement pioneered by Gibson and adopted by Hutchins. Distributed cognition claims that the system is made up of people and artifacts that cooperate with one another to become one cognitive system. If one assumes that the environment in which the task is performed has no effect on the task then Rubinís claim is defendableÖ but you canít make this claim until you know more about the task. If one is assuming that the tasks that are undertaken involve usability issues as they relate to the desktop metaphor-based GUI then Rubinís claim is at least partially supportableÖ that is if the task is a standard routine task performed in a work environment by an expert.
But computing is moving away from the desktop, out of the world of work, out into the world and into the home. WIMP technology constrained to the desktop is making way for the bold new world of UBICOMP where there are many computers for one user and those computers are spread all across the landscape of life, not just relegated to the world of work in an office at a desk.
Evaluation techniques that include the environment in which the system is being used and the people that actually use that system expand the reach of usability further out into the world. When technology moves off the desktop and into the world there are issues that come to the table that, prior to that move, were not issues at all. If there are computers everywhere which one (or ones) do we pay attention to? Human attention may well be the most precious resource that will need to be managed and usability studies that occurs in a lab at a screen completely avoids this problem simply by its design.
With the desktop that is easy, there is only one computer to pay attention to but when the world is saturated with computing technology usability must address the environment in which that computing is embedded. With many computers all competing for your attention one must consider ways to move human attention from one to another as is needed. In their Calm Technology paper (Weiser and Brown) discuss that there needs to be ways developed to move attention from the periphery of humanís attention to the center of attention and then back. These attentional shifts will need appropriate metrics and evaluation techniques developed for this entirely new class of usability issues.
Cognitive walkthrough (Polson) suffers from not being an evaluation in place, by experts, in much the same way that heuristic evaluation (next paragraph) suffers. Additionally this technique suffers from having no users involved in this evaluation. Since it heavily relies on the expertís experience to do a reasonable assessment of usability the experts must have experience in evaluating technology that is off the desktop.
Nielsonís heuristic evaluation is a discount usability study that brings the experience of experts to bear on proposed designs. The expert evaluates the interface against Nielsons list 10 evaluation heuristics. While not including the user in the design or evaluation process it at least takes the step out of the lab, even though it is into the expertís working space.
Polson, Cognitive walkthroughs: a method for theory based evaluation of user interfaces.
Holtzblattís contextual design approach holds promise in that it does include the user in both the discovery and evaluation phases of interface usability testing.
Weiser and Brownís Coming of Calm Technology written at Xerox PARC dated 1996.
Jakob Nielsen's sites on Heuristic Evaluation and discount usability engineering.
Link to this Page