— 20 March 2003 —
More class stuff. (Sorry about typos, I am on the laptop.)
The Evaluator Effect by Morten Hertzum, and Niels Ebbe Jacobsen. Another study to determine whether usability evaluators can detect (roughly) the same usability problems in a system. The study looked at Heuristic Evaluations, Cognitive Walkthroughs, and Think-aloud Evaluations. Guess what! All three methods do a lousy job at finding usability problems. Even when a team of evaluators uses two or more of the methods for the same system. Doesn’t seem to matter if you are a novice or expert evaluator, whether you’re looking for cosmetic or severe porblems, or simple or complex system. The gist of it (as usual) is that it is difficult for usability evaluators to agree on what is a problem.