Inspection Methods

Cockton and Woolrych from back in 2001. Can’t find online version of the study, but Cockton’s site mentions it (no link).

Had 75 heuristic evaluators that only detected 74% of the actual problems. 5 problems experienced by users were totally missed by the evaluators. For the 14 overall accuarately predicted problems there were 71 heuristics applied, 61% of which ended up being inappropriately applied.

So why the hell do we continue to do heuristic (pr expert) evaluations?

Because they are cheap, fast, and often easily actionable on found problems. Doesn’t mean you are getting all the problems, but it should make you feel good.

Personally, I vote for heuristability.

One reply on “Inspection Methods”

  1. Cockton, among others, argues that expert evaluations are of limited value because many of the identified “problems” end up not being problems at all. Others suggest that expert evaluations tend to find problems that are the least important to fix. One might conclude that expert evaluations “make you feel good” only because they produce long lists of “problems” quickly. As long as no one looks at the quality of the problems found, you can keep on feeling good ;)

Comments are closed.