— 29 April 2004 —

Heuristics vs. Usability: Evaluating the Vendor Experience

I won’t go in depth on what heuristics actually are and are not except to say that there are two reasons to do and not to do an heuristic evaluation:

Do, if:

  • You have no time/money/opportunity to do usability.
  • Your feedback will not have any impact on the implemented design, but you want to influence the next iteration of the product.

Do not do, if:

  • There are no standards or guidelines on which to build an heuristic checklist.
  • It’s only your opinion (however educated it might be) on the successes and failures of the product or service in question.

I am not a big fan of using an heuristic evaluation to “test” a product, but there have been times of late where I used the “rules of thumb” above on a few projects where I was brought in well into the development cycle. I’d much rather do usability (even if the feedback isn’t going to affect the release of the product) even just once with only 5 users than sit down and run a checklist test. The beauty an heuristics evaluation is that you can be sure the product meets or does not meet the needs of the checklist.

Recently, I found myself thrown into a project where we are only a couple of months from implementation. The product is a vendor app, and we will have very limited ability to change anything about it, for this release or perhaps ever. So, lacking time, the first thought was to do an heuristic evaluation. I am still not sure if there will ever be an opportunity to optimize the interface in future releases, but it is worth a shot.

Luckily though, one of the success factors of the project is buy in from the impacted areas; those receiving the new product. Even though the users will be using the product, the general feeling of those responsible for this effort is that the users should feel pretty good about making the change to the new product. The downside is there is no user access; all I get are SMEs.

Problem: I have no time to do usability, and I don’t want to do an heuristic evaluation. Solution: Do some heuristability. Yup, it’s hybrid time. The plan is to take our “rule of thumb” checklist (because thankfully there are standards and guidelines available), create a few scenarios based on the top two or three primary tasks for each user group, and give the checklist (with guidance) to the SMEs and have them run the scenarios and give the feedback.

Not at all an ideal situation, but I think the technical term is “rolling with it.” We’ll see how it goes. I’ve never done it this way before, but this way I get some interaction feedback, some interface feedback, and it will be done by people who are closer to representing the user groups than I am.

Any time you get someone telling you there is only one way to do find out if a product is useful, usable, and satisfying you are either talking to the VP of User Experience who actually “gets it” (see next paragraph), or you are talking to someone who is perceived as a roadblock to development. I don’t want to start an “is usability a science or an art” debate because the reality is, of course, it depends. You just hope that what you do doesn’t qualify you as a Mad Scientist (unless that’s your career goal), or that your work wouldn’t qualify for an NEA grant.

There is an ideal world out there that is attained by some companies (please hire me if this describes your company;) where usability and contextual inquiry drive changes to existing products and opportunities for new products. Where once change is initiated, iterative design and frequent testing drive what will be developed. Where visual design and interaction specifications are coded to. Where pilot (beta) testing includes usability validation testing. Where the process begins anew post-implementation. Where you are involved every step of the way.

But this harmonic convergence rarely occurs, and usually you have to do the best you can with what you have, especially when your company is implementing a vendor product.

So this is how you get to the point of doing heuristability. One of the upsides of knowing the rules is know when it’s okay to break them. My good friends Ron Zeno and Mike Rundle will agree, but will also remind us that when you start talking about usability, you just cannot be sure anyone even knows the rules.

Respond Eloquently Below

April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 April 2007 March 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 March 2004 February 2004 January 2004 December 2003 October 2003 September 2003 August 2003 July 2003 June 2003 May 2003 April 2003 March 2003 February 2003 January 2003 December 2002 November 2002 October 2002 September 2002
Snook Approved!