— 7 January 2005 —

Performance Requirements

Usability objectives oft times include a timing element. Such as: The user shall enter all personal information and submit the application within two minutes, with no more than one error.

That’s a pretty typical usability objective. Generally speaking, I don’t often care how long a participant takes to complete a usability test task, though I will cut them off after 5 minutes or so. But dealing recently with “performance” people got me thinking.

The performance people were concerned with the performance of the system; that is the computer system. They always ask, “How fast does this need to happen?” Which makes me laugh because the answer is always, “As fast as possibly… in an instant.” Which never really happens.

We all know that those pesky users will notice a change (or lack thereof) after 250ms of taking an action. So I sometimes say to the system performance people, “I need it to paint the new screen in 251 milliseconds.” That makes them laugh. Truth is, for what I spend my time designing, time really isn’t of the essence. I am just looking for things to work quickly.

Quickly is a subjective view of course, which is why we are pushed to put actual timings into our objectives so that when test time arrives we can show to what extent we met the objective. Conceptually I have no problem with this. What I do have a problem with is that I focus on the human element, they focus on the computer system element. Instead we should be focusing on the task element.

Caveat for those who haven’t read here long or in a while: I work in a land where the job of design and development are separated. If these jobs were combined, I don’t think there would be much of a disconnect. The disconnect is easily overcome, but I find after almost 6 years of doing this, the disconnect still exists.

When I first started, I toed the line and added what I thought would be good times to beat in completing a task. But now, even when I am dinged for it, I come back with the defense that building a time-based usability objective is unrealistic as usability objectives are often defined (and rightly so) in the requirements stage of development. The other aspect of performance, the computer system, is not looked at until actual building of the new software. And that too is appropriate. Computer system performance is a technical constraint that must be dealt with on all software dev projects. And you don’t necessarily know what the technical constraints will be until you try to code the dang thing.

There is certainly a constraint based on the user half of the overall System of work, but that is a little more malleable. It also has far many more variables that are impossible to design for completely. The computer system on the other hand has limited variables (in comparison) and it is usually much easier to focus on building a computer system that functions quickly for a user to interact with.

And that’s why I don’t really care about timings from a user perspective. They don’t care about their time anyway. You heard me. :) They care about the computer being fast enough, not about the time it takes for them to enter information, make a decision, or understand copy on a screen.

One stated aspect of usability is efficiency. Performance is part of that. I say just build systems that seem fast. Always build in progress meters like the spinning hour glass, or the venerable, “Loading…” People are willing to wait for a screen to paint if they feel the wait is worth while and that they are making progress overall in the task. If the load time is more than say, 3 seconds, toss in the progress meter. Sure, why not.

I have almost always found usability objectives, as exampled above, to be passé. All I ever care about is could the participant complete the task, and how many times did they mess up. Then I can look at what kept them from success under that measurement.

I will bet most people reading have never written a usability objective before. I envy you. I get why it is done. You want to design a system that allows you to measure it, thereby defining the level of quality. And everyone loves quality. So to measure something, you need a goal to meet in the first place. And to be on the up-and-up, that goal needs to be stated before you build.

Respond Eloquently Below

April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 April 2007 March 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 March 2004 February 2004 January 2004 December 2003 October 2003 September 2003 August 2003 July 2003 June 2003 May 2003 April 2003 March 2003 February 2003 January 2003 December 2002 November 2002 October 2002 September 2002
Snook Approved!