Performance Requirements

Usability objectives oft times include a timing element. Such as: The user shall enter all personal information and submit the application within two minutes, with no more than one error.

That’s a pretty typical usability objective. Generally speaking, I don’t often care how long a participant takes to complete a usability test task, though I will cut them off after 5 minutes or so. But dealing recently with “performance” people got me thinking.

The performance people were concerned with the performance of the system; that is the computer system. They always ask, “How fast does this need to happen?” Which makes me laugh because the answer is always, “As fast as possibly… in an instant.” Which never really happens.

We all know that those pesky users will notice a change (or lack thereof) after 250ms of taking an action. So I sometimes say to the system performance people, “I need it to paint the new screen in 251 milliseconds.” That makes them laugh. Truth is, for what I spend my time designing, time really isn’t of the essence. I am just looking for things to work quickly.

Quickly is a subjective view of course, which is why we are pushed to put actual timings into our objectives so that when test time arrives we can show to what extent we met the objective. Conceptually I have no problem with this. What I do have a problem with is that I focus on the human element, they focus on the computer system element. Instead we should be focusing on the task element.

Caveat for those who haven’t read here long or in a while: I work in a land where the job of design and development are separated. If these jobs were combined, I don’t think there would be much of a disconnect. The disconnect is easily overcome, but I find after almost 6 years of doing this, the disconnect still exists.

When I first started, I toed the line and added what I thought would be good times to beat in completing a task. But now, even when I am dinged for it, I come back with the defense that building a time-based usability objective is unrealistic as usability objectives are often defined (and rightly so) in the requirements stage of development. The other aspect of performance, the computer system, is not looked at until actual building of the new software. And that too is appropriate. Computer system performance is a technical constraint that must be dealt with on all software dev projects. And you don’t necessarily know what the technical constraints will be until you try to code the dang thing.

There is certainly a constraint based on the user half of the overall System of work, but that is a little more malleable. It also has far many more variables that are impossible to design for completely. The computer system on the other hand has limited variables (in comparison) and it is usually much easier to focus on building a computer system that functions quickly for a user to interact with.

And that’s why I don’t really care about timings from a user perspective. They don’t care about their time anyway. You heard me. :) They care about the computer being fast enough, not about the time it takes for them to enter information, make a decision, or understand copy on a screen.

One stated aspect of usability is efficiency. Performance is part of that. I say just build systems that seem fast. Always build in progress meters like the spinning hour glass, or the venerable, “Loading…” People are willing to wait for a screen to paint if they feel the wait is worth while and that they are making progress overall in the task. If the load time is more than say, 3 seconds, toss in the progress meter. Sure, why not.

I have almost always found usability objectives, as exampled above, to be passé. All I ever care about is could the participant complete the task, and how many times did they mess up. Then I can look at what kept them from success under that measurement.

I will bet most people reading have never written a usability objective before. I envy you. I get why it is done. You want to design a system that allows you to measure it, thereby defining the level of quality. And everyone loves quality. So to measure something, you need a goal to meet in the first place. And to be on the up-and-up, that goal needs to be stated before you build.