— 18 June 2004 —
Disclosure: I live in an ivory tower.
Self-admonition: I need to start writing articles about the things I take for granted, because the topics tend to make people go WOW when others do it.
I get that most people don’t have the luxury of working in a development methodology that integrates usability (as a concept) at the front end of the cycle. But what’s up with making it seem like usability testing is useless? Yes, qualitative testing is valuable, and frankly it’s what I spend most of my time doing. There are even methods and tools for making your qualitative analysis more rigorous so you know you are getting good information and intelligence from your data.
But I continue to be befuddled by this kind of thinking represented in the Adaptive Path article. Yes, there is a difference between Usability and Usability Testing, but it is the same difference as Unit Testing and System Testing code. The first part you just see if things work as you think they should work. The second part things will work as they are prescribed to work, otherwise they are considered to be not working.
Let me set this up right here: there is no such thing as a usability issue/problem if all the requirements are satisfied. In reality of course there are usually still problems, but those problems always trace directly back to bad requirements. Requirements that weren’t specific, measurable, action-oriented, realistic and (if appropriate) time bound. And requirements gathering and analysis comes before design.
When done right, usability testing will improve your Web site and your development process, but the current culture surrounding Web site usability testing is such that it rarely benefits the design.
I agree with this to a point; but it is a myopic statement that doesn’t take into account a product lifecycle. Usability Testing often isn’t able to have direct impact on the current development cycle. But Usability Testing can have a big impact to a product between development cycles. Field testing can be done in a quantitative way to flesh out problem areas (and subsequently suggest design solutions), and help a company identify if the product is useable after say, 3 months of use.
Quantitative metrics are important for many reasons other than direct impact on users. Think also of tying the metrics to the bonuses Project/Product Managers get when they complete an implementation. Or to see if, over time, your business analysts and designers are doing a good job at gathering user requirements. Or perhaps you want a task to take more than 5 minutes… could happen… okay maybe not.
If the Adaptive Path article does anything it shows that, in this case as Web designers, we need to design with usability in mind. At this point I really want to say, “Well duh,” but again I live in a world where this is the de-facto was of doing IT business.
What I want is for you to think even bigger. Usability and Usability Testing should be a part of the development of a business case. Put it right smack in there with the section on Cost/Benefit. Then if the business case gets approved, and a project is funded, design with usability in mind. I find that everyone I work with on the design team, and the project team in general, buys into usability (and even usability testing) easily if I say one sentence: Just keep in mind that there are actual people who will be using what we build. Some of you are designated to take support calls, right?
That tends to get their attention.
We need to abandon the idea that user testing on the Web is a quantitative process. Focusing on numbers to the exclusion of other data leaves researchers with nothing more than noticeably dubious statements like, “Design A is 5% more usable than design B” (or “90% of all usability testing is useless”). Instead, user research for the Web should delve into the qualitative aspects of design to understand how and why people respond to what has been created, and, more importantly, how to apply that insight to future work.
I am fine with this if the first sentence adds on word: solely. Right in front of quantitative. As was pointed out in another comment on the Adaptive Path article [link], there are many contexts in which business should be very concerned about timings. Call centers is an excellent example. Suppose a company’s strategic direction is to go Web-based on all applications. Now, the call centers have been using a stable GUI for years and the new Web app, solely because it is a Web app will be slower. And in a call center if something goes 10% slower it means they need to hire 10% more people to cover the same amount of calls.
Usability Testing can measure if a product will be able to meet usability and performance objectives. If the designers did their job right, perhaps there will only be a 4% increase in on-call time. Usability Testing can measure this and funnel that information to the stakeholders who are interested in seeing just how many people they will have to hire “before this thing goes live.”
And here then we come to the crux of everything we do. How do you know usability testing is being done correctly? How do you know if you have a good designer? How do you know that requirements were gathered properly? The answer to all three, of course, is: you don’t. Not for the project at least. Perhaps over the life cycle of the product…
My assessment of the Adaptive Path article is that you can get rid of usability testing 90% of the time if you just have good designers that design with usability in mind. Again, how do you know they are good?
Just in the same way great writers aren’t born, they’re edited, great designers aren’t born, they’re usabilitied. Yes, have designers and developers and business analysts who believe in making useful, usable and satisfying products and services. But yes also test to see if you are making the products you think you are making.
Now. If you want to know more about how to infuse your corporate culture with Usability, while at the same time taking Usability Testing more seriously… Let me know. I can help you redesign your company with usability as a state-of-being goal.
This site has always been intended to be “for your edification and my pontification” only. But I just hate to see these articles every few months that make people drop their coffee cups and say, “Why didn’t I think of that?”
The thing is, I can almost guarantee that there are many people in your company who already think this way and want to do something about it. If you need a culture change in your company, promote and hire the people who already have the culture you need.
Enough ranting; let’s make the world a better place. Right now.