— 28 June 2007 —

How To Run A Card Sort and A Little Bit of Why

Humans love to organize and categorize things. We do it all day, everyday. The trouble comes when one person, or a small group of people, sit down and decide how a set of items should be organized and categorized for everyone. Card Sorting takes a set of cards (imagine that!) with one item listed on each card. You’d then have someone organize the cards in such a way that it makes the most sense to them. It’s an easy, low cost way to gather data about (typically) the “best” way to organize a list of items. Which when I put it that way makes me wonder why I don’t see it being done more often.

I have a set of cards I use to make my case for doing a card sort. I took the receipt from a grocery run and wrote each item on a card. I can then give that set of cards to the project team and show them how similarly and (more importantly) differently they categorize things. It helps to have a list of items that everyone deals with, but that is not related to the work at hand. Once they’ve gone through this short exercise, it’s an easy sell to get actual users involved to help sort out the mess.

Why Would You Do a Card Sort?

Even if I wasn’t able to run people through my test sort, I’d start by asking a few questions… Is there a better way to organize the functions (navigation) of this application? Is the web site organized by how the company is run, or how the customers perceive the company and its offerings? Is this a new app/site? If the answer is yes, your why is built in. Unless it is acceptable to have poorly optimized navigation, silo-based design, or new functions that can’t be found… and for the record, if the functionality can’t be found you might as well have not built it.

Depending on the answers, a card sort with even a few customers will likely save you time (and therefore money) later on. It doesn’t take much prep, work, or time to run a card sort and make decisions based on the data. Of course, depending on your sample size and how confident you want to be in your results, you may want to calculate a bit before you start. :)

That calculator may tell you to get 1056 people to test, but if you are fine with “good enough” results like most businesses who are not NASA are, you’re probably wasting your time after 10 people. You will continue to get valuable data with more people, but practically speaking you likely won’t have time to do more than 10. But if you do have more time, go for it.

The big reason why, though, is that it’s a ridiculously easy way to get feedback from customers. Thus making connections with real people you can then contact at a later date for usability testing or user research.

Who Should Run the Sort?

Contrary to what you think I might say, you don’t need a usability person to run your sort. Anybody can do this and that’s why I feel this can be such a powerful tool in your design toolbox.

As we all know, everyone designs; they can’t help it. My feeling, as I have said before, is that we usability and design people shouldn’t try to stop “the others” from designing. We should make them more comfortable with some of the tools we use so they don’t muck it up before they start.

A common case is a developer who gets a Word doc with “requirements” to build a new app. Said developer must code the back end as well as the UI. Said developer could spend 5 hours over the course of a week and have the menu structure set without much fuss just by running a card sort.

Preparing for the Sort

Preparation is important as you don’t want to waste your time, nor the time of your participant. Included all the items you want feedback on. I know this is a “duh” statement, but you’re preping here and it’s good to make sure so you don’t forget one like I did once. :)

Write one item per card with as similar handwriting as possible. If you do all caps on a couple of cards, your participants may end up grouping those together. Believe me, it’ll happen. Make two or three copies of your cards as back up or so you can run multiple sessions at once. I’ve done one session of 30 people in teams of three. It was interesting. Good thing I had extra cards.

Write up instructions (see below) so all participants get the same spiel.

Provide blank cards and pens as your participants will likely need them. You’ll forget something. Depending on the user group, they might know something you missed. They may want to rename something and you don’t want your original set messed up.

Running the Sort

Make people as comfortable as possible. Similar to a usability test, your participant can do no wrong. Remind them of that. Twice. Wink the second time as company policy permits. Not in the “psych!” kind of way, but in the “I’m here for you in a totally fraternal, non-threatening way” way. Get some simple demographic data from each participant. Age, gender, experience with whatever your are working on, email for contacting later… things like that.

Make sure the cards are well shuffled before each participant starts. Have participants work on their own, but let them know they can ask questions at any time. Always answer initially a question with a question (and let participants know you will be doing that). You don’t want to lead them, even if they need to be led. Get them to try again, but don’t let them flounder. If they do, just tell them to put the card in a pile and move on dammit. Okay, not the dammit part.

Dealing with the Data

This is the part that takes the longest in my experience. It would be good to set forth from the start, if you are working with/for a project team, that one person (you) will make the initial pass at the data to come up with some findings. You don’t want to do this in a group or you might as well have just used the group in your study. Do whatever you can to limit discussion on how the data was gathered and what it means. Instead focus on an actionable set of possible solutions that the team can discuss.

Organize the data by participant. Some of that demographic data may tell you they whys of the participants’ choices.

For simple (small) lists, lay out all the cards in columns, in the order the participants set to look for correlative items and groupings. For complex (large) lists, consider using some of the available tools (both online and downloadable) to enter your data and see where the correlations lie. You can use some of these tools to collect the data initially, but I find it easier to have people play with index cards over learning how to use a tool just so your data collection life can be easier.

Do the least amount of work possible to get an answer you (and your project team if need be) are comfortable with as being accurate. Be prepared for data that tells you nothing. If the “right” answer is important, do the study again. If it’s not, meh… No worries. More than likely you’ll get information you can use, but it’s possible that if you run this by 10 people you’ll get 8 different ways of doing things.

*Instructions — Feel Free to Use and Modify

Welcome, Valued Customer/Client/Boss/Patsy/Evil Overlord!

Today I’d like you to use the cards in front of you/under your chair/in a safety deposit box located in Switzerland and arrange them in the order that makes the most sense to you.

There’s no wrong way to organize the cards. You can make one pile with all the cards — in which case you’re already finished, well done! Or you can make n piles with one card each — in which case I won’t like you because I already know I am not going to do that in the site/app/junk drawer that I am designing.

Do the exercise as quickly and as accurately as possible, but keep in mind it’s lunch time soon and I am hungry. You wouldn’t like me when I’m hungry, Mr. McGee. Yes, I know your name isn’t Mr. McGee, but play your cards right and it could be, Tiger!

Once you’ve finished organizing the piles, take one of the blank cards and a pen and write down what you thing the best title for each category should be. If you can think of a couple, write them all down in order of which you feel works best to least best. Sort of a on a scale of one to five how much bestness does each of your ideas have.

Please also fill out this short form so that I may contact you in the future with any follow up questions/because I’m lonely/to ask for bail money.

Thanks for agreeing to participate. Your work today will help me design the right interface/make me look good/fill up the “people time quota” set by my therapist.

Plenty has been written about Card Sorts, but I’ve been using them from time to time over my career and thought I would finally write some of this stuff down. Apparently I’ve been suffering from Familiarity Blindness and it didn’t occur to me until recently to write about something as “simple” as a card sort.

6 Responses to How To Run A Card Sort and A Little Bit of Why