Okay let me be up-front; I am bias on this issue. I think in-person card sorting is more useful than online card sorting. To be clear, online card sorting has value and am not saying otherwise, but I do think the quality of the data from in-person card sorting trumps the quantity you get from online.
So what’s card sorting?
If you’re unfamiliar with card sorting, it is simply THE best way to understand how your users organize information and the nomenclature they use to do it. For a website, the process would be to do a content inventory and list all of the end-points of your website on index cards (about 60 of them). Next, you’d invite a cross-section of your users (usually 2 to 4) to come in and organize the cards into buckets. I’d recommend at least four sessions (or until you start seeing clear trends) with different users.
In an open card sort, you’d let the users create and label the buckets to get a sense of what groups they would create and the nomenclature they would use. In a closed card sort, you’d give them the buckets and see what cards they put into which bucket (great for verifying your existing navigation and limited verification of your IA).
Quality vs. Quantity
But this article is about in-person vs. online card sorts. Typically, I would summarize the options as quality vs. quantity.
In-person card sorts are the quality. They are harder to pull together, require more recruiting, getting your users to show up while you monitor the exercise over 90 or so minutes and time to record the session and analyze the results. While time consuming, being able to listen to your users talk, debate and reason why each item belongs in a particular category and what they would label each category, is UX gold. The insight into how your users think is exactly what UX is all about and hard to get anywhere else.
Online card sorts add value as well and are the quantity. Whether you use OptimalSort, Websort or another online tool, the card sort is easy to set up and allows you to send it to any number of users you wish. They can complete the card sort when it is more convenient for them and the results are automatically combined into some very detailed reports. But you lose the ability to hear the rationale behind why users organized the information the way they did and why they created the category labels they did. Were they tired, did they make a mistake or simply not understand what information the card represented?
A Remote, In-Person Card Sort? Yes.
An important caveat to an in-person card sort is that your users don’t necessarily need to be near you. I know that seems a bit contradictory, but what I’ve done with success in the past is to gather remote users in a single location and video-conference a remote, in-person card sort.
Let me explain that.
While it is necessary to have your users all together so they can see and interact with each other, you don’t need to be there. Set up the card sort on one of the online services or in PowerPoint/Visio if that is easier and initiate a tele-conference. Project the online cards through the video-conference and ask the users to follow the same process and discuss how each of the cards should be sorted. I’d recommend bringing along another person to create categories and move the cards while you take notes, but you will be able to hear all of the conversations and debate about what goes where.
Ultimately if time allows, I recommend using both options to get the rationale from the in-person card sorts and more in-depth trends from the quantity of online card sorts. But if you can’t do both and have to choose, I’d personally recommend the in-person card sort.
I did warn you that I was bias, didn’t I?