Conducting user research and usability tests with real participants who match the audiences you’re targeting for your site or application is a perennial challenge. You may not have ready access to the kinds of people who match your target audience profile. You may struggle with coordinating schedules to agree on a time that works for everyone involved. You may find that your participants don’t show up on time—or worse, they don’t show up at all. You may find that if they do show up, they don’t match the kind of customer who’s most likely to use the site, in which case you realize only too late that you won’t be able to use whatever data you collect from the session.

So what can you do to make sure that you’re doing site research with the right users? Here are five easy steps:

Step 1: Define terms and criteria for recruiting
This includes (A) establishing what criteria the participants must meet, from demographic, psychographic, and/or business perspectives, and (B) agreeing on a fair means of incentivizing participants.

Establishing recruiting criteria is important because if you’re testing the usability of a financial services application, you’re going to want to weed out anyone who’s not intended to use that application. Another example: if you’re designing a site intended for use by the elderly, then you want to be sure you’re not recruiting 20-something’s fresh out of college. Standard criteria that often come into play include (A) demographics like age, income, and domestic lifestyle (married? kids?), (B) psychographics like level of expertise with computers or various web technologies and level of knowledge about the subject matter, and (C) business criteria such as whether they’re a current or prospective customer, and if they’re a current customer, how much of the product / service do they purchase or use in a given month/quarter/year? Of course, you may not need all three kinds of criteria. For example, demographics like income and gender may be moot if you’re doing research on a web application that involves scheduling a doctor’s appointment, or checking email.

(more…)

Stories go a long way when trying to communicate the value of User Centered Design (UCD). I have a great story to share. It starts with a grand jury inquiry into the usability of a dispatch and mobile response system used by the San Jose police department. The grand jury ruled that users (here, police officers) MUST be involved in the design of any future system to be used by officers (see finding and recommendation #4 from this Civil Grand Jury Report).

The grand jury ruled this because police officers weren’t involved and didn’t participate in the design of a system whose primary users were police officers. Contextual inquiry would have been invaluable: Simply have the system’s designers ride along with the officers to observe them in action. Usability would have been just as invaluable: test not merely whether the system works, but whether a given user can use the system to accomplish a specific task in a given situation or scenario.

Why weren’t users consulted up front? As with any project, there are lots of reasons why best practices weren’t followed. One reason is one we hear frequently: Many people assume that users or customers don’t need to be interviewed; their managers will do just fine, or customer support will substitute nicely for customers. In this case, instead of consulting users, management was consulted. The assumption is that managers know what workers need, so managers can field any questions about what users need in order to complete their work. A similar line of thinking is to talk to customer support instead of talking to customers, since customer support deals with customers on regular basis. But there are multiple problems with these lines of thinking. For example, both managers and customer support will introduce a bias. Moreover, customer support tends to be privy to only one aspect of the customer’s experience: namely, the “I’ve-got-a-problem-and-I’m-really-frustrated-and-I’ve-been-on-hold-for-10-minutes-listening-to-Muzak!!!” user.

The result of not following a user centered design process in this case?

Worst. ROI. Ever. Oodles of money, time, and resources spent on a system that in the end wasn’t usable—and thus, wasn’t used. And that’s just one consequence. A more dire consequence of the unusable system was endangering lives—not just the officers’ but those who looked to the officers for safety.

To learn more about the problems the officers faced with the usability of the system and how a UCD approach saved the day, check out the case study, “Almost Dead on Arrival” and also this NY Times article, “Wanted by the Police: A Good Interface.” I personally find this a great story that really drives home the importance of user centered design.