———————

The soul of remote research is that it lets you conduct what we call Time-Aware Research.

By now UX researchers are familiar with the importance of understanding the usage context of an interface–the physical environment where people are normally using an interface. Remote research opens the door to conducting research that also happens at the moment in people’s real lives when they’re performing a task of interest. This is possible because of live recruiting (the subject of Chapter 3), a method which that allows you to instantly recruit people who are right in the middle of performing the task you’re interested in, using anything from the Web to text messages. Time-awareness in research makes all the difference in user motivation: it means that users are personally invested in what they’re doing, because they’re doing it for their own reasons, not because you’re directing them to; they would have done it whether or not they were in your study.

Consider the difference between these two scenarios:

1. You’ve been recruited for some sort of computer study. The moderator shows you this online map Web app you’ve never heard of, and asks you to use it to find some random place you’ve never heard of. It’s This task is a little tricky, but since you’re sitting in this quiet lab and focusing–and they’re not going to let you can’t collect your incentive check and leave until you finish–you figure it out eventually. Not so bad.

2. You’ve been planning a family vacation for months, but you’ve been busy at work so you procrastinated a bit on the planning, and now it’s the morning of the trip and you’re trying to quickly print out directions between finishing your packing and getting your kids packed. Your coworker told you about this MapTool website you’ve never used before, so you decide to give it a shot, and it’s not so bad; that is, until you get stuck because you can’t find the freaking button to print out the directions, and you’re supposed to leave in an hour, but you can’t until you print these damn directions, but your kids are jumping up and down on their suitcases and asking you where everything is. Why can’t they just make this stupid crap easy to use? Isn’t it OBVIOUS what’s wrong with it? Haven’t they ever seen a REAL PERSON use it before???

Circumstances matter a lot in user research, and someone who’s using an interface in real life, for real purposes, is going to behave a lot differently–and give more accurate feedback–than someone who’s just being told to accomplish some little task to be able to collect an incentive check. Time-awareness is an important concept, so we’ll bring it up again throughout our Remote Research book to demonstrate how the concept relates to different aspects of the remote research process (recruiting, moderating, and so on).

(We understand that as a commercial entity, there is no legal premise of fair use for this image, so we’re clearly violating all kinds of copyrights by using it…)

Remember that diagram in Back to The Future II? Doc argues that messing with time has sent the world crashing hopelessly toward an alternate reality where things are horrible: the “Wrong 1985.” And that’s sort of what happens when you try to assign people a hypothetical task to do, at a time when they may or may not actually want to do it: you’re meddling with their time, and it’ll create results that look like the real thing but are all wrong.

When you schedule participants in advance and then ask them to pretend to care, you’re sending your research into the Wrong 1985. If you don’t want to create a time paradox–thereby ending the universe–you should do time-aware research.

In fact, a lot of liars can be screened out by writing a really good screener questionnaire. For example, here’s a decoy question that the Mozilla metrics team used in their recent Test Pilot survey.

This is really handy to know when live recruiting users from your website—the risk for fakers is even higher when the recruiting pool consists of anyone who comes to your website. Chapter 3 of our Remote Research book also touches on this subject, specifically as it relates to live recruiting. Here are two more pointers:

—Occasionally when people catch wind of a paid survey offer, they like to post it on “bargain hunting sites” like FatWallet. If you get a sudden surge of recruits, that may be the reason; check the referrer data in your traffic log or analytics to confirm where users are coming from.

—Use open-ended questions to test people’s motives for coming to the site. If someone responds to the question “Why did you come to the site today?” with a vague answer like “To check the offerings” or “Just looking around”, consider that a yellow flag, and follow up with more specific interview questions.

In an article for Boxes and Arrows, Paul Nuschke lists five phases of a usability study:

Step 1: Sales & Kickoff
Step 2: Recruitment
Step 3: Preparation
Step 4: Testing
Step 5: Analysis & Reporting

This post is about that second step, where you’re recruiting users to participate in your study. Traditionally this has been done one of two ways: either hire a third-party recruiting agency to find users according to specific criteria, or do it with your company’s contacts, typically customer / member email lists. Thing is, recruiting agencies are quite expensive, typically running around $200 or more per recruit (not including participant incentives), and even though they’re usually quite professional in their screening practices, who knows where they’re getting their users from? On the other hand, using your own company’s contacts bears another set of issues: what if you don’t have a very big list? What if you want to do several studies–do you bombard everybody with emails repeatedly? And so on.

Now, one of the best parts about remote research is that there’s no reason you have to schedule users in advance; since you test your users while they’re at their computers anyway, you can begin a study right when they agree to participate. Here’s where live recruiting comes in: by intercepting visitors with pop-ups or forms, you can intercept them, screen them, and call them within minutes of their arrival. This is a big advantage for lots of reasons: you can bypass much of the sometimes-lengthy recruiting step, you can have greater control and transparency over the source of your users, and most importantly, you can talk to real users who came to your site because they wanted to, not because they’re getting paid to.

So now the question is, how do you recruit users live? Well, you could hack together a form that users could use to opt-in to your study, but that requires you to hand-code the form and mess with your page content. Some remote research tools and web services like UserZoom and WebEffective offer Javascript-based intercept forms as part of their service, but they require you to sign up for the whole enchilada.

What does that leave? I am so glad you asked.

Ethnio is a free web-based service we made for the express purpose of recruiting users for remote research. All you do is stick one line of Javascript near the bottom of whichever page you want to recruit from, and users who visit that page will be greeted with a DHTML pop-up screener, which they can fill out in under a minute (here’s an example). You have complete control over the questions in the screener, so you can screen users however you want. When they finish, you see their responses come up immediately in a nice little recruiting table, and you can call whomever looks like a good fit for the research.

To illustrate, we have made a stop motion movie using felt. Please enjoy it with all your heart.

(Photo credit: bryanwright5 on flickr)