Michael Rawlins presented this at the CT UPA on March 4th:
View more presentations from Michael Rawlins.
Why Remote Usability Testing Kicks Ass – by Heybaloo
Cool write-up on heybaloo.com on two remote usability tools – usertesting.com and feedbackarmy.com. Excerpt:
“Remote testing neatly sidesteps these issues: the tester is probably sitting in his own house, using the site at his own pace. He’s probably on his own, so he’s happy to chat away and let you know what he thinks and what he’s doing the whole time.”
Remote Design Research at Interaction 10
Here are the slides from Nate Bolt’s talk on Remote Design Research at IxD 10 in Savannah, GA.
Remote Research at IxD10
View more presentations from bolt peters.
Using Remote Research to Inform Social Interaction Design (SxD)
There’s a guest post by Brynn Evans over on the Bolt | Peters Blog, on the topic of using remote research methods to inform social interaction design. What’s social interaction design, you ask?:
Social interaction design (SxD) is the practice of designing for person-to-person interactions mediated by a computer interface, going beyond pure usability and human-computer interaction. Even fairly solitary experiences like editing a Wikipedia page occur in a social context in which other users’ past interactions influence what new editors contribute.
Read it here!
Time-Aware Research
An excerpt from our forthcoming Remote Research book, out soon by Rosenfeld Media!
———————
The soul of remote research is that it lets you conduct what we call Time-Aware Research.
By now UX researchers are familiar with the importance of understanding the usage context of an interface–the physical environment where people are normally using an interface. Remote research opens the door to conducting research that also happens at the moment in people’s real lives when they’re performing a task of interest. This is possible because of live recruiting (the subject of Chapter 3), a method which that allows you to instantly recruit people who are right in the middle of performing the task you’re interested in, using anything from the Web to text messages. Time-awareness in research makes all the difference in user motivation: it means that users are personally invested in what they’re doing, because they’re doing it for their own reasons, not because you’re directing them to; they would have done it whether or not they were in your study.
Consider the difference between these two scenarios:
1. You’ve been recruited for some sort of computer study. The moderator shows you this online map Web app you’ve never heard of, and asks you to use it to find some random place you’ve never heard of. It’s This task is a little tricky, but since you’re sitting in this quiet lab and focusing–and they’re not going to let you can’t collect your incentive check and leave until you finish–you figure it out eventually. Not so bad.
2. You’ve been planning a family vacation for months, but you’ve been busy at work so you procrastinated a bit on the planning, and now it’s the morning of the trip and you’re trying to quickly print out directions between finishing your packing and getting your kids packed. Your coworker told you about this MapTool website you’ve never used before, so you decide to give it a shot, and it’s not so bad; that is, until you get stuck because you can’t find the freaking button to print out the directions, and you’re supposed to leave in an hour, but you can’t until you print these damn directions, but your kids are jumping up and down on their suitcases and asking you where everything is. Why can’t they just make this stupid crap easy to use? Isn’t it OBVIOUS what’s wrong with it? Haven’t they ever seen a REAL PERSON use it before???
Circumstances matter a lot in user research, and someone who’s using an interface in real life, for real purposes, is going to behave a lot differently–and give more accurate feedback–than someone who’s just being told to accomplish some little task to be able to collect an incentive check. Time-awareness is an important concept, so we’ll bring it up again throughout our Remote Research book to demonstrate how the concept relates to different aspects of the remote research process (recruiting, moderating, and so on).
(We understand that as a commercial entity, there is no legal premise of fair use for this image, so we’re clearly violating all kinds of copyrights by using it…)
Remember that diagram in Back to The Future II? Doc argues that messing with time has sent the world crashing hopelessly toward an alternate reality where things are horrible: the “Wrong 1985.” And that’s sort of what happens when you try to assign people a hypothetical task to do, at a time when they may or may not actually want to do it: you’re meddling with their time, and it’ll create results that look like the real thing but are all wrong.
When you schedule participants in advance and then ask them to pretend to care, you’re sending your research into the Wrong 1985. If you don’t want to create a time paradox–thereby ending the universe–you should do time-aware research.
UXCampVancouver 2009
At next weekend’s UXCampVancouver, Elizabeth Snowden will be giving a talk on remote usability methods. Check it out if you’re in the area!
Update: Sold out!
Screening Out Liars From Your Usability Study
A new article on 90 Percent of Everything discusses a few ways to screen out potential “fake users” who lie about their qualifications to participate in your study:
In fact, a lot of liars can be screened out by writing a really good screener questionnaire. For example, here’s a decoy question that the Mozilla metrics team used in their recent Test Pilot survey.
This is really handy to know when live recruiting users from your website—the risk for fakers is even higher when the recruiting pool consists of anyone who comes to your website. Chapter 3 of our Remote Research book also touches on this subject, specifically as it relates to live recruiting. Here are two more pointers:
—Occasionally when people catch wind of a paid survey offer, they like to post it on “bargain hunting sites” like FatWallet. If you get a sudden surge of recruits, that may be the reason; check the referrer data in your traffic log or analytics to confirm where users are coming from.
—Use open-ended questions to test people’s motives for coming to the site. If someone responds to the question “Why did you come to the site today?” with a vague answer like “To check the offerings” or “Just looking around”, consider that a yellow flag, and follow up with more specific interview questions.
22 Cheap or Free Web Usability Tools
Over at MarketingProfs’ 22 Cheap or Free Web Usability Tools series, our recruiting tool Ethnio gets a shout-out. Here are the pros and cons they mention:
Pros: Enables usability researchers to acquire actual users from the website for testing.
Cons: Researchers must be available when a participant response is received, and the tool is for recruiting participants only. A separate screen-sharing service is required, as is the usability testing researcher (with test protocols) to conduct the test.
Actually, both of those cons aren’t strictly true: while we encourage live recruiting to talk to your users right after recruiting them, you don’t necessarily have to; you can always just leave the recruiting screener on, contact your users at any time, and even schedule an in-person test at another time. Anyway, check out the article!
Oldie but Goodie
From UserCentered.net, an overview of moderated remote research, which they call “remote synchronous research”.
‘Remote user research and testing’ is where the user and the facilitator are in different places. Remote user research can fall into two categories, ‘synchronous’ and ‘asynchronous’.
- In synchronous protocols, a facilitator interacts with a participant who is remote and leads the research activities in real time.
- In asynchronous голова болит секс protocols, observers do not have access to the participants in real time, and there is no facilitator interacting with them during data collection.
Synchronous methods are enabled by online conferencing tools. Online conferencing tools allow two or more users to share their desktop, open an audio channel, record the video of the sharing session, send files or links via a chat, and much more. Usability practitioners now have a wide choice of online conferencing tools. The most famous are Web-Ex, GoToMeeting and Yugma, but there are many tools offering similar features on the market.
Remote Testing Tools Round-up by Liz Bacon
Liz Bacon of Devise maintains a great list of remote usability testing tools on Google Docs; check em out!
(We’ll probably have something like this in the near future as well; for now, check out our Tools page!)
