• Userlytics. Another recording tool that records users’ facial expressions and audio comments as well as logging their webcam video (!) and screen movements on the website, according to a text script that you give participants. Then you watch flash videos of their experience and read summaries. Free trial, $299 per 5 testers and $47 for each additional tester.

• Trymyui. Very similar to usertesting.com and userlytics  - they find users for you, rate them, and make sure they are good at recording their own thoughts while they user your site or prototype.  Free trial and then $25 per user.

• IntuitionHQ. Very similar to Usabilla, you can create tasks and record where people click on a static images. The difference is that IntuitionHQ automatically creates static images of your live site, hence making it seem like people are interacting with a live site. They are not. But that’s cool, we actually really like these kind of tools for creating cool heatmaps of where people click on static images. $9 per test with unlimited users.

• Plainframe. This is the first UX tool I’ve heard of that let’s you create an interactive IA and site structure for users to interact with and track their behavior. You quickly put your navigation structure into a clickable “white site.”¹ They record the interactions as data for analysis, and let you play them back so you can see how the user interacts with the site dynamically. In Closed Beta as of May, 2010.

Moderated Tools

• UserVue by TechSmith. This is our trusty stand-by for moderated one-on-one interviews. Enables you to view a participant’s screen in real-time while talking to them on the phone. Any number of observers can join in to watch the session as it happens. The service has integrated calling and chat, and at the end of a session, the calling and screen recording are automatically synced and rendered to a video file (either WMV or Morae’s proprietary RDG video format). You can buy either month-long or year-long licenses.

• LiveLook. A browser-based screensharing service. We like this tool because it’s pretty cheap (they charge using prepaid minutes), no download is required, and it runs on Windows, Mac, and Linux–as long as you’ve got Java, it’ll work. It’s also very lightweight, so if you’re testing on a slow connection, this is one of your best bets. Drawbacks: no recording, no audio, and you have to give your users the account login to let them share their screen, which means that you’ll have to switch the password fairly often if you want to keep the account secure.

• VULab by York University. An open-source remote usability tool which, like UserVue, allows you to record video and audio on your participants’ computers as they perform tasks. They said that they’d be releasing it earlier this year, but we’re still waiting on it, so yeah.

• Skype 2.8 for Mac OS X. A new free feature of the popular internet communication client. Screenshare and video chat integrated; relatively cheap international calling, free domestic calls. Requires all participants, observers, researchers to have the Skype client installed.

Unmoderated / Automated Tools

• ClickTale Beta. Records user interaction with webpages using javascript. Provides “movies” of “in-page” behavior of users, as well as “heat maps” that visualize how people are clicking, scrolling, and entering data. Relatively cheap!

• UZ Self-Serve Edition by UserZoom. UserZoom is an international user experience research company specializing in remote testing, and they’ve now made their remote research tools available for do-it-yourself studies. It’s a completely web-based tool that allows you to manage multiple UX projects, gather clickstream data, prompt users to perform website tasks, card sorts, surveys, and recruit users from either a panel or from your own website. They also offer full-service UX testing.

• WebEffective by Keynote. Another web-based tool for conducting in-depth customer experience, branding and market research studies. Users answer survey questions and complete tasks in pop-up windows, with no download required. Keynote employs a big panel of web users (the “Keynote Research Panel”) to provide quantitative clickstream and behavioral data, survey feedback, structured task completion data. You can also intercept users from your own website.

• m-pathy Tracks mouse-movements and clicks without installing anything on the user’s computer. It’s in German, so you might need to go here to figure out what the heck their website is saying.

• RelevantView Provides card sort, surveys and questionnaires with branching logic,  clickstreams, and more. Comes in self-, partial-, and full-service flavors; full-service gets you complete study management, from designing the survey to recruiting panels of users to fulfilling incentives to analyzing data. Probably really expensive.

• ChalkMark by OptimalWorkshop. A brand-new service that allows users to complete tasks on static images, providing “heat map” feedback similar to ClickTale’s. It takes a “keep it simple” approach by limiting each task to a single click on a single image, so it’s best for relatively simple testing.

• SMT (Simple Mouse Tracking) by Luis Leiva. An open-source project, providing mouse-tracking functionality similar to that of VULabs.

• ClickHeat by LabsMedia. Another open source project, providing heatmap functionality similar to that of VULabs and ClickTale.

• Userfocus. Browser-based service with no download or install required. Uses a panel of paid researchers.

Moderated Research

Moderated research has the research facilitator (a.k.a. “moderator”) speaking directly to one or more participants. Examples of moderated research include one-on-one interviews, ethnographies, and group discussions. The major benefit of moderated research is that you can gather very in-depth qualitative feedback: not just opinions, but physical behavior, tone-of-voice, facial expression, and so on. A moderated discussion also allows the moderator to probe on new subjects as they arise over the course of a conversation, which makes the research more flexible in scope and makes it possible to explore interesting ideas and usages that were unforeseen during the planning phases of the study–we like to call these “emerging topics”.

Unmoderated / Automated Research

Unmoderated research is, of course, the complement of moderated research: the moderator does not speak directly with the participant, but instead uses a web-based tool or service to gather the feedback automatically (hence the alternate “Automated research” moniker). Typically, unmoderated research is used to gather quantitative feedback from a large (i.e. hundreds or more) sample. There’s all sorts of feedback you can get this way: you can use online surveys to get open- and closed-ended (multiple choice) opinions, use flash- or Ajax-based card sorting tools to understand the way users mentally categorize things, or use clickmaps and mouse tracking to see where users are clicking on a page to accomplish a particular task.

Which Type Should I Use?

Good question! It mostly depends on the specificity and nature of the thing you’re trying to find out. Are you trying to figure out if your webpage is generally easy-to-use, or trying to root out problems you might not have foreseen, or trying to get insight into the way your product fits into people’s lives? Then you’ll want to go with moderated research, which provides you with a very rich portrait of users’ behavior and usage context. On the other hand, are you trying to decide what color a webpage should be, where to place a particular button, or how to organize a navigation bar? For those specific small-scale questions, it’s probably best to go with an unmoderated method, which will allow you to get a broad look at how a large sample addresses a particular task or question.

We’ll be posting more on specific methodologies in the future; keep an eye out!

(Photo credits: foundphotoslj and racatumba on Flickr)

Rashmi Sinha and I created this graph of different UX research methods for User Experience Week in D.C. in 2006, and posted to remoteusability.com in 2007. This week, Christian Rohrers posted a similar yet more detailed graph of all user experience research methods on Alert Box. They are pretty similar, but I actually like his axis labels better, and the display of methods rather than tools. Especially because a bunch of the tools from 2006 are gone now. Ah, the web.