We've heard - and experienced - a lot of discussion about the technology hurdles of remote testing: picking the best testing tool, getting participants to download screensharing tools, dealing with proprietary software that participants don't have access to...Tech issues run the gamut.

But we hear considerably less about the interpersonal challenges of running a remote usability session. It’s not just about finding the right tool or getting your participant to download Webex; it’s about maintaining your participant’s motivation and engagement by showing (gasp!) empathy and compassion during a remote session.

Tried-and-True (But Face-to-Face)

Why, you may ask, are we inclined not to show empathy and compassion while moderating? Why do we adopt that passive, less-than-friendly approach for in-person tests? Well, that's tried-and-true: Participants sitting next to you shouldn't be your friend; if they're your friend, they're more likely to try to please you, and that means biased responses and biased feedback. You should be so neutral and mild that it almost seems like you're not even in the room with the participant. They should feel like they're just hanging out on the Internet, completing some tasks, talking to themselves. That's the moderator's role in the traditional usability test.

But what happens when your participant is across the state, the country, or the world?

A History of Silence

We’ve conducted dozens of remote usability tests for various clients over the past eighteen months, and we've had some disastrous (albeit hilarious!) interactions with our participants: from general lack of engagement to participants multitasking and checking email during a test. We’ve had people deteriorate from being uninterested, to bored, to nearly non-responsive over the course of a 60-minute test. (And we’re solid moderators! Just ask our clients.)

Here are a few gems:

"Hang on - let me take this call real quick."

"Hang on - let me pop open the system we currently use so I can show you how awful it is."

"Hang on - actually, can we cut this short? A tee time just opened up at my golf course."

This, of course, is very different from our experiences with in-person usability testing, where the participant is essentially trapped in a room with us and we're calling the shots. With face-to-face sessions, we've got nearly undivided attention. There's no personal email, no bosses demanding attention, no bookmarked websites that users want to complain about.

So why the drastic contrast? Well, we've done some experimenting, and we've confirmed some theories.

Moderator Roboto

Our new and improved approach to remote usability testing emerged after a series of unfortunate sessions. We sat down and identified the major structural differences between the two testing methodologies, then did some serious soul searching. Here's where that landed us:

What if it's us, the moderators?

Maybe it's not the tech hurdles or the distractions or the time commitment. Maybe - just maybe - that super-passive persona we assume as moderators of an in-person test (neutral, reserved, non-emotional) is not effective at maintaining users’ attention or willingness to complete a study remotely. Maybe because the participant and moderator are interacting via a computer, remote users treat a neutral moderator like...a computer—and not in an endearing C3PO or Rosie Jetson way.

Shiny Happy Participants

To test out our theory, we started conducting remote tests with a little more personality thrown in. We added some pizzazz, some sass, some chutzpah. And - in confirmation of our theory - since moving away from that traditional, neutral approach, we've had much more success engaging with our remote users and in turn getting some great, actionable feedback. Yes, we're straying from the standard "don't make friends" routine, but it's a necessary evil. Observe:

Participant: "Oh, I didn't see that button at first...but I'm old...so that's probably why."

Moderator: "You're not old, Barbara; you're just experienced! Let's try the next task."

Moderator: "What do you expect to happen after you select 'Save'?"

Participant: "Oh! Sorry; I already selected it."

Moderator: "You're too fast for me! Can you cover your eyes?"

Participant: "Hahaha. I will just press the back button."

So, at the end of the day, even if the tasks, protocol, and recruitment criteria are the same for a group of in-person participants and a group of remote participants, the rapport you build with participants online needs to be much more personable and friendly in order to get the type of high-value feedback that emerges from in-person tests. Showing your remote participants a little bit of empathy works wonders to keep them on track and get them talking - and they're less likely to try to please you because you're still a world away from them.

Published by Design For Use.