Write the Right Thing: Iterating on User Tests

20 May 2015

There’s a trap that people who write software often fall into. We can’t see our app in the same way a user sees it. This can sometimes lead us to create features that we think are super awesome, but it’s not actually what the users want. Or it’s too hard to use. Or it’s too confusing. Or maybe it’s just plain irrelevant.

One way we’ve found to remedy some of this is by observing people use our app. We call this “User Tests,” and we just wrapped up a round of user tests this week.

User Test Run Through

The first step is finding people to test your app. I suggest finding people mainly in your target audience, because then you can tailor the app to better suit their needs. This wasn’t too hard for us, because we took current users of some of our other apps.

Next, we get together with each user individually. Each session runs about 30 minutes to an hour. We bring them into a meeting room in our office with maybe 3 team members to watch.

When the test begins, we have the user download the app. We give them some free time to explore the app on their own. See if they find the features we want them to find. Take notes about what they are finding, or more importantly, what they aren’t finding.

After the user becomes a bit acquainted with the app, we ask them to perform small tasks. We hope that the user can very naturally figure out out to do them. Some tasks we had them perform are things like:

• Make a blog post
• Edit a blog post
• Change your blog’s title
• Upload a user icon

Depending on your app and its target actions, the tasks could get more complex than this. The main thing is that you observe their behavior in performing the tasks.

With this, we try and see how the user naturally tries to perform these tasks, and recognize sticking points. For example, our users tried to change their profile picture by tapping it. The app didn’t react, because the only way to change the icon was from the settings page. There’s one thing we can improve.

After running through the tasks, we talk with the user about the app. Do they understand what it’s for? What were some of the things they found confusing? How would they change the app? We try to keep this more of a conversation than an interview.

Info Crunch

After a user test session, there is usually a lot of raw data floating around. Everyone takes notes; sometimes we take videos of the session; the experience is still fresh. Immediately after the user leaves, we convene to compare notes.

The first meeting after the user test is basically one big brainstorming session. People still have a clear memory of the entire thing. Any ideas for improvement, big or small, are welcome. Sometimes all someone wants to do is change some wording, and sometimes we need to completely rethink some of the screens and interactions.

Depending on the outcome of the brainstorming session, there may or may not be other meetings later. The goal is to come up with the next step to take – what improvements we should make. How long this takes really depends on the changes that we want to make.

Iterate

Okay, so we got some user feedback. We determined what to change based on that feedback. We’ve gone an implemented the changes. Now the new and improved app is released. What do we do now?

Repeat!

I believe cycle of “User Test -> Improve -> Release -> Repeat” can be immensely beneficial to creating a quality piece of software that users want to use.

---

Pro Tips

Be Informal: I’ve found that when being informal, users will be more honest. When being formal, users try to give the right answer as opposed to the true answer, even if they don’t realize it. They answer with their brains, not their hearts.

Record Video: If the user is alright with it, definitely record the session. People’s memories are fallible, and having a video record can be an immeasurable resource.

Watch More Than Listen: Humans are very bad at knowing what they want. If the user’s action contradicts what they say, trust the actions. This is another reason to take video.

Spoken Thoughts: Have the user speak their thoughts out loud as they use the app. It gives you a really good idea as to what’s going on in their head as they use it. I’m talking about very natural thoughts, like “So I want to make a post. That’s what this plus button is for? Okay yep, this looks like a posting page. Oh, is this button for adding a image? Neat!”