Throughout the course of our summer camp series, we’ve covered popular marketing topics like lead nurturing and reporting, as well as other tech subjects like website design and product testing. Today, we’re joined by Pardot veteran and Usability Analyst Matt Miller, who is here to share a little bit more about what it takes to plan and conduct successful user testing — and how usability differs from user experience. Matt has a Masters in Human-Computer Interaction from Georgia Tech, and is a long-time member of the Pardot Monday-night soccer team (This Might Get Messi).
Take it away, Matt!
It seems like a lot of people get confused between ‘usability’ and ‘user experience.’ Can you explain the difference between the two?
Sure. Both usability and user experience revolve around how somebody looks at a thing that you built. Usability focuses on user research and asks questions like: can people do the thing you’re trying to help them do? Is it usable, i.e. can you accomplish a task? Is it useful, i.e. is that what you were trying to accomplish? It can be both usable and useful but still not be a pleasurable experience. That’s where user experience comes in.
Can you describe the basic process that you go through to conduct a user test? How often should you conduct user testing?
You want to test as early and as often as you can — definitely before you release a new feature, and ideally, multiple iterations thereafter. The first thing you want to do is make sure you have the right set of people — the people who will actually be using the feature you’re releasing or the website you’re testing. It kind of follows the scientific method from there.
Second, you’ll want to have a plan, and you’ll want to start recruiting. I typically try to document everything that I want to do and what I want to find out. A lot of what we do here is called “think aloud protocol,” where you have people use the product and talk aloud as they do it. You want them to be able to achieve a specific goal without telling them how to do it, which is why it’s important to have a script that allows you to go through the testing in a non-biased way. Keep an eye on your users’ behaviors, but not so much the “I like this” or “I don’t like this.” What’s more telling is the non-verbal stuff, like sighs and facial expressions. You’re looking for where they get confused and why, so that you can better understand what they’re trying to do and what they expect from whatever it is you’re trying to test.
In the end, you’ll analyze the results of your test. One method of doing this is called affinity mapping, where you write down all of your notes and stick them on a wall. You can then group similar notes and see where there are recurring instances of people getting stuck or misunderstanding a certain part of the test. When you see one issue come up more than another, then you can really dig in and try to better understand what’s happening to slow people down or trip people up. Once you’ve identified the problem, you bring the feedback to the rest of the team and suggest a solution.
You’ve been at Pardot since we first started putting a process behind user testing. What advice do you have for people who are just getting started?
Easy: just get started. Watching someone else use your stuff is eye-opening. You’ll pick up on things very quickly. From there, be open-minded — you need to remove the bias. Hear what’s actually happening, not just what you hope to hear.
How can Pardot clients get involved in Pardot user testing?
We regularly conduct interviews, usability tests, and beta tests for a whole host of features and improvements. Every time we roll out new Pardot features, we keep an eye on what people interact with and what they don’t. That helps us determine what we need to focus on next. If you’d like to get involved in our testing process, and get a chance to see future updates and provide feedback, then contact our UX team here!
What about marketers? How can they get involved?
Because marketing has an awareness of the industry, what’s out in market, competitive solutions, and client pain points, they understand the weaknesses you’re trying to solve for. They’re familiar with the story of how people use things, and that story forms the basis of why you’re going through another design, or trying to improve an existing feature set. Marketing also gets to tout all of the work that a UX organization does, and can even help with the recruiting efforts through marketing channels to get more participants involved in user testing.
Awesome, glad to hear that we marketers are helpful to you! Let’s move on to our fun questions. What is one thing you always wanted as a kid, but never got?
Super powers. I tried to use the force once, and was unable to move anything at all…
I’m sure that’s the story with plenty of kids, don’t worry. Okay, last one — if you could project yourself into the past, where would you go?
I would want to go to MIT in the 50s when the original members of the tech model railroad club were starting that, because that’s what led to the hacking culture.
Very cool! Alright folks, stay tuned for the last post in our summer camp series later this week, which will focus on how to use marketing automation with a small team (or even a team of one!). See you then!
