- Benefits of Qualitative Research
- User Experience Reviews
- Observational Customer Research
- Other Research Methods
- Case Study: Comcast
- Take a Step Back
- Up Next
- References
This chapter is from the book
This chapter is from the book
This chapter is from the book
Observational Customer Research
Directly experiencing a site can be a great way to generate questions that lead to test ideas, but it’s no substitute for spending time with real customers. As noted in Chapter 1, users aren’t very good at articulating exactly what they need, so rather than asking customers for their opinions, observational research instead pays close attention to their behavior as a way of understanding their perspective. Team members strive to watch customers interacting naturally with the site; they provide them with opportunities to give open-ended feedback and avoid guiding them too much or prompting them with leading questions.
Customers are recruited in a variety of ways; although some volunteer, they’re more often compensated through some form of payment, such as a check or gift card. Even though customers know they’re being observed, the goal of this type of study is to create as natural a setting as possible, so it may take place not only in research facilities, but also in cafés, customer homes, and other venues. Sessions are conducted one-on-one, with additional researchers observing behind one-way glass or through a video feed.
Depending on the business, sessions can last anywhere from 10 to 60 minutes. High-level trends usually start to emerge after watching four or five customers, but to be on the safe side, researchers often observe eight to ten customers during each study. The frequency of observational studies varies according to the company, but most businesses should conduct them at least quarterly. Many sizable and successful companies conduct this type of research whenever they have pressing unanswered questions about their business, which could arise several times per month.
The following list outlines the basic framework for this type of research, as well as some best practices for observing customers online:
Recruitment
- The first few times a business conducts this type of research, it may be helpful to go through a recruitment agency and use an expert moderator to conduct the sessions. Later, the business can recruit by placing an ad online on a site like Craigslist or work with existing customers contacted through an internal email list.
- It is useful to recruit customers based on an experience they recently had or an upcoming experience that the business can help with. For example, a hospitality business might recruit customers who have recently booked a hotel room online or who need to book a room for an upcoming trip.
- If possible, the customers should not know the name of the business, because this information might influence their behavior. For example, recruiters can tell them the study is about online user behavior related to the general category of tasks the business is interested in observing (purchasing clothing, reading the news, searching for general information, and so on).
Setting scenarios
- When customers arrive, the researcher welcomes them and asks questions that prompt them to talk about a wide range of recent online experiences, such as “What have you recently shopped for online?” or “Have you planned any trips recently?”
- The researcher provides customers with a computer or mobile device that allows them to browse the web while recording their online session and facial expressions. (Of course, customers are informed in advance that their session will be recorded.)
- The researcher invites the customers to go through a scenario they had mentioned earlier that pertains to the business (e.g., booking a hotel room for an upcoming stay). Customers are asked to speak aloud while using the computer to let the researcher know what they’re thinking.
Observations
- The researcher then sits back, pays close attention, and tries not to say anything. The key is to leave customers alone while carefully watching how they try to reach their goal—and what obstacles they face along the way—and paying close attention to shifts in body language, facial expressions, and so on.
- The researcher refrains from asking customers what they think about specific websites or designs. For example, the researcher does not ask whether certain sites are easy or helpful. Instead, the researcher focuses on what customers show through their actions; the more customers are prompted, the more likely they’ll be to say or do something based on what they think the researcher wants to hear. If customers get confused and ask what to do, the researcher simply says something like, “Please do whatever you would normally do.”
On prompting the customer
- Some customers have a tendency to remain silent throughout the session. If this is the case, the researcher will occasionally prompt them with a kind reminder to vocalize their thoughts.
- If, toward the end of the session, customers haven’t used the online business, the researcher may prompt them by saying something like, “I’m interested in seeing you do the same thing with a few specific websites.” The researcher will give the customers a list of sites that includes the business being studied as well as competitors not yet visited.
- The researcher should wait until the end of the session to ask any specific questions about the customer’s behavior, and even then, they should avoid asking for any opinions. For example, a researcher can ask customers to explain what they were thinking at a specific moment rather than asking if they thought the experience was good or not. When listening to their answers, the researcher should be on the lookout for a mismatch between what customers say and their observed actions. People tend to jump around online from website to website very quickly and usually have difficulty remembering all the actions they took online, let alone why they took them.
- At the end of the session, the researcher should thank the customers for their time and pay them the agreed-upon fee.
After the session, the researcher reviews the video to capture any findings that provide insights into the list of qualitative research questions. These insights, in turn, may lead to the formulation of new questions for further exploration in the iterative process.