This chapter is from the book
This chapter is from the book
This chapter is from the book
Research Planning
Doing research, especially a larger research project with multiple locations, requires some planning up front for it to be successful. That planning involves finding subjects to research and locations for research, and figuring out the activities and interview questions that will get you the information you need. In short, you need to figure out who you are going to research and what you are trying to find out.
Designers can help themselves focus by creating a hunt statement before going out into the field. A hunt statement is a tool for narrowing down what the designer is researching and why. Hunt statements typically take this form: I am going to research X so that I can do Y. X is often an activity, and Y is usually a project goal or subject area. Here's an example: I am going to research how doctors use laptops on the job so that I can design a laptop for them. Hunt statements should be developed before doing research so that there is a purpose to each piece of research. The more specific the hunt statement, the better.
Costs and Time
One myth of design research is that it is expensive and time consuming. And while it can be—some rare design research projects cost millions of dollars and take place over years—most design research takes place over days or weeks and costs in the tens of thousands of dollars. It is time and money well spent.
The materials necessary for design research can be as simple as a notebook and a pen, or as complicated as specialized software and video-recording equipment. Ideally, a research team will have two of everything: two notebooks, two cameras (in case one breaks), and four pens. The research team itself should consist of (at least) two people who can trade off interviewing and moderating duties during research sessions.
The amount of time spent doing research can vary widely. Even a single day spent doing research will improve the outcome of the project. Ideally, however, designers will want enough time to interview and observe a representative group of users. In most cases, this will be more than 10 people, but fewer than 40. Time needs to be set aside not only for doing the research itself, but also for recruiting subjects, which can be quite time consuming itself. Generally speaking, most design research takes from a week to two months to execute from beginning to end.
Recruiting
The validity of your research data is entirely dependent on finding the right subjects to research. Before going into the field, determine who you should be speaking to and then try to find them. This requires figuring out a set of characteristics of the people you want to speak to. These can include basics such as age, gender, geographic location, and other "marketing segmentation" type characteristics. It should also include behavioral criteria such as level of expertise, attitude toward the product, and frequency or likelihood of product use or activity engagement.
Unless your users (or prospective users) are an extremely narrow group, some diversity is essential in order to make sure you are getting valuable research data and enough variety of viewpoints and, especially, behavior. Be careful to avoid unconscious bias in choosing subjects. As humans, we often unconsciously choose to engage with people who appear similar to us. In design research, unconscious bias might keep you from valuable subjects with markedly different viewpoints.
Once you have the criteria, you should figure out the number of users you will need for each criterion. A good, basic rule of thumb is around four to six subjects per major characteristic. Of course, you can combine criteria into clusters, such as "women 18–30 who are current users," which can help with recruiting. It helps to have a spreadsheet to track which criteria remain to be recruited.
You should create a screener to help make sure you are getting the right people. A screener is a set of initial questions to make sure a subject is a good fit and matches all the criteria you need. You want to make the screener as specific as possible so that you get the right combination of subjects. The screener should not only ask characteristic questions ("Are you female?" "What is your age?" and so on) but also specific questions in the subject area to make sure that the potential subject isn't lying about what they know. There are people who participate in research studies strictly for the money, regardless of whether they are qualified, so it behooves you to try to root out those people before you waste your time researching them. If, for example, you are doing a project for active stock traders, you may want to ask potential subjects about common ticker symbols such as Microsoft (MSFT) just to make sure they really are who they say they are. It'll probably become obvious during research that a subject is an impostor, but it is far better to find this out during a screener than in the midst of an interview. In order to get a large number of potential subjects to choose from, you should disclose the incentive before asking potential subjects the criteria questions.
There are firms that will recruit research subjects when given a screener. Often companies will have a list of customers to recruit from that they can check the screener against. There are also ways to recruit subjects over the Web, using sites like Craigslist, or using a tool like Bolt|Peters' Ethnio (Figure 4.1) to intercept potential subjects at particular Web sites with an online screener.
)
Figure 4.1 Research subjects can be recruited online using tools such as Ethnio, which essentially let you present your screener as a Web site popup.
However it is done, recruiting takes time. You can expect, in some cases, for recruiting to take as much time as the actual research itself, especially if the subjects are difficult to find.
Moderator Script
In order to make sure you are getting good data, you need to ask the right questions, so it is best to create a moderator script before doing a research session. A moderator script guides the person or people running a research session (the moderators) on what to say and in what order to say it.
Moderator scripts (sometimes called discussion guides or protocols) should contain not only questions designed to entice the right data from a subject, but also instructions to the person conducting the research. Research sessions can be stressful, and it is easy to forget basic tasks like turning the video camera on, or resetting an activity. Writing down those instructions will help make the sessions run more smoothly.
When possible, moderators should avoid questions that can be answered with a yes or no; instead focus on drawing out stories and answers to how, what, and why questions: How is this activity done? Why is it done this way? What tools do you use to do it? If I took those tools away, how would you do it? And so on.
You should avoid leading questions ("So how good is this product?") and instead present as neutral and objective a tone as possible. Remember that the research subjects are people in the unfamiliar situation of being interviewed and observed. They want to give researchers what they want and will look to the researcher for cues as to what they should say and do. Thus, you can easily influence their responses through the phrasing of your questions ("Wasn't that a confusing widget?"). Revise questions that might suggest a given (your) answer.
The moderator script should be treated as a living, working document. After a few sessions have been run, it will likely become clear where the script has to be tweaked and refined, so moderators should do so. There is also usually no harm, unless it eats up all your session time, in deviating from the script to follow an interesting or revealing line of questioning. In fact, this is to be encouraged (within reason). However, make sure your goals remain consistent so you don't deviate from the point of the research—refer to the hunt statement if you need to.