Conducting Design Research
Anthropologist Rick E. Robinson1 has outlined three main rules drawn from anthropology for conducting design research:
- You go to them. Designers shouldn't read other people's research on their research subjects from the comfort of their offices. Designers shouldn't make subjects come to them, to an artificial testing environment in an unfamiliar location. Observing the environment—where activities are performed—is an essential component of any research.
- You talk to them. Designers shouldn't just read about their subjects. Nor should they ask other people about them. Designers should have subjects tell their own stories in their own manner. The nuances of how a story is told can often tell a designer as much as the story itself.
- You write stuff down. The human memory is faulty. If designers can't write down what they see and hear directly as they do their research, then they should do so immediately afterward.
What Not to Do
Years of marketing methodology has left its mark. The first thing that most people think of when they think about talking to users is assembling a focus group. Don't do this. Focus groups are artificial constructs that, like juries, can be swayed and manipulated by strong participants in it, throwing off natural results. And that's to be expected—focus group facilitators assemble people into a synthetic group in an artificial setting (usually a conference room with a two-way mirror) and pepper them with scripted questions. This is not a good way to do design research. Rule #1: You go to them.
Nor is it a good idea to rely solely on the research of others, unless they are on the design team. Without knowing the circumstances and methods of the research, designers typically can't verify that the results are good and that they record the data that is most important to the designer: what the subjects did, said, or made, and the environment they were in. This dictum is especially true for data derived from marketing research. Marketing research typically focuses on demographics and attitudes—some of the least interesting data from a designer's point of view. Rule #2: You talk to them—emphasis on you.
Designers shouldn't rely on a videotape or transcript to capture what they need to remember. Reviewing audio or videotape is a tedious process and will seldom be done, except to find specific moments. Transcripts of tapes, while useful, take time to create even when using a transcription service, and the designer may need the information before the transcript is complete. And there is always that dreadful possible moment when the video camera malfunctions. Designers need to take their own research notes, both for safety and to focus their observations. Rule #3: You write stuff down.
When conducting research, designers should strive to treat their subjects ethically. Not only is this the right thing to do, but it will yield better results as well, since the subjects will likely open up more if they know and feel that they (and their data) are being treated well. Ethical research requires following these guidelines:
- Get informed consent from subjects. The designer should tell the subject that he or she is conducting a research study and explain the purpose. The subject must understand what is going on and agree to participate, preferably in writing. With studies involving minors, parental or guardian approval in writing is a necessity. An exception to this guideline is observations in public spaces where it would be impossible or impractical to get consent from everyone in view.
- Explain the risks and benefits of the study. Some studies carry with them risks. The designer may hear or see something that the subject doesn't want him to. The presence of a designer could be dangerous or make certain tasks cumbersome. But the designer should also explain what he or she hopes will improve as a result of the study ("We're going to build a better system for tracking shipments of ball bearings"), both to reassure the subject and to ensure good research results.
- Respect the subjects' privacy. Never use subjects' real names or other data that might identify them. Blur or hide faces in photographs. This will ensure that anything that subjects do or say won't have personal repercussions for them.
- Pay subjects for their time. People's time is valuable, and people who give some of it to provide insights to designers should be paid for it, at least a token amount. This payment doesn't necessarily have to be cash, although it should have value to the subjects.
- If asked, provide data and research results to subjects. Some subjects will want to see what you have recorded and the outcomes of the research. Designers should respect these requests.
What to Look For and How to Record It
When in the field, designers can get overwhelmed with the amount of data they are suddenly receiving. Often the designers are in a strange environment interacting with strangers. The newness of everything makes everything seem important. But the designer needs to focus on observing the things that are truly essential—namely, specific activities, the environment where activities take place, and the interactions among people that take place during activities.
Patterns and Phenomena
In the field, the main things a designer looks for are patterns and unique phenomena.
Patterns can be patterns of behavior, patterns in stories, patterns of responses to a question—any action or idea that keeps recurring. The rules of thumb are these:
- See or hear it once, it's a phenomenon. Write it down.
- See or hear it twice, it's either a coincidence or a pattern emerging. Write it down.
- See it or hear it three times, it's a pattern. Write it down.
Sometimes patterns won't emerge until after the research data has been analyzed (see Chapter 5). Sometimes a pattern is obvious in the midst of doing the research. Indeed, one good rule of thumb is that when you start noticing many patterns, you've likely done enough research to draw some meaningful conclusions.
Phenomena are interesting to a designer as well. Unusual behaviors—especially unusual methods of working—can suggest directions that will benefit larger numbers of people in their work. Say an accountant has created a different use for a spreadsheet; perhaps this approach can be built into the spreadsheet so that others can use it as well.
Writing down observations and key phrases is essential. Paper notebooks are best and less distracting than laptops or mobile devices, unless the environment is one where a notebook might seem more conspicuous, such as in an office environment.
All field notes should start the same way: recording the name of the person doing the research and the day, time, and place where the research is taking place. These details are crucial, especially for reference later in the project when these items can provide cues for recalling details. ("Remember that woman in the diner? The one we talked to last Tuesday. What did she say again?") Although the designer may record subjects' names and other data to provide compensation, for instance, this data should be kept separately from field notes, which should use pseudonyms instead of real names to preserve the anonymity of the subjects. Another thing to leave out, no matter how tempting, are personal opinions about the subjects, the observed activities, or overheard conversations. Doing otherwise is simply asking for trouble. Subjects, clients, and teammates may want to see the field notes, and showing bias in them is not only unprofessional, but bad research. Bias in research can't be helped, but it can (and should) be minimized.
It's a good idea, however, for the designer to have a separate area on the page to jot down thoughts and feelings that arise during the research sessions, including possible patterns. This should be a place to capture quick reflections or flashes of insight that can be explored more fully later.
Other findings that should be written down in the field notes are:
- Exact quotes with indications of emphasis and tone—Bob: "I sure do love these controls" (said ironically).
- Sketches of the location, annotated with comments and detail.
- The history, steps, and context of any activities.
Still pictures should be taken when and where feasible. Ideally, these will be printed, attached to the accompanying field notes, and annotated with captions or other notes. When taking pictures, make sure you are capturing not just the subject, but also the environment, any objects that are mentioned, and especially any activities being performed or demonstrated.