Merit Over Marketing: A Manifesto
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
The stories travel in whispers from client to consultant, colleague to colleague, designer to designer. They're told during phone calls, in conference rooms, over drinks. They're told when no one else is around to hear them, and quietly enough to be sure.
"My company contracted so-and-so to do a design overhaul and now we get more complaints than ever." And, "My friend heard so-and-so speak at a conference last year, and his advice was terrible." And, "We did what the so-and-so agency said to do and our conversion rate went down."
It's under a veil of darkness that we decide which consultant to hire, which speaker to listen to, which blog post to trust. Because so far, there's no vetting process — no reputable rating and review site for design service providers, no trustworthy, independent certification. Rather than ask for case studies, we trust who has the most Twitter followers. Rather than ask for evidence, we trust the biggest names.
We believe this kind of marketing for the same reason we do a lot of things: because if other people are doing it, it's probably the right thing to do. When deciding whether or not to jaywalk through a crowded intersection, this is perfectly acceptable. When choosing which consultant to spend thousands of dollars on, we need a higher burden of proof.
Even agency recommendations that come from a friend or colleague are tenuous unless that person has seen evidence of the agency's quality firsthand. Word-of-mouth is powerful because it's passed along by people we trust, but it has a fatal flaw: those doing the passing are just as susceptible to marketing as the rest of us. Minus evidence, even recommendations from our most trusted friends are meaningless.
Without proof of their effectiveness, the act of deciding who will do the best work is a coin toss, and the so-and-so's are holding double-headed silver dollars. The so-and-so's are winning. And too often, this results in mediocre solutions with unremarkable results.
When we accept marketing over merit, we pay more for lesser solutions. We believe what the speaker on stage tells us without further examination. We accept the work of a well-known agency as good without questioning the outcome.
It's time to start asking to see the other side of the coin.
And before you question whether this call to action is born from a jealousy of such well-known consultants and agencies, understand that I am one of them. I am one of the consultants who has benefitted from this kind of marketing. In appealing to you to ask for evidence of a designer's effectiveness, I am calling for my own inconvenience. I am asking you to make me prove I have the experience, talent, skill, and knowledge to help you achieve your goals, and that my ideas are worth acting on.
Most often, the companies who come to me do so just after someone there has finished reading one of my books. Others are referred to me by a colleague who did. Some reach out after hearing me speak at a conference. Others haven't read or seen anything I've done — they've merely heard that I have done these things.
In other words, they come to me not because of my design work, but because of my writing and speaking. And they stop asking questions right there. They simply accept that because I have done these things, I must be good enough. They don't ask for proof.
I have been a full-time consultant for four-and-a-half years, and I can count the times I've been asked for my portfolio on two hands. And because that question crosses a client's lips so rarely, I have never even bothered to build a portfolio site. Each time someone has asked, I have rushed to throw together a collection of design strategy documents and screenshots, zipped them up, and emailed them with a bare-minimum explanation of what I did and how it turned out.
By believing what we hear over what we see, we tell consultants and agencies that marketing is a more valued skill than design. It's time to turn that around. It's time to call for merit over marketing. It's time to ask for evidence.
Fortunately, from speakers and consultants alike, even as difficult as it can be to track and demonstrate the value of design work, there is a load of evidence to be had.
Case Studies
Whether described on an agency website, over the phone, or during a conference presentation, case studies are invaluable. Very little can be communicated through images alone, as they can't effectively show the decisions that went into a project or their effects. Case studies solve this by offering insight into the problem at hand, the design thinking that went into its solution, and a summary of the outcome.
Ask to see (or hear about) these case studies. Find out what makes this designer or agency the right one. Then ask about a case where a design failed to meet expectations, and why it happened.
Data
While determining which numbers to gather and how to evaluate them is a subject well beyond the scope of this article, the point stands: there is arguably no better way to prove a design's effectiveness than through cold, hard data. Click paths and other stats from site metrics, A/B tests, usability test results, and the effects of design changes on revenue are all ways that data can demonstrate the value of a design decision.
Ask your designer to look at the data and explain what can be learned from it.
Research
Conventions and best practices are easy enough to discover through a little crafty Googling and by studying successful sites. Beyond that, though, there are indeed companies who cite data when they present their research. User Interface Engineering, Jakob Nielsen, and the Pew Internet frequently publish the studies they've performed, and they're chock full of data, and even hints about how to do your own studies. Whatever the question, chances are someone has done some helpful research and has published it online.
Ask your designer if there's research to back up his recommendation.
Observations
Any designer who's been at it for a while should have loads of personal observations to pull from. They come from being involved in usability studies, analyzing site metrics, fielding customer complaints, talking to users, and hearing anecdotes from colleagues. And though not at all scientific, these insights are what make a designer a designer. They are what give a designer his or her instinct — that special sixth sense about what will work and what won't. It's frequently quite wise to trust a seasoned designer's instincts, but don't give that trust away based on a resume.
Ask your designer what she's seen before to explain why she believes what is saying.
Examples
One effective way to think out a design idea is to find other sites that have done similar things and consider what's good or bad about it, and why. Extrapolating lessons from other peoples' work can help you improve your own.
Ask your designer if other companies have done what he is proposing, and if so, how it can be improved.
Explain Yourself
However you do it, and whether it's a consultant, agency, or conference speaker you're questioning, ask to see evidence to back up his or her belief. It is your obligation to yourself and to your business to ask your designers to rationalize their recommendations. A vetted recommendation is always stronger than a baseless opinion.
If there is no clear evidence, ask her to articulate why her hunch is her hunch. Be skeptical of sentences that start with, "I think." Ask for sentences that include "because."
"I think" doesn't matter. "Because" matters.
Be skeptical of speakers and consultants alike who espouse universal truths like they've just come down from a mountaintop with a stone tablet in each arm. There are no universal truths in design. What's right for one situation may not be right for yours.
Be skeptical of consultants who can't explain the Why behind their recommendations, even (or perhaps especially) if that consultant is a known expert.
Be just as skeptical of the agency with 100,000 Twitter followers as you would of your own team.
Whatever the problem, and however you approach the solution, don't just take someone's word for it.
Ask for the evidence.