Sometimes Market Research Lies

Nissan is working on developing a new material for their cars. Studies show that consumers find the touch of this material to be one of the most enjoyable experiences. To make the material, Nissan is going to have to do careful work to get temperature, smoothness, and moisture into the perfect balance. What is this new wonderful material? It’s upholstery made to feel like human skin. Finally, someone is bringing the supreme reading comfort of the Necronomicon and other flesh-bound books to cars!

Research is great, but sometimes we make bad associations.

I’m not saying I’m smarter than the good folks at Nissan. They’re brilliant enough to make synthetic human skin. That sounds crazy difficult. But what I am saying is that research showing that human skin is something that people like to touch probably didn’t include the idea of using it in furniture. Have these guys heard of the uncanny valley? Maybe it will work because people will just think of it as synthetic leather, but if it actually feels like human flesh? Ain’t no one sitting in that car.

But remember, these are smart people! These are some of the brightest minds out there. It brings to mind the great toilet paper debacle of 2001. Oh, you don’t know about the great toilet paper debacle? It’s remarkable that a company spent $100 million dollars on a product launch that no one has ever heard of. The story goes like this: companies that make toilet paper were looking for the next big thing. They did buckets of research and found out that people often put water on toilet paper to…you know…whatever—this post is already gross enough without me explaining toilet paper to you. At any rate, all the research in the world told our intrepid heroes that making moist toilet paper would be a home run. In fact, one study by Kimberly Clark found that 60% of U.S. consumers thought using moist toilet paper (maybe we should just call it wet wipes?) was “cleaner and more refreshing than using toilet paper alone” (source).

Throwing money at a flawed plan just makes your problem more expensive.

Hundreds of millions of dollars and 10 years later, the wet toilet paper market still hasn’t gone anywhere. Despite an aggressive ad campaign, the product never went anywhere. Everyone—all the product people, all the financial advisers, all the investors—predicted that, based on extensive research, this product was going to be a winner. All the focus groups told them so. Why did research fail them? Probably because at the end of the day people aren’t willing to pay for a product that they don’t understand and they don’t feel like they need.

I wasn’t involved in any of the research but I suspect that it was a big case of confirmation bias. Remember, the people actually doing the research, driving the product, etc., are all incentivized to find new ways for their employer to make money. You come up with a plausible idea and you want it to be a winner. So you go and do research that will back up what you’ve probably already decided to do. It’s not that the researchers were dishonest; it’s more likely that they just didn’t dig deep enough because they found the answers they were looking for early on.

Framing questions changes the questions.

So we’re ready to free ourselves of biases by hiring a third party to do our research for us. Great! But that doesn’t solve the problem. A company hiring a third party will frame the problem. The third party does not strictly care about answering a question; they care most about making their client happy. It’s sort of like when you have a friend that you know is wrong but you don’t correct them because you don’t want them to get mad at you, except that in some cases not correcting them will lead them to waste tons of money and human capital. In that case, it’s more like dating.

All this headache may lead some to think research just isn’t worth it. After all, didn’t Steve Jobs say he never did market research because “customers don’t know what they want until we’ve shown them”? That’s a fun quote from Steve Jobs in 1982 right before the launch of the Lisa, a total flop. Granted, Steve Jobs didn’t launch the Lisa; he was taken off the project a few months before it launched, but the point still stands. The Macintosh launched in 1984 to some moderate success and then Steve Jobs left to start NeXT, which never found a market. In my reading of Walter Isaacson’s wonderful biography of Steve Jobs, it seemed it was only at this point that he started actually talking to his customers and then using those insights.

In Bob Gilbreath’s post on the subject of Steve Job’s research he writes, “Your job at the early stage of innovation is to get into the shoes of your customer, understand her life, and look for insights that give you ideas on products that you could create that would surprise and delight her. …[that] is exactly what Jobs was so good at.”

Contrary to popular belief, Steve Jobs did consumer research. He just didn’t suck at it.

What does a product look like when it’s a pure product of research?

The answer? Politicians! Every idea, every sound bite, every commercial, and everything else that politicians do are subject to focus groups. Politicians, or at least their public relations people, listen to groups of people, watch their body language, and poll like crazy. The end result is a product that’s watered down and nearly incomprehensible. In a recent episode of the podcast Planet Money, they did their own political focus group and found that people were frightened by the word eliminate. The researcher commented, “This is why candidates sound the way they do often times, because their statements have been focus-grouped to death. And it does mean they modulate their own opinions so strongly to try to line them up with American voters that probably some of their own views are lost.”

To not-quite-quote Shakespeare, “the fault, dear Brutus, is not in the focus groups, but in ourselves.” Research is very useful as long as when we look at the research we respond appropriately. Looking at the research, you have to know when to push consumers, when to hold back for more research, and even when to walk away. In the end, market research is a lot like poker.