Three big questions to ask when analyzing big data

According to a report commissioned by IBM with Forrester, 90% percent of firms say personalization is imperative to their overall business strategies. The report suggests that to achieve personalization companies must better utilize all the data they have. There is only one small problem. Half the regular people surveyed agreed that brands are trying to get to know them, but they have not seen their shopping experience improve as a result. In part this might be because the intent of many personalization strategies is not to better serve people's needs but sell more stuff or reduce the cost of customer service. Even when companies do aim to deliver a more relevant and beneficial experience, however, I am not sure that true personalization is ever achievable. We can improve the probability of a successful interaction, but we can never guarantee it. 

What if personalization is like turning lead into gold?
How can companies deliver a more personalized shopping experience? Given that IBM sponsored the Forrester report perhaps the answer should come as no surprise. Harness big data and use it better. The report claims,

"Even with immature personalization strategies, firms see an almost 6% increase in sales revenue, a 33% increase in customer loyalty and engagement, and an 11% decrease in marketing costs."

Honestly, that sounds pretty good. But what if that is as good as it gets? What if the quest for mature personalization turns out to be on a par with turning lead into gold? (Apparently, the alchemists were onto something, you can turn lead into gold, but the input of energy required is so great that the cost would dwarf the value of the gold obtained.)

Lots of data is not the same as valuable data
It is so easy to accept statements about the power of big data and analysis at face value. Take for example this McKinsey & Company article that recommends companies use "data and analytics to harness predictive insights to connect more closely with their customers, anticipate behaviors, and identify CX issues and opportunities in real time."

Sounds good. Central to the pitch is the fact that,

"Today, companies can regularly, lawfully, and seamlessly collect smartphone and interaction data from across their customer, financial, and operations systems, yielding deep insights about their customers."

But having access to lots of data does not guarantee that data is useful or valuable. It may even be misleading. Here are three big questions to ask yourself when analyzing big data.

1) Is your legacy data the right place to start?
At best, the type of data that McKinsey talks about is obtained from how business is done today. It is derived from how customers interact with a company's current systems. And all too often those systems have been designed to be as cost efficient as possible, shifting the burden of solving problems onto the customer. True competitive advantage comes from being customer-centric, but the current experience offered by many companies is deliberately self-serving (pun intended).

Think about your own experience. Have you tried to change something with your healthcare provider, telecom company, or insurance company recently? How about your own company's HR system? Notice how "self-serve" is touted as a good thing, but ends up wasting your time and often adding to your frustration? Worse, most self-serve systems appear to be set up with no clue what real people might want, what words they might use, and how they might think to search for something.

When it comes to trying to create a better digital or customer service experience, as the old joke has it, "If I were you, I would not start from here." If you think I am wrong, try sitting down and watching your Mum try to navigate your website, customer service system, or app. Because to paraphrase David Ogilvy, "The customer is not a moron. She's your mother." By all means analyze the data you have to understand the current roadblocks, but it is not going to give you the deep insights you need to outperform your competition. Instead, I would start with a clean sheet of paper and a long conversation with the people who are trying to use your existing system. Old fashioned maybe, but a lot more likely to trigger a transformational insight.

2) Is your data as accurate as you might want?
This article provides a nice primer on using data and explores the idea that data is the new oil, listing reasons why that analogy applies and why it does not. One of the reasons that the analogy does not apply is that oil is consumed, and data is not. As author Avol Mavuduru states,

"What this means is that data is an asset that doesn't have to go away and can remain useful for a long time."

But is longevity really an asset or can it also be a deficit?

Somewhere out there in the digital marketing ecosystem is a database which has me classified as female. At least, I assume that is the case based on the ads for female apparel that regularly show up on some of the web sites I visit. This has been going on for years.

Maybe I clicked on an ad for dresses by mistake? I do not remember doing so. However, if I did, the database will not forget. Or maybe I searched for too many recipes? The analytics might infer that I was female as a result. Whatever the reason, I am sure there is a probability attached to that classification. Maybe it is only a 51% probability that I am female based on my online behavior, but the problem is that the targeting algorithm treats that probability as a certainty, guaranteeing it is 100% incorrect. And that mistake is going to last until someone purges the database.

In practice classifying me as female might not matter much for advertising purposes. It might drive down the advertiser's ROI marginally, while incrementally improving overall response rates. But when it comes to better serving an individual customer to make a sale or help with a service complaint being accurate matters a lot. So, make sure that your algorithms are working with the most accurate, complete, and up-to-date data set possible.

3) Do you really have all the data you need?
Our behavior is the result of far more than just interaction with one company's systems. Despite increasing surveillance online and in-store, much of our lives is still unobserved. And the blind spot is far greater than just assuming behavior only happens in digital channels. Real personalization is more than context specific; it is moment specific. But a lot of potentially conflicting forces dictate how a customer responds to anything in the moment – including your customer service system.

You got data on all that? No. So be very cautious about making claims for what big data can really tell you about how and why people will respond in the moment. A misdirected ad will not lose you a customer. A fumbled customer service enquiry may well do.

A journey worth taking
Unfortunately, no matter how much data there is available today, I believe we are still a long, long way from being able to anticipate people's specific behavior with any real accuracy no matter what the hype merchants tell us.

That should not stop us trying to close that gap. Thinking of customer service specifically, a badly designed system can leave customers feeling incompetent, confused, and angry. They needed help and what they got was an obstacle course. Far better to use data wisely and create a system that can flex to the needs of the individual and solve them quickly and easily, leaving the user feeling satisfied, grateful, and enthusiastic. The end result should be more retained customers, lower operating costs, and more positive word of mouth. But to achieve those benefits, companies really need to make sure they are analyzing the right data, not all data. 

So, what do you think? Is big data lead, oil, or gold? Please share your thoughts.