Is big data in trouble?
10th March 2015 | Colin Strong
As organisations struggle to get value from big data, the solution lies in combining social science and consumer research to create commercial advantage.
Investment in big data is huge and set to grow further. IDC forecasts that global spending on business analytics will rise from $51.6 billion in 2014 to $89.6 billion in 2018. These huge sums are part of a ‘big data arms race’ by organisations looking for new ways to drive growth, profitability and market share. There are many ambitious projections for the way in which big data analytics will deliver. McKinsey, for example, have estimated that a retailer using big data can potentially increase their margin by more than 60%.
However, there have been murmurings that perhaps we are now in the ‘trough of disillusionment’ of big data, the hype around it having surpassed the reality of what it can deliver. Gartner suggested that the “gravitational pull of Big Data is now so strong that even people who haven’t a clue as to what it’s all about report that they’re running Big Data projects”. Indeed, their research with business decision makers suggests that organisations are struggling with how to get value from big data.
So what to do? I think that the answer, at least in part, lies in a report published by the Campaign for Social Science. The point made by the chair of the campaign, David Walker, sounds very pertinent to big data:
“Without a better grasp of people and their motives, technological advances may fail to realise their potential and may be frustrated or blocked.”
With this in mind, it is of concern that the big data analytics industry has to date mainly been led by those from ‘numeric’ disciplines such as statistics, computer science, applied mathematics, and economics. I would argue that these skills are necessary but hardly sufficient to get value from data. Because behind the data points, for marketeers at least, sit people. And where people are involved we move from linear systems which align to identifiable ‘laws’ to complex system which are non-linear in nature. Which means we need social science to guide us through big data, to provide the analytical frameworks that help us to explain the patterns in the data and to create hypotheses to test.
An example of this is the analysis of social relationships. We increasingly understand that many of our beliefs, attitudes and behaviours are shaped by our social connections rather than, as classical marketing would suggest, our own individual preferences and experiences. Historically we have not really had the tools to explore this; we have had to collect data from individuals rather than being able to see the pattern of relationships between them. Which has, of course, coloured our view of the way in which the world works.
But what big data can do is to put us in a position where we can look at the way in which social effects rather than individual preferences are shaping markets. There is now huge amount of data which tracks exactly this – phone logs, social media, messaging and so on. Studies by people such as Facebook’s Duncan Watts have demonstrated how these relationships are a key driver to preference formation in markets such as music downloads. There is plenty of evidence that they operate in other fields too, potentially giving brands a huge opportunity to find new ways of engaging consumers and creating attitude and behaviour shifts.
Academic researchers such as Scott Golder and Richard Macey are getting it. Now we just need the business world to catch up. Why? Because using a combination of social science and consumer researchers creates commercial advantage that can be hard to beat. And surely that’s worth looking at.
About Colin Strong: Colin Strong is a leading UK-based consumer researcher who has worked with a wide range of global brands to help shape their consumer strategies. Consumer data is an essential component of Colin's role, both in terms of using it to drive insight that was once the preserve of surveys and to advise on ways to shape new consumer brand relationships. Behavioural science runs throughout his research practice, not only to design experimental approaches, but also to guide data analytics. Colin is also a regular speaker at conferences and a contributor to publications and blogs including those of ESOMAR, the Market Research Society, The Huffington Post, Wired, AdMap and Market Leader.
Buy Humanizing Big Data today and save 20% with discount code MKTHBDB