At The Data Team, we realize that “big data” and “data science” are hyped and over-used terms, whereas in reality, organizations find it challenging to go beyond the initial hype and see real value from the data they have collected and the infrastructure they’ve set up to collect and analyze it. The main reasons are a lack of clarity on what to expect from “big data” and “data science”, and the absence of a mature strategy to leverage the data that’s being collected. In this post, we will demystify the term “big data” and then touch upon what constitutes “data as a strategy”. These two concepts are so closely related that the latter is in fact the framework that leverages the former at the right time. In a subsequent post, we’ll be dissecting the term “data science” – and we will relate this to data strategy as well.
Let us begin by seeing how the popular conceptions of “big data” fall short. Big data is not about the Three V’s as we are so often told by the popular media. After all, large volumes have been handled by massively parallel processing architectures for a while now (for instance, my ex-employer). Big Data is not about velocity alone either, since rapid ingestion and action on data too has been around from the time of transaction processing systems.
Big data is not about a use case. At The Data Team, we have come across innumerable companies claiming to offer “big data products” or “be” big data companies, whereas in reality, most of them play in the social media/digital marketing space. Social media or digital marketing is probably not the first use case your company will be solving with big data, since deriving value from social media requires a reasonably high penetration in various social media channels, needs firms to have some marketing maturity to take advantage of such a setting, and legal clearances. Big data should not, therefore, be mistakenly equated to some specific technology. It is therefore not a farm where all animals are equal and the elephant is more equal than the rest.
The hype around big data is, in one way, certainly justified. We postulate that this is because of the emphasis big data has placed on promoting a culture that uses data for furthering business. For decades, companies that have been called highly profitable have tried leveraging statistical and data driven methods to drive decision making, and hence, new entrants in this space, be they technologies or companies, are likely to get attention. This data culture demands of the organization the ability to allow anyone to analyze any data of any size by using any (combination of) tool to serve business objectives, thereby uniquely democratising the collection, storage and use of large data sets to take decisions. This data culture doesn’t obey traditional organizational boundaries like business and IT, and is motivated by feedback and sharing internally and externally, doesn’t shy away from large data sizes, and in fact thrives when challenged with frugality and complexity. Some of the tools the practitioners use have been around in the enterprise ecosystem for a while now, and some are relatively new. A select few are powered by research at the cutting edge of computer science (for example, deep learning).
We therefore argue that it is this data culture that is the fundamental disruption that big data has brought to the market. Not all companies have the need to analyse terabytes of information from day one. Companies might not need sophisticated data algorithms or the highly trained data scientists that write them. However, almost all companies have data, and that data, if used strategically, will impact their business positively. So, every B2B and B2C organisation needs to embrace this data culture in an evolutionary and yet holistic manner. This is verily the process of “Data as a Strategy”. A successful data strategy provides benefits that are immediate and revolutionary, and at the same time also charts a roadmap for growth and further data-derived benefits by incorporating big data principles into its fold.