How to Get Over Your Inaction on Big Data
February 26, 2014 Editor 0
It’s common to see surveys, polls, and reports showing that “most” organizations are embracing big data. For instance, a 2013 Gartner survey found that 64% of enterprises were deploying or planning big data projects, up from 58% the year before.
But I find these numbers hard to believe, for three reasons. First, they contradict what I’m seeing in the field. Second, they’re inconsistent with the history of technology. Yes, we live in what Ray Kurzweil calls an era of accelerating technological change, but no transformative application or device has ever achieved critical mass within three years. Finally, as Bill Schmarzo of EMC points out, most organizations are still feeling their way into big data. They’re at the low end of what Schmarzo calls the big-data “maturity” spectrum.
I would wager that for every Amazon, Apple, Facebook, Twitter, Netflix, and Google, there are still thousands of midsized and large organizations that are doing nothing with big data beyond giving it lip service.
One big barrier to implementation is the incessant noise around big data from consultants, vendors, and the media. The din leaves many, if not most, CXOs confused and intimidated. They wonder: Do we start small or large? Is big data just another IT project that can be run by a unit head? And what’s the ROI going to be, anyway?
To begin with, big data is not just another IT initiative. Traditional tools (business-intelligence applications, relational databases) simply can’t handle petabytes of unstructured information. Nor is it amenable to traditional discussions about return on investment. Outcomes of big-data deployment are inherently uncertain—predicting its ROI is an exercise in futility.
In fact, the beauty (and the horror, to some people) of big data is that you don’t know where it’s going to take you. You might discover fascinating and valuable insights. Or you might find nothing of interest—at least not yet.
That uncertainty doesn’t bother companies like Netflix, which exhibit a deep curiosity about their data. For example, Netflix goes so far as to analyze the color content of shows’ and movies’ packaging. If responses to certain colors or combinations are among the factors driving consumers’ choices among the company’s nearly 80,000 sub-genres of movies, Netflix wants to know.
The capabilities of companies on the mature end of Schmarzo’s spectrum are downright scary. But you can’t expect to go from zero to Netflix overnight. These firms have been capturing, storing, and analyzing massive amounts of data for more than a decade. During that time, they’ve followed a number of critical steps:
- They’ve accepted that big data requires a large commitment throughout the organization. Doing big data right necessitates the full support of the leadership and everyone in the organization.
- They’ve methodically built up their internal data repositories.
- They’ve invested in new tools.
- They think about big data’s ROI in holistic ways. Big data is not comparable to ERP and CRM applications. These suites lent themselves to traditional ROI calculations because they largely automated manual business processes. Rather than focusing on the costs of action, mature big-data users consider the costs of inaction. They don’t want to be playing catch-up to competitors that have figured out how to utilize new sources and forms of data. They’re painfully aware of the Matthew Effect.
If you’re starting on the big-data path, the first few months are critical. Look closely at your organization’s current data practices. If a company doesn’t do a good job of managing relatively small amounts of structured data, it isn’t likely to do well with big data. Does the company have a culture of making decisions based on data, or do politics, tradition, and policy rule the day? Is failure punished so severely that no one is willing to take risks?
These are all danger signs. If you squirm when reading this, don’t embark on the big-data voyage until your company has learned to manage small data well, has committed to basing its decisions on real information, and is agile and tolerant of risk-taking.
Assuming your company is ready to take on big data, the first year should serve as petri dish of sorts. What works? What doesn’t? Are certain data sources more valuable than others? Trying to do everything at once is never a good idea. It’s important to achieve—and communicate—little victories, such as getting employees to learn new tools and ask better questions of their data sources. It’s these victories that build the foundation for future insights.
It’s unlikely that your organization possesses the data and financial and human resources that Netflix does—but that shouldn’t dissuade you from embarking on your big-data journey, a point that I made in a previous post. A mere decade ago Netflix couldn’t collect, store, or analyze this type of information either.
So forget the ROI mind-set and the project mentality, and imbue big data into your overall business strategy.
- Does Bigger Data Lead to Better Decisions?
- Big data, interactive access: How Apache Drill makes it easy
- Big steps toward Ghana’s digital future
- Aren’t Medicine and Water More Important Than Telecommunications?
- Will Google and Facebook Drive the Drone Agenda in International Development?
- All things to all people: what the Internet means to our lives
Subscribe to our stories
- Digital transformation in the banking sector: surveys exploration and analytics August 3, 2020
- Why Let Others Disrupt You? Take the Smart Self-Disruption Journey! August 3, 2020
- 5 Tips for Crowdfunding During the Pandemic August 3, 2020
- innovation + africa; +639 new citations August 3, 2020
- SME Innovation: 10 Priorities for Support Post-COVID-19 July 7, 2020