The Problem with “Big Data”

big data

One of the most popular terms that seems to be popping up on every technology site nowadays is big data. I think the industry is doing a disservice with its overuse and the inaccurate picture it depicts of what’s really happening.

Big data is a buzzword, or catch-phrase, used to describe a massive volume of both structured and unstructured data that is so large that it’s difficult to process using traditional database and software techniques. According to Webopedia

The problem is that big data is not just a big database. Big data is basically a 2-dimensional description. The problem is that companies are not just battling large databases, they’re battling the velocity of the data. Giant streams of data are coming in real-time that have to be normalized and presented in a way that provides an analysis of what’s occurring over time.

I believe a more accurate depiction might be streaming data. Streaming data has both the promise of finding nuggets of information that marketers can capitalize on, as well as real-time, trending and predictive analysis that can provide marketers with opportunities to adjust their strategy to maximize results. Systems have to normalize, archive, present and predict for us to truly capitalize on the massive data streams that are available.

Don’t be fooled by the marketing speak surrounding big data. The solutions are already present to process massive volumes of data. Tapping streaming data is what we’re really in need of.


  1. 1

    I completely agree with your definition and how “big data” has become the hot buzz word. I was having a conversation this morning with a colleague about “buzz words.”

    The problem is, with overuse, you water down the true purpose and meaning behind it until the majority who have heard and use it don’t really understand it. Similar things happened with “cloud computing” and the list goes on.

  2. 2
  3. 3

    Great article Doug. Tapping streaming data is the key! Pulling together data from internal system and external sources, joining it in real-time, cleansing the data, maybe do some fuzzy matching and then delivering insights, alerts and notifications to make it actionable is a beautiful thing. The companies that can move their marketing to real-time are going to have have a significant advantage. A company might start off using streaming data to get quick wins by creating a 10-15% bump in engagement, but they’ll soon find it has ancillary benefits to their manufacturing, sales, shipping, fulfillment, etc. This has been our experience.

What do you think?

This site uses Akismet to reduce spam. Learn how your comment data is processed.