How Big is Big Data? – Twitter Edition

  2 m, 17 s

People talk about “Big Data” so much that it has become difficult to attach much significance to the term. It’s the new buzzword of choice in the analytics industry. To put it at its simplest, Big Data is really just oodles and oodles of data of every kind, produced at a rate that is getting hard to keep up with using existing technology. But my question is, how big is Big Data really?

Just saying there’s a lot of data out there is kind of like saying that outer space is really big. Yes, it’s true, but it doesn’t even begin to do justice to the vastly, hugely, mind-bogglingly big-ness of space.

Okay, so then how big IS Big Data?

Well, let’s take one specific example: say your marketing team would like to dedicate time and resources towards monitoring Twitter – and I mean everything on Twitter.

There are, on average, 278,000 tweets per minute. If tweets are an average of 15 words, that’s 4.17 million words published to Twitter each minute.

That’s the equivalent of 7 copies of Tolstoy’s War and Peace every sixty seconds.

If someone were to strip the text from every tweet as it was published, and then compile and print them, just one minute’s worth of Twitter would fill 9,267 pages.

If you were to then stack those pages together and measure the height of that stack:

    • In 1 minute it would measure 5 feet tall
  • In 1 hour it would be as tall as the length of a football field
  • In 4 hours it would reach the height of the Empire State Building
  • In 4 days it would be as tall as Mt. Everest
  • In 7 weeks it would reach outer space
  • And in 1 year that stack of paper would measure 512.8 miles tall

If you were a particularly sadistic sort of person, and decided to task someone with reading these pages:

  • Reading 1 minute’s worth of tweets would take them 9.7 days
  • Reading 1 hour’s worth of tweets would take them 1.6 years
  • And reading 1 day’s worth of tweets would take them 38 years

You would need to employ 14,000 people working 24/7 without interruption just in order to keep up.

So what’s the big (data) picture here?

Twitter is just one small contributor to text production, and text is just one of many forms of data to analyze. Like trying to explain just how big the universe is, answering the question “how big is big data” is difficult and best done using samples and clever points of comparison. People may say Big Data is overhyped, but when you look at the sheer numbers, “Big Data” is really a massive understatement.

But that’s why we’re here – to save you those 14,000 additional paychecks.

(You’re welcome).

Categories: Social Media, Text Analytics