Big Data: Asking the right questions

In this interview by Mac Slocum over at O'Reilly Radar, Alistair Croll, Strata online program chair, weighs in on the differences between Big Data and more traditional Business Intelligence.

Big data is a successor to traditional BI, and in that respect, there's bound to be some bloodshed. But both BI and big data are trying to do the same thing: answer questions. If big data gets businesses asking better questions, it's good for everyone.
Big data is different from BI in three main ways:

1. It's about more data than BI, and this is certainly a traditional definition of big data.
2. It's about faster data than BI, which means exploration and interactivity, and in some cases delivering results in less time than it takes to load a web page.
3. It's about unstructured data, which we only decide how to use after we've collected it and need algorithms and interactivity in order to find the patterns it contains.

When traditional BI bumps up against the edges of big, fast, or unstructured, that's when big data takes over. So, it's likely that in a few years we'll ask a business question, and the tools themselves will decide if they can use traditional relational databases and data warehouses or if they should send the task to a different architecture based on its processing requirements.
What's obvious to anyone on either side of the BI/big data fence is that the importance of asking the right questions — and the business value of doing so — has gone way, way up.

Croll's last point--that asking the right questions is critical--bears highlighting. There are many reasons that traditional data warehousing and business intelligence has been, in the main, a disappointment. However, I'd argue that one big reason is that most companies never figured out what sort of answers would lead to actionable, valuable business results.
After all, while there is a kernel of truth to the oft-repeated data warehousing fable about diapers and beer sales, that data never led to any shelves being rearranged.