Beyond Big Data: Why Small Data is Taking Off

Glass image1

Big Data is everywhere. Yet many companies lack a clear vision for rolling out big data in practical, measured steps. At the same time, with social networking, BYOD, and expectations from interactions with brands like Amazon, Apple, and Nike, most employees have seen the potential, yet many aren’t equipped to harness this power in their workplace. The small data movement — which I’ve been writing about for the past 6 months (and thinking about for many years) — aims to address these challenges and re-envision the “last mile” of big data via consumer-style, more responsive, more social apps that truly turn insight into action.

Not surprisingly, this idea is getting a LOT of attention. In fact, over the past 6 months there’s been as much published about small data as there was in the previous 3 years. Clearly small data has gone big time as a theme, concept, and set of tools.

So why is small data taking off?

  • First, while understandably there’s a lot of excitement about big data, there’s a growing reality that doing it at scale and waiting for all the trickle down benefits can take a lot of time. Especially if you’re not in the C-suite.
  • Second, (and here’s where I jump on my soapbox), the last mile of big data is really where the value is created, opinions are formed, insights are shared, and action are made. By non data scientists. Everyday.
  • Third, there’s some great consumer examples of small data hitting the stage, most notably the much hyped but potentially game-changing Google Glass, which brings together small data and visualization with wearable computing, all in a semi-stylish package.

So who is fueling the small data movement? I wrote about 10 vendors who get the value of enabling simple, smart, responsive, socially aware tools and solutions in my last post. And a couple weeks back I wrote about 5 additional vendors and a new study I’m doing with Digital Clarity Group in a post on the DCG blog, which you can read here.

As small data takes off, I will continue to provide updates and thoughts both here and on the DCG blog. As always I’d love your thoughts and suggestions in the comments below.

Thanks for stopping by!


Small Data Moves into the Spotlight – 10 Vendors Worth Watching

In the 4 months since I started to research my last piece on small data for, I’ve had a LOT of conversations about why it’s time to bring the power of big data to the masses, who is doing something about it, and where to focus first. I’ve taken a lot of notes, and I’ve also had the good fortune to run my emerging ‘small data manifesto’ by some real smart folks including Nobby Akiha, Bill Blundon, Jeff Boehm, Dorie Clark, Mike Gualtieri, Esteban Kolsky, Mitch Lieberman, Richard Pasewark, and my colleague Scott Liewehr (I’m sure I’m leaving out some others).

What have I learned? First off, the small data concept is resonating with a lot of the folks I’ve met. And looking at the volume of new posts and articles since the start of the year – in places like Forbes, Inc, Xconomy, and a number of marketing blogs – there seems to be a groundswell building that points to the value of thinking small.

Second, I’ve refined my ‘watch list’ of vendors that are powering this movement and ‘get’ the value of a creating/enabling simple, smart, responsive, socially aware tools and solutions. I’ve intentionally tried to focus on specialty tool providers vs the IBMs and SAPs. And I know that I’ve just scratched the surface – these are 10 vendors you’ll want to know, but not the only vendors in the space! So here’s my list, in alphabetical order, with a few comments on why they fit the bill.

10 Vendors Worth Watching

Actuate – One of the dashboard and reporting pioneers and founder of the Eclipse BIRT open source project, Actuate’s track record promoting ‘BI for the masses’ is well established (disclaimer: I actually helped with some of this back in the day). Actuate’s recent purchase of Quiterian gives the company a leg up over some of its peers when it comes to combining big data analytics and small data delivery.

Attivio – On the heals of a $34M investment, Attivio is poised for big things with its next-gen database that pulls together data from multiple sources and offers to bridge the worlds of big and small data. I love the focus on correlation and breaking down silos, and making it easy to see both the big and small picture.

GoodData – Driven by a $25M series C found in mid-2012, GoodData has become one of the leaders in bringing big data to life for all types of businesses. How? There’s a lot of small data thinking at work, as a quick tour of the company’s blog illustrates. One of the poster-children for why small data will be a big business.

Google – Starting with search, I’d argue that Google was the original small data company. Simple? Check. Smart? Oh yeah. Mobile? Yup. Social. Ah…getting there! With it’s purchase of Wildfire and improvements in Google+ and YouTube, plus resources second to none, Google could be to small (and big) data what Microsoft was to PCs. Seriously.

NetBase – I love how NetBase (former client) has created its Brand Passion Index to make its high-end analytics (using NLP and text analytics and other cool stuff) approachable and fun. Plus another $9M in funding this past January and a key partnership with SAP is bringing its tools to the wider enterprise market. Great strategy.

Nimble – one of the pure-play social CRM vendors founded by GoldMine CRM founder Jon Ferrara, Nimble is all about simple, smart apps and tools that users will want to use. It’s clear these guys understand that if you drive adoption by focusing on the end-user experience, ROI will follow (Saleforce gets this too!).

QlikTech – In terms of powering simple, mobile, contextual apps QlikTech is very much aligned with the small data vision, and one of the more complete offerings in the space. Also announced partnership with Attivio in January.

Tableau – Driven by the goals of powering fast analytics for ‘everyone’ and storytelling on the Web, Tableau’s positioning is lock-step with the vision and opportunity of using small data to bring big data to the masses. I also really dig the company’s messaging and overall creative. Nicely done.

Twitter – By nature of enforcing a small view of messaging and communication, Twitter should be in the small data hall of fame. But it’s really the company’s recent purchase of Bluefin Labs that moves these guys to the head of the class. Brilliant move.

Visible Technologies – Coming at small data from a social analytics perspective, Visible has a super-intuitive dashboard product, and a great handle on making data highly consumable. It’s clear the management team gets the small data potential, and for good measure the company was just named one of 9 Twitter Certified Products partners.

So who would you add to my watch list?

Social Scorecard: Big Data Vendors Who Make the Grade

As social channels continue to feed the big data hype machine, I figured it would be insightful to fire up my favorite social search tool to see which vendors are making the grade (or dominating the conversation) when it comes to social discussions. This post summarizes what I found after a couple evenings of analysis, so it’s not meant to be exhaustive by any stretch, but hopefully provides some interesting insights – and a starting point for further analysis.  Thanks for checking it out!

For a first cut, I used Topsy, an excellent search tool we’ve used for social strategy projects at The Pulse, and took a look at mentions of the hashtag #bigdata along with 35 of the top vendors in the space (e.g., #bigdata AND Google) over a 30 day period ending before the end of November, from ALL sources (links, Tweets, Photos, Videos, Experts) on Twitter. Not surprisingly, IBM is dominating the discussion, with over 1000 mentions, followed by EMC (381), SAP (375), Google (355) and Microsoft (310).

Among upstarts, Cloudera was right behind Oracle and HP at 158 mentions, with Splunk (72) and MapR (58) getting their share of social buzz. Interestingly, some much bigger players like Hitachi (only 15 mentions), Xerox (8) and Fujitsu (8) are failing to generate much social chatter related to big data – at least on Twitter.

Here is the full roundup:

To get a better view of social activity weighted by resources I compared the social mentions to the latest vendor data from Wikibon – and more specifically looked at the ‘mentions’ rank vs the revenues rank. So IBM, with the top social mentions AND the top revenue, was right at mid-table (or ‘0’). While standouts (largest difference between social rank and revenue rank – what I’ll call ‘Social IQ’) were QlikTech, Tableau, MapR and Cloudera. Perhaps not a surprise since all are relatively young/small, but well-funded and clearly hitting the social trail big time.

Perhaps a surprise in this cut of the analysis is the strong performance of Microsoft, Oracle, SAP and EMC, showing that all are getting the word out and associating their offering with the big data movement. Note that if a vendor was not included in Wikibon they won’t be in the following chart!

Here’s the full results for the Social IQ of the top big data vendors based on mentions vs revenues rank:

So, who are the ‘social stars’ at this point? Among big companies, IBM, EMC, SAP, Microsoft and Oracle stand out.  And among upstarts, clearly QlikTech, Tableau and Cloudera are making the grade.

I’d love to hear your thoughts on other vendors I should include in this study. And please suggest other measures as well for the next cut at the analysis. Thanks again for stopping by!

A Small View of Big Data

Large organizations are in love with big data and big analytics. In fact, back in March, IDC forecast that the big data technology and services market is expected to grow from $3.2 billion in 2010 to $16.9 billion in 2015, while it’s been reported that Deloitte estimates the size of the big data market at $1.3-$1.5 billion in 2012. Yet many employees aren’t equipped to harness these insights in their everyday decision making. And perhaps even worse, many executives and boards are still oriented towards an ‘inside’ data view vs customer or employee engagement type data that has increasing value as social and mobile networks take off, as Barry Libert argues.

Meanwhile, in the consumer world, brands like Amazon, Apple and Nike are shifting their own big data engines into high gear to deliver more personalized offers, recommendations and experiences that drive loyalty and sales.

What’s the connection? While companies (and computers!) like big data. Most people only need small data. Not only in terms of what is ‘good enough’ when it comes to the size of the data set. But also in terms of:

  1. Delivering simple, consumer-style, self-service apps and devices vs. complex toolkits,
  2. Providing context-driven, highly accurate answers and explanations (note I said answers, not data),
  3. Applying the latest Responsive Web Design and social marketing techniques to deliver a great experience across all Web-enabled devices and sharing channels.

Google may do this better than anyone currently when it comes to consumer applications. And on the business front, one can argue that Salesforce has done the best job to-date blending enterprise functionality with social insights and Web-style ease of use to create CRM solutions that your typical employee actually wants to use.

Of course other vendors, brands and investors can benefit from this small data approach, if they focus on creating simple, smart, responsive, socially aware tools and solutions.

Who is driving this small data revolution – even if they don’t know it yet?  In terms of tools, my list would start with the new generation of speciality social analytics and business intelligence providers like Bluefin Labs, GoodData, NetBase, QlikTech, and Visible Technologies.  In future posts I plan to explore these companies and other vendor categories, look at the role of rich media in both creating and delivering insights to the right audience, and highlight organizations like Nike who are leading the way in taking a small, localized view of big data.  I look forward to hearing your thoughts and welcome recommendations on who I should spotlight in future posts.

thanks for stopping by!