Defining Small Data
With a flurry of new articles on small data in various publications recently – including Wired, Technorati, and AllThingsD – there’s a growing number of voices contributing to the small data movement (a great thing!). But with these new perspectives, I feel like it’s a good time to re-ground the conversation in an actual definition of what we are talking about when we refer to “small data” – or at least what I’m talking about!
So in this post I wanted to share my first cut at a proper definition. Yes, I’ve framed the pillars for small data in many other places, going back to my first guest piece in Forbes on the topic a year ago (!), and more recently I’ve embraced the idea of describing small data as “the last mile of big data.” But these were descriptions or principles vs. definitions for the most part.
So, after spending the last couple month working on Digital Clarity Group‘s new multi-client study on “Bringing the Power of Big Data to the Masses” (sponsored by my friends at Adobe, Actuate, HubSpot, and Visible), seeing how marketers are looking to make analytics more accessible and actionable – and creating some new starter use cases, an interesting thing happened: a definition emerged!
In fact, while our final report is still a week or so away from being available from DCG and our sponsors, I shared the definition with our audience at the Digital Pulse Summit this past week during my panel on small data, and given the response, I wanted to provide it here as well.
A New Definition for Small Data
Small data connects people with timely, meaningful insights (derived from big data and/or “local” sources), organized and packaged – often visually – to be accessible, understandable, and actionable for everyday tasks.
Note, as we describe in our report, this definition applies to the data we have, as well as the end-user apps and analyst workbenches for turning big data set into actionable small data. The key “action” words here are connect, organize, and package, and the “value” (the 4th V of big data) is rooted in making insights available to all (accessible), easy to apply (understandable), and focused on the task at hand (actionable).
In fact, I hope it’s as much a mission statement, as it is a definition. What do you think? Did we nail it?
Small Data Group 
Love the focus on “small data” vs. the hype of “Big Data” Seems like we are constantly bombarded with big data…. just some thoughts about how to wrestle the beast so to speak: http://analytics-ideas.com/2013/10/20/analytics-is-a-way-of-life/
Hi Brendan – thanks for the note and sharing your link. Will definitely check it out!
Allen
Why must small data be derived from derived from big data? I suppose the ‘and/or local sources’ is something of a catch all here. I’d suggest removing “(derived from big data and/or “local” sources)” from the definition. My two pennies.
Agreed with Kevin. Small data is what makes up big data. The ability to drill down to actionable items (to the trees, leaves, and fruits) from the bigger landscape (the forest) has been a capability that is achieved with integration of Analytical and Transactional systems. So, what’s new again?
Have you ever considered publishing an ebook or guest authoring on other websites?
I have a blog based on the same ideas you discuss and would really
like to have you share some stories/information. I know my audience would enjoy your work.
If you are even remotely interested, feel free
to send me an email.
So should we call it “Reduced Data” instead of “Small Data”? It’s less appealing but may be more true. If you “prune” Big Data to make data actionnable, accessible and so forth, it’s more a top down approach : a reduction. I don’t think it’s small data at first.
What about all these databases created first in Excel by Business Experts, that finally (and not in every cases) move to the IT department to be transformed as new app/dashboard/workflows (with or without predictive functions)? In this case, Small Data will be more concerned about what forms the long tail of software (cf. https://www.gv.com/lib/the_long_tail_o). Am I out of focus?
You forgot to mention about small data internal to the enterprise. For example in my experience we have lots of excels and emails which we use to communicate our daily activities, but just communicating this small data is done with meetings that is interruptive to our overall productivity. I really wanted to find an agile solution that could map our company strategies, personal goals and relate them to team project information giving everyone transparent visibility into all daily operations. A startup called Chronality (https://www.chronality.com) offered us a platform that gave us one centralized solution for all the ways we needed to map our operational small data as a heirarchy(just like big data) within our functional area accessible for daily decsions.