Big data has never been bigger. On the heals of a new study (reported here) that shows that investors have pumped $3.6 billion into big data startups this year, and Gartner’s latest Hype Cycle which shows big data quickly approaching the top of the hype “cliff” (along with Consumer 3D Printing and Gamification), one is tempted to see more of the same in 2014. Yet all the steam coming out of the big data hype machine seems to be obscuring our view of the big picture: in many cases big data is overkill. And in most cases big data is useful only if we can do something with it in our everyday jobs.
Which is why against the backdrop of more hype (and more confusion) about the real role for big data, the case for thinking small looks more and more attractive. In fact, I recently penned an opinion piece for ZDNet in Paul Greenberg’s excellent column on Social CRM on 10 Reasons 2014 will be the Year of Small Data. You can read the full piece here, and here’s the Cliffs Notes version of the first 5 reasons:
- Big data is hard. Doing it at scale takes time, especially when the tech guys own it (I used to be a data scientist, so I can say this!).
- Small data is all around us. Social and mobile signals can help us understand customer needs – today.
- Small data powers the new CRM. Small data is central to the rich profiles essential to targeting offers and experiences.
- ROI is the thing. To realize the full value of data-driven apps they must be accessible, understandable, and actionable for everyday work (remember our definition).
- Data-driven marketing is the next wave. As more consumers want to access, and consume and even wear useful data, there’s an unprecedented opportunity for savvy marketers and platform providers like Adobe, HubSpot, and Salesforce, along with data specialists and tool providers like Actuate, Localytics, TrueLens, and Visible to power the next wave of apps, experiences and devices.
What do you think? Are you finalizing your big and small data strategy for 2014? If you’d like DCG’s perspectives as you formulate your plans I’ll be hosting a special Webinar on Dec 12 at 1:00pm ET to share our outlook and 2014 research agenda. Hope to see you there!
UPDATE: missed my Webinar? You can view the replay by clicking here.
One of the best parts of my research over the past several months as I worked on DCG’s “Bringing the Power of Big Data to the Masses” study (report now available here) has been getting to sample both current viewpoints as well as some of the literature and developments from the past 20+ years on the topic. In fact, big data and today’s analytical tools are very much a product of the “generations” of data and data processing that came before them.
Here’s a cool illustration we created for the report that highlights some of the key developments, deals, and publications related to big data, analytics, and small data since the invention of the web in 1989. We had to leave out a number (many) of potential items, so I’d love to hear what milestones would you have included – enjoy!
With a flurry of new articles on small data in various publications recently – including Wired, Technorati, and AllThingsD – there’s a growing number of voices contributing to the small data movement (a great thing!). But with these new perspectives, I feel like it’s a good time to re-ground the conversation in an actual definition of what we are talking about when we refer to “small data” – or at least what I’m talking about!
So in this post I wanted to share my first cut at a proper definition. Yes, I’ve framed the pillars for small data in many other places, going back to my first guest piece in Forbes on the topic a year ago (!), and more recently I’ve embraced the idea of describing small data as “the last mile of big data.” But these were descriptions or principles vs. definitions for the most part.
So, after spending the last couple month working on Digital Clarity Group‘s new multi-client study on “Bringing the Power of Big Data to the Masses” (sponsored by my friends at Adobe, Actuate, HubSpot, and Visible), seeing how marketers are looking to make analytics more accessible and actionable – and creating some new starter use cases, an interesting thing happened: a definition emerged!
In fact, while our final report is still a week or so away from being available from DCG and our sponsors, I shared the definition with our audience at the Digital Pulse Summit this past week during my panel on small data, and given the response, I wanted to provide it here as well.
A New Definition for Small Data
Small data connects people with timely, meaningful insights (derived from big data and/or “local” sources), organized and packaged – often visually – to be accessible, understandable, and actionable for everyday tasks.
Note, as we describe in our report, this definition applies to the data we have, as well as the end-user apps and analyst workbenches for turning big data set into actionable small data. The key “action” words here are connect, organize, and package, and the “value” (the 4th V of big data) is rooted in making insights available to all (accessible), easy to apply (understandable), and focused on the task at hand (actionable).
In fact, I hope it’s as much a mission statement, as it is a definition. What do you think? Did we nail it?
Reaching and engaging today’s social, mobile consumer creates unique challenges for retailers. With more than one billion Facebook users, and nearly everyone on the planet with a mobile phone, there’s never been more potential to reach new audiences, deliver compelling omni-channel experiences, and gather new insights on consumer likes and dislikes. Yet these same trends have also created new distractions, potential threats (and opportunities) from the phenomenon of “showrooming,” and a shortage of expertise for sifting through and make sense of a growing mountain of customer data.
Against this backdrop, the importance of connecting with buyers via tailored offers, recommendations, and experiences has never been more important. Even in B2B, among best-in-class content marketers, 71% tailor content to a profile of the decision maker, according to a recent CMI/MarketingProfs study. But at the same time, delivering convenience and bringing your brand and data to where your customers are, and factoring in the role of influencers, product reviews, and word of mouth needs to be in the mix as well. Clearly, in this environment, merchandising is both an art and a science!
This is why I’ve been seeing a tremendous opportunity for retailers who embrace their customer data as a both an asset and a product, and also invest in the notion of small data by tapping digital breadcrumbs, social signals, and profile information (as sources) and committing to making insights actionable and available to (the broadest set of) staff and customers alike. In fact it’s a theme that I’ve validated in several discussions with retail consultants, planners, and merchandising professionals over the past several months.
The end goal is greater understanding, better performing campaigns, and a more rewarding experience for your customers.
More specifically, as I recently discussed in a Digital Clarity Group Brief sponsored by my friends at SDL (you can download the paper as well as some excellent presentations here), it’s clear that:
- Retailers can boost their understanding of customers and better serve them (on their own terms) across all channels by looking to blend transactional processes and insights with social interactions and data.
- Innovators are focusing on making insights actionable via recommendations and predictive targeting, smart apps/kiosks for associates, and location-based offers.
- There are clear benefits to thinking small – targeting local campaigns and data, starting with high potential segments, and streamlining key stages of the path to purchase.
Most importantly, we can’t forget to ask customers what they think. Encourage social feedback and make it easy to share offers with friends. If you can’t measure it (conversions, sharing, etc.), don’t do it!
The bottom line as retailers look to position their products, create timely and compelling offers, and deliver on the promise of true omni-channel commerce, is to leverage all insights to streamline back-end processes and simplify front-end interactions. This means chunking down the path to purchase, and looking for ways to present the right product at the right time, offer help (if needed), and make it easy for consumers to finish their journey.
What do you think? Do you buy it?
As I prepared my presentation on small data for the recent DCG Insights day in New York, I did a quick review of some of the latest coverage of the topic, and discovered a number of new articles worth checking out. This list just scratches the surface, so I’ll add to it in subsequent posts, but wanted to share this first batch as a reading list for those interested in the topic. Enjoy!
5. Inc.com – Why Small Data May Be Bigger Than Big Data – in this piece which came out in April, x-McKinsey consultant and loyalty expert Victor Ho argues that small data is the key to solving the “data divide” for local businesses.
4. ITworld – Big Data Benefits with Small Data – tech journalist Brian Proffitt questions the necessity of big data for non government agencies or businesses that don’t have giga-scale ecommerce sites, and references the notion of personal data stores – a distinct small data use case. His closing point is a good one:
“Thanks to big data, many businesses recognize the value of data analysis. But there may be several new paths that will open up to help them achive the benefits of data decision making.”
3. MediaPost – Does Big Data Require a Big Rethink? – in this commentary Michael Hemsey of Kobie Marketing urges brands to focus on “the little details,” calls out his recommendations for thinking small when it comes to data assets, and talks about the age of mini-measurement – which is a cool way to put it!
“I don’t want to have a relationship with a marketing department. I don’t want to be your friend. I don’t want to engage in conversation with you. I feel no loyalty towards you. When I say I like you I’m not entirely sincere.
And yet I chose to share an enormous amount of my life with you. The detail I’m able and happy to share has grown big. Really, really big. But understand this: my reason for sharing this data is entirely motivated by self-interest. You see, I know as much about you as you do about me. I know how valuable my data can be to you. So I expect you to use this data for my benefit. Because you can be damn sure I will be.”
His conclusion is that small data is specific and concrete, which makes it easier to to “make good use of it.” And more so, requires us to understand our customers, their lives and where we fit within them. This is exactly the connection I’m looking to make between the worlds of CRM and small data, in the research I’m ramping up for Digital Clarity Group.
1. The Guardian (UK) – Forget Big Data, Small Data is the Real Revolution – one of the most widely referenced pieces of the last few months, Open Knowledge Foundation founder Rufus Pollock frames big data as the latest “centralization fad” and notes:
“the real opportunity is not big data, but small data. Not centralized “big iron”, but decentralized data wrangling. Not “one ring to rule them all” but “small pieces loosely joined.”
I love the point about decentralization, which ties nicely into the social aspect of small data. In fact Pollock ends with a great framing of small data as part of a larger movement driven by digital disruption and democratization of IT:
“This next decade belongs to distributed models not centralized ones, to collaboration not control, and to small data not big data.”
In terms of older articles, check out my guest post in Forbes from last October, and the early piece by Patrick Gray in TechRepublic that argues for a practical, consolidated approach to data and reporting.
What articles did I miss? What’s on your “must read” list?
Big Data is everywhere. Yet many companies lack a clear vision for rolling out big data in practical, measured steps. At the same time, with social networking, BYOD, and expectations from interactions with brands like Amazon, Apple, and Nike, most employees have seen the potential, yet many aren’t equipped to harness this power in their workplace. The small data movement — which I’ve been writing about for the past 6 months (and thinking about for many years) — aims to address these challenges and re-envision the “last mile” of big data via consumer-style, more responsive, more social apps that truly turn insight into action.
Not surprisingly, this idea is getting a LOT of attention. In fact, over the past 6 months there’s been as much published about small data as there was in the previous 3 years. Clearly small data has gone big time as a theme, concept, and set of tools.
So why is small data taking off?
- First, while understandably there’s a lot of excitement about big data, there’s a growing reality that doing it at scale and waiting for all the trickle down benefits can take a lot of time. Especially if you’re not in the C-suite.
- Second, (and here’s where I jump on my soapbox), the last mile of big data is really where the value is created, opinions are formed, insights are shared, and action are made. By non data scientists. Everyday.
- Third, there’s some great consumer examples of small data hitting the stage, most notably the much hyped but potentially game-changing Google Glass, which brings together small data and visualization with wearable computing, all in a semi-stylish package.
So who is fueling the small data movement? I wrote about 10 vendors who get the value of enabling simple, smart, responsive, socially aware tools and solutions in my last post. And a couple weeks back I wrote about 5 additional vendors and a new study I’m doing with Digital Clarity Group in a post on the DCG blog, which you can read here.
As small data takes off, I will continue to provide updates and thoughts both here and on the DCG blog. As always I’d love your thoughts and suggestions in the comments below.
Thanks for stopping by!
In the 4 months since I started to research my last piece on small data for Forbes.com, I’ve had a LOT of conversations about why it’s time to bring the power of big data to the masses, who is doing something about it, and where to focus first. I’ve taken a lot of notes, and I’ve also had the good fortune to run my emerging ‘small data manifesto’ by some real smart folks including Nobby Akiha, Bill Blundon, Jeff Boehm, Dorie Clark, Mike Gualtieri, Esteban Kolsky, Mitch Lieberman, Richard Pasewark, and my colleague Scott Liewehr (I’m sure I’m leaving out some others).
What have I learned? First off, the small data concept is resonating with a lot of the folks I’ve met. And looking at the volume of new posts and articles since the start of the year – in places like Forbes, Inc, Xconomy, and a number of marketing blogs – there seems to be a groundswell building that points to the value of thinking small.
Second, I’ve refined my ‘watch list’ of vendors that are powering this movement and ‘get’ the value of a creating/enabling simple, smart, responsive, socially aware tools and solutions. I’ve intentionally tried to focus on specialty tool providers vs the IBMs and SAPs. And I know that I’ve just scratched the surface – these are 10 vendors you’ll want to know, but not the only vendors in the space! So here’s my list, in alphabetical order, with a few comments on why they fit the bill.
10 Vendors Worth Watching
Actuate – One of the dashboard and reporting pioneers and founder of the Eclipse BIRT open source project, Actuate’s track record promoting ‘BI for the masses’ is well established (disclaimer: I actually helped with some of this back in the day). Actuate’s recent purchase of Quiterian gives the company a leg up over some of its peers when it comes to combining big data analytics and small data delivery.
Attivio – On the heals of a $34M investment, Attivio is poised for big things with its next-gen database that pulls together data from multiple sources and offers to bridge the worlds of big and small data. I love the focus on correlation and breaking down silos, and making it easy to see both the big and small picture.
GoodData – Driven by a $25M series C found in mid-2012, GoodData has become one of the leaders in bringing big data to life for all types of businesses. How? There’s a lot of small data thinking at work, as a quick tour of the company’s blog illustrates. One of the poster-children for why small data will be a big business.
Google – Starting with search, I’d argue that Google was the original small data company. Simple? Check. Smart? Oh yeah. Mobile? Yup. Social. Ah…getting there! With it’s purchase of Wildfire and improvements in Google+ and YouTube, plus resources second to none, Google could be to small (and big) data what Microsoft was to PCs. Seriously.
NetBase – I love how NetBase (former client) has created its Brand Passion Index to make its high-end analytics (using NLP and text analytics and other cool stuff) approachable and fun. Plus another $9M in funding this past January and a key partnership with SAP is bringing its tools to the wider enterprise market. Great strategy.
Nimble – one of the pure-play social CRM vendors founded by GoldMine CRM founder Jon Ferrara, Nimble is all about simple, smart apps and tools that users will want to use. It’s clear these guys understand that if you drive adoption by focusing on the end-user experience, ROI will follow (Saleforce gets this too!).
QlikTech – In terms of powering simple, mobile, contextual apps QlikTech is very much aligned with the small data vision, and one of the more complete offerings in the space. Also announced partnership with Attivio in January.
Tableau – Driven by the goals of powering fast analytics for ‘everyone’ and storytelling on the Web, Tableau’s positioning is lock-step with the vision and opportunity of using small data to bring big data to the masses. I also really dig the company’s messaging and overall creative. Nicely done.
Twitter – By nature of enforcing a small view of messaging and communication, Twitter should be in the small data hall of fame. But it’s really the company’s recent purchase of Bluefin Labs that moves these guys to the head of the class. Brilliant move.
Visible Technologies – Coming at small data from a social analytics perspective, Visible has a super-intuitive dashboard product, and a great handle on making data highly consumable. It’s clear the management team gets the small data potential, and for good measure the company was just named one of 9 Twitter Certified Products partners.
So who would you add to my watch list?
Large organizations are in love with big data and big analytics. In fact, back in March, IDC forecast that the big data technology and services market is expected to grow from $3.2 billion in 2010 to $16.9 billion in 2015, while it’s been reported that Deloitte estimates the size of the big data market at $1.3-$1.5 billion in 2012. Yet many employees aren’t equipped to harness these insights in their everyday decision making. And perhaps even worse, many executives and boards are still oriented towards an ‘inside’ data view vs customer or employee engagement type data that has increasing value as social and mobile networks take off, as Barry Libert argues.
Meanwhile, in the consumer world, brands like Amazon, Apple and Nike are shifting their own big data engines into high gear to deliver more personalized offers, recommendations and experiences that drive loyalty and sales.
What’s the connection? While companies (and computers!) like big data. Most people only need small data. Not only in terms of what is ‘good enough’ when it comes to the size of the data set. But also in terms of:
- Delivering simple, consumer-style, self-service apps and devices vs. complex toolkits,
- Providing context-driven, highly accurate answers and explanations (note I said answers, not data),
- Applying the latest Responsive Web Design and social marketing techniques to deliver a great experience across all Web-enabled devices and sharing channels.
Google may do this better than anyone currently when it comes to consumer applications. And on the business front, one can argue that Salesforce has done the best job to-date blending enterprise functionality with social insights and Web-style ease of use to create CRM solutions that your typical employee actually wants to use.
Of course other vendors, brands and investors can benefit from this small data approach, if they focus on creating simple, smart, responsive, socially aware tools and solutions.
Who is driving this small data revolution – even if they don’t know it yet? In terms of tools, my list would start with the new generation of speciality social analytics and business intelligence providers like Bluefin Labs, GoodData, NetBase, QlikTech, and Visible Technologies. In future posts I plan to explore these companies and other vendor categories, look at the role of rich media in both creating and delivering insights to the right audience, and highlight organizations like Nike who are leading the way in taking a small, localized view of big data. I look forward to hearing your thoughts and welcome recommendations on who I should spotlight in future posts.
thanks for stopping by!