At the beginning of the year it looked like 2014 was shaping up to be a year when Small Data moved out of the shadows and into the mainstream (or maybe even THE Year of Small Data as I predicted in this op-ed for ZDNet). Judging by the volume of articles, social chatter, and casual conversation I’ve had at numerous industry events, Small Data has arrived. But it still means different things to different people. And that’s cool, since there’s a lot we’ve learned over the past couple years about why it’s important – not only as an alternative to Big Data – but also as a design philosophy and movement that shifts the conversation from processing power…to people!
So if you’re new to the topic, or just curious how Small Data has grown up, I’ve collected my “Top 10” articles for sampling the early concepts as well as the latest thinking. There are many more pieces I’ve left off, but this might serve as a core reading list. And yes, I’ve included just 2 of my own articles, but if you like them you can find many more on this blog or on the Actuate buzz page.
The Early Days – Foundations
Forget Big Data, Small Data is the Real Revolution – Rufus Pollock’s (@rufuspollock) original post on the power of Small Data, collaboration and “decentralized data wrangling” from April 2013. Consistently one of the top 5 most cited pieces on the topic, it’s a great companion to my Forbes piece below.
These Smart, Social Apps Bring Big Data Down to Size – my piece with Mark Fidelman (@markfidelman) from October 2012 that introduced the pillars of small data design: make it simple, be smart, think mobile (be responsive) and predicted a new wave of user-centric business intelligence.
What happens when each patient becomes their own “universe” of unique medical data? – Prof. Deborah Estrin (of @cornell_tech) talk at TEDMed 2013 has key ideas around managing our own personal Small Data. Especially relevant to not only privacy discussions but also those pondering the next wave of wearables and fitness tracking devices.
Data-driven Marketing – 3 Perspectives
Small Data Can Help Businesses Be More Human – great post on the need for “human scale” in marketing by Brand Networks CEO Jamie Tedford (@jamietedford).
What the “Small Data” Revolution Means for Marketers – it’s all about delivering more targeted, more personalized and all-around more intelligent marketing campaigns says CommandIQ (@CommandIQ) CEO Noah Jessop. Right on!
What’s The Big Deal About ‘Small Data’? – recent roundup of perspectives on the marketing front (yup, including my own) by my friends at CMO.com.
Is Small the New Big? – Mainstream Thinking and New Voices
Focus on data value, not its ‘bigness’ – one of my recent op-eds on data value, decentralization and the new CRM. Good overview piece if you want to get a high level perspective and stir up thinking within your own organization.
In Praise of ‘Small Data’: How Targeted Analytics — Not ‘Big Data’ — Are Transforming Education Today – new post by Brian Kibby (@BrianKibby), President of McGraw-Hill’s Higher Education Group.
Your Sushi May Be Getting Smarter – new piece in The Atlantic by Tech Editor Adrienne LaFrance (@AdrienneLaF) on “smart objects,” food safety and the power of Small Data (and IoT).
How To Create Incredible Customer Service Through The ‘Small Data’ Advantage – new piece by customer service speaker and author Micah Solomon (@micahsolomon). While a lot of my focus to-date has been on marketing use cases, I think there are some great ones in customer service/loyalty as well as this post suggests.
What articles did I miss? What would you add if I made it a “top 20” list instead?
In a thought provoking interview with CNET published this past week, Fitbit designer Gadi Amit explores the use of wearables in everyday applications – and introduces the notion of “wise” devices that provide just the right information, when and where we need it.
Beyond the fact that Amit’s firm is designing wearables for unique markets like babies (well I guess really for their parents) and pets (!), what struck me in this piece was Amit’s perspective – certainly shaped by his role as president and lead designer at design firm NewDealDesign – on the state of wearables, their future, and our relationship with them. Specifically, in response to a question about how wearables will be integrated into our daily lives, he states:
The interesting thing is when I say that, people immediately jump to the conclusion that we will be cyborgs. My goal with designing this is that we won’t be cyborgs. We actually will become more human and more free from the technology. What we have now in the design business is two camps: there is the camp that wants to create a lot of data and wants to analyse a lot of data; and there is the other camp which I belong to that tries to create devices that are not smart, they are actually wise. They are more than smart, they are wise enough to understand you, to filter and allow you to go on with your life with all their data processing in the background giving you hints of what is essential when it is essential.
Having data processing in the background and focusing on what information is essential is of course very much in line with the small data “aesthetic” we’ve been promoting here and in a number of venues over the past 2 years, so it’s cool to hear validation from another corner. As a former AI/machine learning guy, I also like the idea of “wise” devices that understand context and personal preferences, and can make a case that small data will in fact be the new “OS” for these devices (more in a future post).
But even more so, if we think of the cyborg comment as a challenge to all of us, I think we need to consider the element of “humanness” as we create new apps and digital experiences. And perhaps provide better opportunities and incentives to untether/unplug (partially?) from our digital devices, even as consumers clamor for faster, more personal, more portable, and ultimately more satisfying data-enriched experiences.
Designing Data-driven Apps
Speaking of the new data consumer, I’ve been spending more time with developers and those thinking about the future of customer facing apps, and recently created a talk on design principles that builds on some of the work you’ve read about on this very blog. As always I believe that data-driven design is an art and a science, so it’s been fun to brush up on the science/tech part for sure.
Of course our first job is still to think about the end-consumer, and how we can inform, connect, and motivate them to get involved or take action. As an aside, if you’ve paid attention to how I’ve presented this last point, I’ve always used Nike Fuelband as my example, so with news that Nike is getting out of the fitness hardware business (good analysis in this Gigaom piece), it’s been interesting to see Fitbit and even Samsung step up their efforts ahead of the likely fall iWatch debut.
On the business side, beyond understanding the value of data along the customer journey and focusing on “last mile” functionality, having a scalable foundation that can potentially support millions of users and large data sets from many sources (before it is transformed into useful small data) is essential as we look to bring powerful, yet human-scale, smart (wise) apps to the masses. So is a community to drive innovation – like the 3.5 million BIRT developers, or 600K+ Drupal users and coders.
Many of these ideas (and some examples) were covered in the talk I did with SD Times recently. There’s a link to the replay and a summary by my colleague Fred Sandsmark on the Actuate blog – which you can read here.
I also presented a longer version focused on bringing the power of advanced analytics to “everyday tasks” at the CAMP IT big data event this past week, (a well-produced event by the way) and plan to post those slides to my slideshare shortly.
Finally, I will be moderating a very cool expert panel on “building the next big app” at a special event Actuate is hosting in San Jose on the evening of July 10. Scheduled to join me on stage will be Eclipse Foundation Executive Director Mike Milinkovich, plus industry watcher and enterprise apps futurist Esteban Kolsky, along with 1-2 other special guests. We’ll explore how consumer experiences will (and are) be shaped by new devices and data, open source driven innovation, and next-generation design tools and practices.
Be sure to let me know if you’ll be in the area and want to join us, since I have a limited number of VIP passes to share.