Small Data Group

With all the hype and media coverage around ChatGPT and the tech behind it, it’s always good to revisit the foundations for big (and small) data applications and what AI technique works best in which applications. Especially for marketers looking for smart machines to help them spot trends, automate their daily work, or even grab the next headline. And even beyond the headlines, artificial intelligence (AI) – in all its forms – is truly all around us, in our smart speakers and appliances, in our cars and on our favorite eCommerce sites. AI has become ubiquitous at home, and in our work too. If you are not already, it’s time for marketers and brand strategists, their data friends, and even creatives to get on board with today’s AI.

But first a bit of history. AI is certainly not a new topic. In fact, scientists gathered for the Dartmouth Summer Research Project on AI in 1956, and release of the preliminary version of the AI programming language, Prolog, occurred in 1971. Heck, I was building neural net models in 1989. Through its history, AI has been interdisciplinary, with commercial advances funded by both public and private initiatives, and developments in all corners of the globe.

Also, its important for newcomers to know that artificial intelligence has been through boom and bust cycles over the past 60+ years. But one constant is the search for smart machines to solve a wide range of business and society’s challenges, and approaches being inspired by and mirroring the two general types of brain activity: top-down cognitive processes and lower-level or bottom-up perceptual processes.

The two branches of AI

For marketers, the best way to envision the potential of AI is thinking about your last online shopping experience. Odds are you encountered a virtual assistant powered by AI, saw tailored recommendations, and received personalized offers after making a purchase, raving about it on your social channels, or filling out a post-purchase survey (small data!). These functions are commonly powered by a collection of techniques from the two primary branches of AI: reasoning systems and learning systems.

At a high level, reasoning systems offer guidance and help to automate manual processes that teams conduct every day. They are a type of AI that is programmed or guided, follow a logical process, and map conditions to actions. They may operate autonomously (like a factory robot picking an order) or interactively with humans to gather needs, ask clarifying questions, etc. Examples include chatbots, shopping recommendation engines, and guided “helper” tools in everything from your CRM to your creative design tools.

In contrast, learning systems help teams spot patterns in a consumer or other data set, make predictions, or even suggest a design or answer to a question. Learning systems apply algorithms that are trained vs programmed, and can be supervised or unsupervised. They can be focused on two types of trend detection tasks: predicting something that has happened before like seasonal shopping behavior, or spotting an anomaly that hasn’t happened before, like a new consumer fashion trend. In general, a system that applies machine learning will “learn” from some combination of training data, prior knowledge, and trial and error reinforcement (see a helpful ML definition from Ipsos here).

Today, a popular and yet still often misunderstood flavor of learning is so called “deep learning.” These methods apply multi-layered neural networks and can be extremely powerful in high-complexity fields like machine vision and bioinformatics. This area was actually my focus in graduate school, applying models from Fukushima to image analytics.

At the same time, some of the most popular AI applications like natural language processing apply both reasoning and learning approaches together, to help marketers or CX teams better understand what their audience is saying online, track evolving consumer behavior and trends by what they are posting on Twitter or Reddit (or your feedback surveys!), and even identify opportunities for product innovation.

Hybrid AI is here

Most learning systems cannot explain their outcome without a human helper adding context or sharing details on how the system was trained, what performance it achieved, etc. Meanwhile, reasoning systems need to be programmed, and tested, and updated. As ChatGPT shows us, techniques like RLHF blur the lines as direct human feedback or examples are used during the training process, and once deployed, the output can appear remarkably human-like. Yet across all techniques, outputs still need to be checked by humans not only for accuracy, but also for biases, so they can be corrected or presented up front to the user.

For these reasons a growing number of practitioners and advisors like my friends at Forrester are promoting the benefits of a “hybrid” approach in domains like NLP. I have also written about the benefits of humans and machines working together in consumer intelligence applications, even when advanced AI or ML does the heavy lifting behind the scenes. This approach is especially valuable for mission critical tasks or consumer facing applications where there is both data to understand, and a structured process to follow.

So just as marketers need their data friends, machines need their human friends, especially when translating big data-derived insights into small, consumable bits of advice or actions.

Contact me to learn more about emerging ML and hybrid AI use cases, or to share how you are envisioning everyday AI in your organization.