August 16, 2017
From a kindergartener getting a participation ribbon at his swim meet to a Nobel Prize winner having her life’s work acknowledged, everybody likes a little recognition now and again. And companies are no different. We like hearing from impartial analysts that we’re on the right track, that we’re solving real problems in the world, and that our work is vital.
Recently, we were lucky enough to receive just such an acknowledgment. Last month, all of us here at CrowdFlower were thrilled by a spate of recent activity from Gartner, a leading technology research firm that’s consistently been ahead of the curve. Not only were we named a Gartner Cool Vendor, but Gartner, for the first time ever, acknowledged Human-in-the-Loop as a part of the AI landscape. In fact, not only was “Human-in-the-Loop Crowdsourcing” finally included in Gartner’s AI hype cycle, it was noted as a transformational technology, a category reserved for tech that can legitimately change the world.
If you’re unfamiliar with Gartner, here’s a quick introduction and some bona fides. Gartner is a group of analysts and researchers that specialize in emerging technologies and the IT space. They predicted the dotcom bubble burst, the widespread embrace of cloud computing, and get cited hundreds of times a month in the mainstream business press. In addition to their Magic Quadrants, Gartner’s probably most famous for their Hype Cycle publications- research where the company maps out the relevance and adoption cycle of technologies within broader categories.
Of course, marketers love getting Gartner’s nod. We can use them to confirm our market size, rile up our sales team, ground our funding pitches, and kill a couple trees printing all kinds of collateral. And while I’d be happy to brag about our Cool Vendor award, let’s get back to the hype cycle. Specifically, I’d like to highlight the AI hype cycle and where Human-in-the-Loop fits in.
First off, there’s this pervasive sense across myriad industries that AI and machine learning will change business in the same way computing and the internet did decades ago. In the wake of large companies gone by the wayside in the dust of digital, many companies today realize that adopting an AI strategy isn’t about ‘getting ahead’ but about ‘staying in the game.’ And while I’ve lived and marketed through many transformational technology cycles–early internet, APIs, and cloud computing–I’ve never seen any technology adopted with this kind of urgency.
But the dirty little secret about AI is this: behind the scenes, there’s a ton of heavy lifting required to train, tune, and test the models. Behind the curtain, there are humans labeling, categorizing and cleaning troves of data that feed into machine learning models to help them learn. Admittedly, this isn’t the sexy part about AI. It’s not the newest algorithm, chock full of Greek symbols or the flashy implementation that wins a complex board game. Those get great press, but it’s actually the training data that gives these complex models the wings and wherewithal to achieve great things. Without training data, machine learning models are just bundles of unrealized potential. High-quality data is the thing that keeps diseases from being misdiagnosed in an AI doctor, enables a self-driving car to distinguish a lamp post from a pedestrian, or prevents a security robot from rolling into a fountain.
This is why we’ve been touting the term Human-in-the-Loop” for years. And even though that description was sometimes met with quizzical looks, we stuck with it. Because we believe in it. We’re passionate about it. And Gartner’s inclusion of Human-in-the-Loop on the AI hype cycle validates what we’ve been saying all along: algorithms fail without people to power, teach, and validate them. They need training data to succeed.
Within this year’s Hype Cycles, the “Human-in-the-Loop Crowdsourcing” category is depicted at the beginning of the cycle as an innovation trigger with an estimate of 10 years til full market maturity. Here’s where I’d like to respectfully push back a bit. See, there’s a logical loophole. Because the human labeling and structuring of training data is the first step in creating a successful machine learning model. Therefore, we predict that Human-in-the-Loop will ride the Gartner wave at a more rapid pace than predicted. After all, you can’t bake the cake if you haven’t bought the eggs. Check back in next year and we’ll see if I’m right there.
As companies continue to innovate, developing new avenues for machine learning and AI, streamlining processes, even creating new industries that could never exist without artificial intelligence in the first place, every one of them will need the best data they can get their hands on. For us, this is a truly exciting time to be a Human-in-the-Loop pioneer. We get to see these projects grow and evolve. We get to help them succeed. And while most folks will focus on powerful new models and shiny new implementations, remember what’s behind the whole enterprise: the best data possible. And it’s nice to see that recognized.