Arijit Sengupta once wrote an entire book titled Why A.I. is A Waste of Money. That’s a counterintuitive title for a guy who makes his money selling A.I. software to big companies. But Sengupta didn’t mean it ironically. He knows firsthand that for too many companies, A.I. doesn’t deliver the financial returns company officials expect. That’s borne out in a slew of recent surveys, where business leaders have put the failure rate of A.I. projects at between 83% and 92%. “As an industry, we’re worse than gambling in terms of producing financial returns,” Sengupta says. Sengupta has a background in computer science but he also has an MBA. He founded BeyondCore, a data analytics software company that Salesforce acquired in 2016 for a reported $110 million. Now he’s started Aible, a San Francisco-based company that provides software that makes it easier for companies to run A.I. algorithms on their data and build A.I. systems that deliver business value.
Aible makes an unusual pledge in the A.I. industry: it promises customers will see positive business impact in 30 days, or they don’t have to pay. Their website is chock full of case studies. The key, Sengupta says, is figuring out what data the company has available and what it can do easily with that data. “If you just say what do you want, people ask for the flying car from Back to the Future,” he says. “We explore the data and tell them what is realistic and what options they have.” One reason most A.I. projects fail, as Sengupta sees it, is that data scientists and machine learning engineers are taught to look at “model performance” (how well does a given algorithm do with a given data set at making a prediction) instead of business performance (how much money, in either additional revenue or cost-savings, can applying A.I. to a given dataset generate). To illustrate this point, Aible has run a challenge in conjunction with UC Berkeley: it pits university-level data science students against high school 10th graders using a real-world data set comprised of 56,000 anonymized patients from a major hospital. The competing teams must find the algorithm for discharging patients from the 400-bed hospital that will make the hospital the most money, understanding that keeping patients in the hospital unnecessarily adds costs, but so does making a mistake that sees the same patient later readmitted. The winner gets $5,000. The data scientists can use any data science software tools they want, while the high school kids use Aible’s software. The high school kids have beaten the data scientists—by a mile—every time they’ve run the competition, Sengupta says.
The teens, Sengupta says, are able to keep their eyes on the bottom line. They’re not concerned with the particular model that Aible suggests (Aible works by training hundreds of different models and finding the one that works best for a given business goal), whereas the data scientists get caught up on training fancy algorithms and maximizing accurate discharge predictions, but losing sight of dollars and cents.
Sengupta’s point is that ignoring, or not actually understanding, the business use of an A.I. system can be downright dangerous. He describes what he calls the “A.I. death spiral,” where an A.I. system maximizes the wrong outcome and literally runs a business into the ground. Take for example an A.I. system designed to predict which sales prospects are most likely to convert to paying customers. The system can achieve a higher accuracy score by being conservative—only identifying prospects that are highly likely to convert. But that shrinks the pool of possible customers significantly. If you keep running this optimization process using only the small number of customers who convert, the pool will just keep shrinking, until eventually the business winds up with too few customers to sustain itself. Customer win rate, Sengupta says, is the wrong metric—the A.I. should be trained to optimize revenue or profits, or maybe overall customer growth, not conversion rates.
Sidestepping these pitfalls requires a little bit of machine learning understanding, but a lot of business understanding. Sengupta is not alone in hammering home this theme. It’s a point that a lot of those working on A.I. in commercial settings—including deep learning pioneers such as Andrew Ng—are increasingly making: algorithms and computing power are, for the most part, becoming commodities. In most of the case studies on Aible’s website, customers used the startup’s cloud-based software to train hundreds of different models, sometimes in less than 10 minutes of computing time. Then the business picks the model that works best.
What differentiates businesses in their use of A.I. is what data they have, how they curate it, and exactly what they ask the A.I. system to do. “Building models is becoming a commodity,” Sengupta says. “But extracting value from the model is not trivial, that’s not a commodity.” With that, here’s the rest of this week’s A.I. news.
Jeremy Kahn @jeremyakahn jeremy.kahn@fortune.com
Our mission to make business better is fueled by readers like you. To enjoy unlimited access to our journalism, subscribe today.
© 2022 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell My Personal Information | Ad Choices FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice. S&P Index data is the property of Chicago Mercantile Exchange Inc. and its licensors. All rights reserved. Terms & Conditions. Powered and implemented by Interactive Data Managed Solutions.