Stories of AI Failure and How to Avoid Similar AI Fails in 2019

  7 m, 13 s

Starting a new AI project in 2019? Don’t fail prey to the hype. The new year will see a separation of the wheat from the chaff, and an AI fail at this stage could spell disaster for your project, team, and company.

This article presents information and advice to help you build an effective AI solution:

  • Cautionary stories of AI failure from Microsoft, Apple, and others
  • “9 more ways to fail with AI” from the Chief Data Officer at Abe.ai
  • Some tips on AI maintenance from a veteran data scientist
  • A CEO’s guide to gaining real value from AI

Full disclosure if you’re new to Lexalytics: we provide a data analytics platform that uses AI and machine learning for natural language processing. But the stories and the advice presented here are relevant for anyone involved in AI/machine learning.

Fail: Microsoft’s AI Chatbot Corrupted by Twitter Trolls

Microsoft made big headlines in 2016 when they announced their new chatbot. Writing with the slang-laden voice of a teenager, Tay could automatically reply to people and engage in “casual and playful conversation” on Twitter.

Some of Tay’s early tweets, pulled from this Verge article:

Tay grew from Microsoft’s efforts to improve their “conversational understanding”. To that end, Tay used machine learning and AI. As more people talked with Tay, Microsoft claimed, the chatbot would learn how to write more naturally and hold better conversations.

Microsoft won’t say exactly how the algorithms worked, of course. Perhaps because of what happened next.

Less than 24 hours after Tay launched, internet Trolls had thoroughly “corrupted” the chatbot’s personality.

By flooding the bot with a deluge of racist, misogynistic, and anti-semitic tweets, Twitter users turned Tay – a chatbot that the Verge described as “a robot parrot with an internet connection” – into a mouthpiece for a terrifying ideology.

Microsoft claimed that their training process for Tay included “relevant public data” that had been cleaned and filtered. But clearly they hadn’t planned for failure, at least not this kind of catastrophe.

After a cursory effort to clean up Tay’s timeline, Microsoft pulled the plug on their unfortunate AI chatbot.

Fail: Apple’s Face ID Defeated by a 3D Mask

Apple released the iPhone X (10? Ten? Eks?) in early November 2017 to mixed, but generally positive reviews. The phone’s shiniest new feature was Face ID, a facial recognition system that replaced the fingerprint reader as your primary passcode.

Apple said that Face ID used the the iPhone X’s advanced front-facing camera and machine learning to create a 3-dimensional map of your face. The machine learning/AI component helped the system adapt to cosmetic changes (such as putting on make-up, donning a pair of glasses, or wrapping a scarf around your neck), without compromising on security.

But a week after the iPhone X’s launch, hackers were already claiming to beat Face ID using 3D printed masks. Vietnam-based security firm Bkav found that they could successfully unlock a Face ID-equipped iPhone by glueing 2D “eyes” to a 3D mask. The mask, made of stone powder, cost around $200. The eyes were simple, printed infrared images.

Bkav’s claims, outlined in a blog post, gained widespread attention, not least because Apple had already written that Face ID was designed to protect against “spoofing by masks or other techniques” using “sophisticated anti-spoofing neural networks”.

Not everyone was convinced by Bkav’s work. Publications such as Wired had already tried and failed to beat Face ID using masks. And Wired’s own article on Bkav’s announcement included some skepticism from Marc Rogers, a researcher for security firm Cloudflare. But the work – and this glimpse into the weakness of AI – is fascinating.

[5 stories of AI failure in 2017.png]5 More AI Fails From 2017

Microsoft and Apple aren’t the only companies who’ve made headlines with embarrassing AI fails. In this feature, Srishti Deoras summarizes the “top 5 AI failures from 2017“.

In one story, Facebook had to shut down their “Bob” and “Alice” chatbots after the computers started talking to each other in their own language. And that’s just the beginning. Srishti continues with more examples from Mitra, Uber and Amazon.

Together, these 5 AI failures cover: chatbots, political gaffs, autonomous driving accidents, facial recognition mixups, and angry neighbors.

Srishti argues that these failures suggest companies should be more cautious and diligent when implementing AI systems.

[9 ways to fail with AI.png]9 More Ways to Guarantee an AI Fail

Writing on Medium, Francesco Gadaleta, Chief Data Officer at Abe.ai, explores 9 more “creative ways to make your AI startup fail“.

Francesco’s list is comprehensive, funny, and thought-provoking. It features some classic paths to failure, such as “Cut R&D to save money” and “Work without a clear vision”. But, Francesco says, “there is a plethora of ways to fail with AI”.

My favorite is #2, “Operate in a technology bubble.”

As Francesco points out, AI doesn’t always fail due to technical problems. Sometimes, the problem is a lack of social need or interest.

“Artificial intelligence technologies cannot be built in isolation from the social circumstances that make them necessary,” Francesco writes.

This is a fantastic point. In the rush to stay ahead of the technology curve, companies often fail to consider the impact of their inherent biases. This is particularly dangerous for companies working in data analytics for healthcare, biotechnology, financial services and law.

Just look at Watson for Oncology: data bias and lack of social context doomed that AI project to failure and sent $65 million down the drain.

“Operating in a bubble and ignoring the current needs of society is a sure path to failure.” – Francesco Gadaleta

Francesco’s list is a must-read for any executive, developer or data scientist looking to add AI to their technology stack

[ai requires maintenance.png]Why Maintenance is Critical to Avoiding an Embarrassing AI Failure

Plan for failure; work on your reaction times; adopt a change management model. Manifesto of a management consulting firm? No, it’s veteran data scientist Paul Barba writing for KDnuggets.

Just like a car, Paul explains, an AI can tick along for a while on its own. But failing to maintain it can destroy your project or product, and maybe even your company.

As cars become more complex, insurance companies advise owners to keep up with preventative maintenance before the cost of repairs becomes staggering. Similarly, as an AI grows more complex, the risks and costs of AI failure grow larger. And the longer you wait to repair your AI, the more expensive it’ll be.

“Through auditing, quantitative measuring and proactive organizational responsiveness, you can avoid the equivalent of blowing an AI gasket.” – Paul Barba

Just like your car, an AI requires maintenance to remain robust and valuable. And just like your car, you may be faced with a sudden, catastrophic failure if you don’t keep it up-to-date.

In this article, Paul explains how data scientists can avoid AI failure by maintaining it with new training data, methods and models.

How to Get Real Value from Artificial Intelligence in 2019

Big AI projects, such as Watson for Oncology and self-driving cars, get most of the press coverage. But as the past few years have shown, moon-shots like these are the most likely to fail. And when they fail, they fail spectacularly (as we’ve been discussing).

Related article: How to Choose an AI Vendor

How, then, can you build an AI system that actually succeeds? The answer is deceptively simple:

Focus on solving a real business problem.

Our own CEO, Jeff Catlin, has spent the past 15 years watching AI and machine learning get over-hyped and under-delivered. In this article on Forbes, he examines a number of business applications for AI solutions to:

  • Predict customer churn
  • Create better surveys
  • Read and handle online reviews
  • Craft effective messaging

“Building a business case for AI isn’t so different from building one for any other business problem,” Catlin writes. “First, identify a need and a desired outcome (automation and efficiency are common drivers of successful AI projects). Then undertake a feasibility assessment.”

The key is to look for business use cases where AI is already in action, or where it’s emerging as an effective solution.

Jeff puts it best: “With the right business case and the right data, AI can deliver powerful time and cost savings, as well as valuable insights you can use to improve your business.”

Read Jeff’s article on Forbes: Using AI to Solve a Business Problem

Further Reading on AI Best Practices and AI Applications

When is “Good” Good Enough for AI?

Artificial Intelligence for Disaster Relief

3 Surprising AI Applications in Food, Energy & Airlines

Breaking Free from Unrealistic AI Expectations

Categories: Artificial Intelligence, Insights, Newsletter