Yes, Machines Make Mistakes: The 10 Biggest Flaws In Generative AI

We’re definitely in the middle of an "AI summer," a period where both scientists and the general public get very excited about the possibilities of computer learning. Generative AI models such as ChatGPT and Midjourney are allowing more people than ever before to try this powerful type of tool. But that exposure is also revealing deep flaws in how AI programs are written and trained on data, and that could have major repercussions for the industry.

Here are our picks for the ten biggest flaws in current generative AI models.

1. AI Is Too Eager to Please

If we were to view AI algorithms as though they were living beings, they’re kind of like dogs—they really want to make you happy, even if that means leaving a dead raccoon on the front porch. Generative AI needs to create a response to your query, even if it isn’t capable of giving you one that is factual or sensible. We’ve seen this in examples from ChatGPT, Bard, and others: If the AI doesn’t have enough actual information in its knowledge base, it fills in gaps with stuff that sounds like it could be correct, according to its algorithm. That’s why when you ask ChatGPT about me, it correctly says I write for PCMag, but it also says that I wrote Savage Sword Of Conan in the 1970s. I wish!

2. AI Is Out of Date

Another significant issue is with the datasets these tools are trained on. They have a cutoff date. Generative AI models are fed massive amounts of data, and they use it to assemble their responses. But the world is constantly changing, and it doesn’t take long for the training data to become obsolete. Updating AI is a massive process that has to be done from scratch each time because the way data is interconnected in the source means that adding and weighting additional information isn’t possible to do piecemeal. And the longer the data goes without updates, the less accurate it becomes. 

3. AI Commits Copyright Infringement

Plagiarism is a very real problem in the creative arts, but the output of a generative AI model really can’t be defined in any other way. Computers aren’t capable of what we would consider original thought—they just recombine existing data in a variety of ways. That output might be novel and interesting, but it isn’t unique. We’re already seeing lawsuits in which artists quite rationally complain that training a visual generation model on their copyrighted works and using it to create new images in their style is an unlicensed use of their art. This is a huge legal black box that will influence how AI is trained and deployed in unpredictable ways.

4. AI Learns From Biased Datasets

Implicit bias has been a huge problem with machine learning for decades. There was a famous case a few years back when Hewlett-Packard cameras struggled to identify Black people’s faces but had no problem with lighter-skinned users, because the training and testing of the software were not as diverse as they should have been. The same thing can happen with massive AI data sets—the information AI is trained on can bias the output. As more decisions are made based on AI computation as opposed to human review, bias opens the possibility for massive structural discrimination.

5. AI's Black Box Obscurity

There’s a great anecdote about Google’s search algorithm in Max Fisher’s book The Chaos Machine: The Inside Story of How Social Media Rewired Our Minds and Our World, in which a company insider comments that results are served through so many layers of machine-learning algorithms that a human being can no longer go into the code and trace exactly why the software made the choices it did. That kind of complexity and obscurity can create significant problems with generative AI. An inability to identify the source of inappropriate responses makes these systems extremely hard to debug and refine along any metrics besides the ones the software is trying to serve.

6. AI Is Shallow

Machines are brilliant at sifting through huge amounts of data and finding things in common. But making them delve deeper into content and context almost always fails. A great example is the slick-looking digital art created by tools such as Midjourney. Its creations look amazing on the surface, every brush stroke placed perfectly. But when AIs try to replicate a complex physical object—say, the human hand—they’re not capable of grappling with the intrinsic structure of the object, instead making a guess and giving their portraits seven-fingered penguin flippers more often than not. Not being able to “understand” that a human hand has four fingers and a thumb is a massive gap in how these intelligences “think.”

7. AI Impersonates Real People

While some generative AI models have safeguards to prevent them from impersonating living people, many do not—and the technology is extremely easy to jailbreak. TikTok is full of AI-voiced conversations, say, between Donald Trump and Joe Biden about smoking weed and cheating in Minecraft, and they’re pretty believable on first listen. It’s only a matter of time before a computer-generated simulation of a public figure gets that person canceled, and the victim is rich enough to pursue action against the perpetrator. 

Recommended by Our Editors

What Is Artificial Intelligence (AI)?

When Will ChatGPT Replace Search? Maybe Sooner Than You Think

Man Boasts About Winning Art Competition Using AI-Created Image

8. AI Can Lie

A generative AI model cannot tell you whether something is factual; it can pull data only from what it’s been fed. So if that data says that the sky is green, the AI will give you back stories that take place under a lime-colored sky. When ChatGPT prepares output for you, it doesn't fact-check or second-guess itself. And while you can correct it during your session, those corrections aren’t fed back to the algorithm. This software is comfortable lying and making things up because it has no way not to, and that makes relying on it for research especially risky.

9. AI Isn't Accountable

Who is responsible for the work created by generative artificial intelligence? Is it the person who wrote the algorithms? The people who created the data sources it learned from? The user who gave it the prompt to respond to or the instructions to follow? That’s not really settled law right now, and it could pose huge problems in the future. If a generative AI model provides an output that leads to legally actionable consequences, who is going to be blamed for them? Building a code of legal ethics around AI accountability will be a massive challenge for companies looking to monetize the technology.

10. AI Is Expensive

Creating and training generative Ai models is no small feat, and the cost of doing business is astronomical. Analysts estimate that training a model such as GPT-3 could run up to 4 million dollars. These AI models require massive hardware outlays, often thousands of GPUs running in parallel, to chew through and link their data sets. And as mentioned earlier, that process has to be done every time you update the model. Moore’s Law will eventually downsize this problem, but in the present day, the financial cost of making one of these things can be more than most companies will be able to justify.

This AI robot was made to be your best friend