ChatGPT shows how far AI has come and its acute limitations — like being right

TECHNOLOGY LENS

The most advanced chatbot out there — with room for improvement

The breadth of material ChatGPT can create is what has seemingly captivated the public’s attention. ChatGPT can learn how to write a 10-paragraph history of London in the style of Dr. Seuss or create a new “Seinfeld” scene in which Jerry needs to learn a bubble sort algorithm. (For those wondering, that’s a simple sorting algorithm.) If that last one doesn’t sound funny enough, you can even tell ChatGPT to “make it funnier.”

It’s part of Open AI’s GPT-3.5 series, which is a large language model. Large language models ingest copious amounts of data from the web — generally by searching and indexing the internet in general or specific websites.

It relies on a concept known as reinforcement learning from human feedback, a sort of machine-human dialogue. The person prompts or queries the machine, which responds and adapts as the person makes additional prompts or queries.

Unlike some earlier chatbots, ChatGPT explicitly rejects inappropriate phrases or questions, such as those relating to antisemitism and the creation of violent content. That being said, some users have been able to get around this by posing queries as hypotheticals.

Open AI is upfront about ChatGPT’s limitations though, noting that “ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers.” So while the material might come out sounding original and authentic, there is a good chance it may also be wrong. ChatGPT failed to solve certain math problems, for example, and when asked to create a biography of a historical figure, it got some dates incorrect.

These sorts of kinks are expected, and the mass testing is a reason that Open AI releases these kinds of systems into the wild, so that it can patch and improve them in future releases.

— Benjamin Powers

ETHICS LENS

The potential for misinformation

While ChatGPT has better built-in functions to help it avoid offensive responses and spreading misinformation, it’s no saint-bot.

Open AI uses a content moderation tool, the Moderation endpoint, to help protect against possible misuse, but the chatbot will sometimes still respond to “harmful instructions or exhibit biased behavior,” the company says on its website.

The application has the potential to be a significant purveyor of misinformation, so much so that ChatGPT itself has explicitly said that it “can be used for nefarious purposes, like spreading misinformation or impersonating someone online,” Slate reported.

And in a blog post, Open AI has also admitted that ChatGPT is susceptible to providing “plausible-sounding but incorrect or nonsensical answers.”

Gary Marcus, founder of Geometric Intelligence and author of “Rebooting AI,” told Grid he is concerned “that bad actors will use [ChatGPT] to generate misinformation at scale.”

“The cost of misinformation is basically going to zero,” Marcus explained, “and that means that the volume of it is going to go up.”

He added that people might already be using programs like ChatGPT to bolster search engine optimization by creating fake reviews.

“If you need fake information about something that looks plausible, you’ve got a way to monetize it,” said Marcus. “Either because you’re working for a state government that wants to have a misinformation company, or just somebody who wants to make money off clicks.”

On Monday, Stack Overflow temporarily banned users from sharing answers from ChatGPT because they were often just plain wrong. “While the answers which ChatGPT produces have a high rate of being incorrect,” the company explained, “they typically look like they might be good and the answers are very easy to produce.”

Marcus described the issue with Stack Overflow as “existential.”

“If [Stack Overflow] can’t solve this problem,” Marcus said, “then the value of their information diminishes and the site loses its reason for existence.”

It’s not just the spreading of blatantly false information that we need to be concerned about. As Tyler Cowen pointed out in the Washington Post, ChatGPT can interfere with our ability to accurately measure public opinion. For example, ChatGPT has the ability to impersonate constituents by writing false letters to Congress about a particular policy.

“Over time, interest groups will employ ChatGPT, and they will flood the political system with artificial but intelligent content,” Cowen reported.

— Khaya Himmelman