Move over comedians, there’s a new stand-up act in town.
A recently released study from the University of Southern California found that the AI-generated jokes outperformed those crafted by humans.
Nearly 70% of the participants rated ChatGPT jokes as funnier than those written by regular people. By comparison, 25% favored the human jokes and 5% rated the jokes as equally funny.
While there’s evidence out there for how language models perform on analytical tasks, less is known about their creative side, said Drew Gorenz, a doctoral candidate in the psychology program at USC and one of the study’s researchers.
As a comedy enthusiast himself, Gorenz was curious how ChatGPT would stack up to human comedians.
“They don’t know what it feels like to appreciate a good joke,” he said of language models. “They’re mostly just using pattern recognition.”
The results, he added, “tell us a lot of cool things about humor production that perhaps we don’t need to feel emotions involved in a good joke to tell a good one.”
To conduct the study, both ChatGPT and humans were asked to write jokes based on a variety of prompts. One task involved coming up with funny acronyms for a string of letters. Another was a fill-in-the-blank type prompt based on the party game Quiplash, and the third involved writing a humorous way to describe an unpleasant situation. A separate group then rated the results.
For example: When asked to complete the blank for “A lesser talked about room in the White House: ‘__________,'” humans came up with “The White Padded Room” and “The dog house,” while ChatGPT spun up “Lincoln Bedroom’s Alien Conspiracy Corner” and “The Situation Room’s Snack Closet.”
One important thing to note, Gorenz said, is that stand-up comedy jokes are a lot less funny when you see them in the text only format. “Delivery is such a key part of humor production,” he said.
In a second study, researchers measured how ChatGPT jokes fared compared to those crafted by professional comedy writers by asking the AI chatbot to rewrite headlines from the satirical site The Onion, “America’s Finest News Source.”
Here the human writers fared a bit better: the average humor rating was the same for the Onion headlines and those generated by ChatGPT, said Gorenz.
ChatGPT came up with the top-rated headline “Local Man Discovers New Emotion, Still Can’t Describe It Properly.” In second place was one from The Onion: “Man Locks Down Marriage Proposal Just As Hair Loss Becomes Noticeable.”
The USC study comes at a time when the entertainment professionals — comedians included — are fretting over how AI could reshape their jobs.
In January, the estate of George Carlin filed a lawsuit against a media company, alleging it used artificial intelligence to recreate the late standup comic’s style and material.
As far as Gorenz is concerned, the results of the study indicate that ChatGPT could disproportionately disrupt comedy and entertainment, especially given that the bar for accuracy in those industries might be lower when compared to say science, education and journalism.
Still, he doesn’t think America’s favorite stand-up comedians are going anywhere anytime soon. “I don’t think it’s able to create a John Mulaney level joke,” he said.