OpenAI has created a tool that can successfully detect the use of ChatGPT in text, as reported by the Wall Street Journal. However, the leading A.I. company is internally debating whether the ability to catch students cheating on schoolwork with A.I. outweighs a potential loss of ChatGPT users. Upon its release in November of 2022, ChatGPT immediately presented a new set of challenges for educators looking to ensure their students aren’t using it for schoolwork. As of last November, one in five teenagers who had heard of ChatGPT were using it in assignments, according to a survey from the Pew Research Center. Numerous companies, including OpenAI itself, have attempted to solve the issue by releasing A.I. detection technology. But these tools have proved to be largely inaccurate, with OpenAI pulling its own A.I. detection software last June after it displayed only a 26 percent accuracy rate and a tendency to release false positives.
However, the leading A.I. company for a year has held onto an alternative method that detects ChatGPT in text with an effectiveness of 99.9 percent, as reported by the Journal, which cited internal documents and people familiar with the matter. The tool operates by altering how ChatGPT selects which words come next in a sentence, leaving behind a watermark pattern that the company can detect. OpenAI CEO Sam Altman is reportedly a supporter of the project and involved in discussions surrounding the tool but hasn’t yet urged for its release.
Why hasn’t OpenAI released its anti-cheating tool?
“The text watermarking method we’re developing is technically promising, but has important risks we’re weighing while we research alternatives,” an OpenAI spokesperson told Observer in a statement. “We believe the deliberate approach we’ve taken is necessary given the complexities involved and its likely impact on the broader ecosystem beyond OpenAI.”
One of OpenAI’s primary concerns is that releasing the tool will push users away from ChatGPT and towards rivals that do not integrate the watermarking technique. A test conducted by the company in 2023 found that nearly 30 percent of respondents said they would use the A.I. generator less often if the anti-cheating method was integrated, according to the Journal.
Employees were also worried that the tool would impact ChatGPT’s performance. However, that concern was proven unfounded during a test held by OpenAI earlier this year, which found that the tool didn’t negatively affect the chatbot. “Our ability to defend our lack of text watermarking is weak now that we know it doesn’t degrade outputs,” read an internal document from employees involved in the testing, the Journal reported.
Yesterday (August 4), OpenAI updated a May blog post to confirm that it has developed a text watermarking detection method that it continues “to consider as we research alternatives.” The company said it has prioritized audio and visual detection tools for now, as A.I.-generated images and sound are currently “widely considered to present higher levels of risk.”
When it comes to its text watermarking tool, OpenAI noted that the method is “less robust against globalized tampering,” meaning it could be susceptible to manipulation from bad actors that get around the detection tool with translation systems or by rewording text with other generative models. Research from OpenAI has also suggested that the tool could disproportionately impact non-native English speakers who benefit from using ChatGPT as a writing tool, it said.
