AI

OpenAI releases tool to detect AI-generated text, including from ChatGPT

Comment

OpenAI's logo
Image Credits: OpenAI

After telegraphing the move in media appearances, OpenAI has launched a tool that attempts to distinguish between human-written and AI-generated text — like the text produced by the company’s own ChatGPT and GPT-3 models. The classifier isn’t particularly accurate — its success rate is around 26%, OpenAI notes — but OpenAI argues that it, when used in tandem with other methods, could be useful in helping prevent AI text generators from being abused.

“The classifier aims to help mitigate false claims that AI-generated text was written by a human. However, it still has a number of limitations — so it should be used as a complement to other methods of determining the source of text instead of being the primary decision-making tool,” an OpenAI spokesperson told TechCrunch via email. “We’re making this initial classifier available to get feedback on whether tools like this are useful, and hope to share improved methods in the future.”

As the fervor around generative AI — particularly text-generating AI — grows, critics have called on the creators of these tools to take steps to mitigate their potentially harmful effects. Some of the U.S.’ largest school districts have banned ChatGPT on their networks and devices, fearing the impacts on student learning and the accuracy of the content that the tool produces. And sites including Stack Overflow have banned users from sharing content generated by ChatGPT, saying that the AI makes it too easy for users to flood discussion threads with dubious answers.

OpenAI’s classifier — aptly called OpenAI AI Text Classifier — is intriguing architecturally. It, like ChatGPT, is an AI language model trained on many, many examples of publicly available text from the web. But unlike ChatGPT, it’s fine-tuned to predict how likely it is that a piece of text was generated by AI — not just from ChatGPT, but any text-generating AI model.

More specifically, OpenAI trained the OpenAI AI Text Classifier on text from 34 text-generating systems from five different organizations, including OpenAI itself. This text was paired with similar (but not exactly similar) human-written text from Wikipedia, websites extracted from links shared on Reddit and a set of “human demonstrations” collected for a previous OpenAI text-generating system. (OpenAI admits in a support document, however, that it might’ve inadvertently misclassified some AI-written text as human-written “given the proliferation of AI-generated content on the internet.”)

The OpenAI Text Classifier won’t work on just any text, importantly. It needs a minimum of 1,000 characters, or about 150 to 250 words. It doesn’t detect plagiarism — an especially unfortunate limitation considering that text-generating AI has been shown to regurgitate the text on which it was trained. And OpenAI says that it’s more likely to get things wrong on text written by children or in a language other than English, owing to its English-forward dataset.

The detector hedges its answer a bit when evaluating whether a given piece of text is AI-generated. Depending on its confidence level, it’ll label text as “very unlikely” AI-generated (less than a 10% chance), “unlikely” AI-generated (between a 10% and 45% chance), “unclear if it is” AI-generated (a 45% to 90% chance), “possibly” AI-generated (a 90% to 98% chance) or “likely” AI-generated (an over 98% chance).

Out of curiosity, I fed some text through the classifier to see how it might manage. While it confidently, correctly predicted that several paragraphs from a TechCrunch article about Meta’s Horizon Worlds and a snippet from an OpenAI support page weren’t AI generated, the classifier had a tougher time with article-length text from ChatGPT, ultimately failing to classify it altogether. It did, however, successfully spot ChatGPT output from a Gizmodo piece about — what else? — ChatGPT.

According to OpenAI, the classifier incorrectly labels human-written text as AI-written 9% of the time. This mistake didn’t occur in my testing, but I chalk that up to the small sample size.

OpenAI text classifier
Image Credits: OpenAI

On a practical level, I found the classifier not particularly useful for evaluating shorter pieces of writing. Indeed, 1,000 characters is a difficult threshold to reach in the realm of messages, for example emails (at least the ones I get on a regular basis). And the limitations give pause — OpenAI emphasizes that the classifier can be evaded by modifying some words or clauses in generated text.

That’s not to suggest the classifier is useless — far from it. But it certainly won’t stop committed fraudsters (or students, for that matter) in its current state.

The question is, will other tools? Something of a cottage industry has sprung up to meet the demand for AI-generated text detectors. ChatZero, developed by a Princeton University student, uses criteria including “perplexity” (the complexity of text) and “burstiness” (the variations of sentences) to detect whether text might be AI-written. Plagiarism detector Turnitin is developing its own AI-generated text detector. Beyond those, a Google search yields at least a half-dozen other apps that claim to be able to separate the AI-generated wheat from the human-generated chaff, to torture the metaphor.

It’ll likely become a cat-and-mouse game. As text-generating AI improves, so will the detectors — a never-ending back-and-forth similar to that between cybercriminals and security researchers. And as OpenAI writes, while the classifiers might help in certain circumstances, they’ll never be a reliable sole piece of evidence in deciding whether text was AI-generated.

That’s all to say that there’s no silver bullet to solve the problems AI-generated text poses. Quite likely, there won’t ever be.

More TechCrunch

Apple has published a technical paper detailing the models that it developed to power Apple Intelligence, the range of generative AI features headed to iOS, macOS and iPadOS over the…

Apple says it took a ‘responsible’ approach to training its Apple Intelligence models

A fireside chat on Monday between Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg at the SIGGRAPH 2024 conference in Colorado took a few unexpected turns. It started innocently…

Huang and Zuckerberg swapped jackets at SIGGRAPH 2024 and things got weird

Meta’s machine learning model, Segment Anything, has a sequel: It now takes the model to the video domain, showing how fast the field is moving.

Zuckerberg touts Meta’s latest video vision AI with Nvidia CEO Jensen Huang

Featured Article

The fall of EV startup Fisker: A comprehensive timeline

Here is a timeline of the events that led fledgling automaker Fisker to file for bankruptcy.

The fall of EV startup Fisker: A comprehensive timeline

Hello, and welcome back to TechCrunch Space. In case you missed it, Boeing and NASA decided to keep Starliner docked to the International Space Station for the rest of the…

TechCrunch Space: Catching stars

As failed EV startup Fisker winds its way through bankruptcy, a persistent and tricky question has become a flashpoint of the proceedings: does its only secured lender, Heights Capital Management,…

The question haunting Fisker’s bankruptcy

So-called “unlearning” techniques are used to make a generative AI model forget specific and undesirable info it picked up from training data, like sensitive private data or copyrighted material. But…

Making AI models ‘forget’ undesirable data hurts their performance

Uber is now letting riders in India book up to three rides simultaneously.

Uber now lets users in India book three trips at once

U.S. airports are rolling out facial recognition to scan travelers’ faces before boarding their flights. Americans, at least, can opt out. 

How to opt out of facial recognition at airports (if you’re American)

The promise of AI and large language models (LLMs) is the ability to understand increasingly wider amounts of context and make sense of that information easily, so it makes sense…

Bee AI raises $7M for its wearable AI assistant that learns from your conversations

Featured Article

DEI backlash: Stay up-to-date on the latest legal and corporate challenges

It’s clear that this year will be a turning point for DEI.

DEI backlash: Stay up-to-date on the latest legal and corporate challenges

Bike-taxi startup Rapido, which counts Swiggy among its investors, is the latest Indian firm to become a unicorn.

India’s Rapido becomes a unicorn with fresh $120M funding

Government websites aren’t known for cutting-edge tech. GovWell co-founder and CTO Ben Cohen discovered this while trying to help his dad, a contractor, apply for building permits. Cohen worked as…

GovWell is bringing automation and efficiency to local governments

Critics have long argued that wararantless device searches at the U.S. border are unconstitutional and violate the Fourth Amendment.

US border agents must get warrant before cell phone searches, federal court rules

Featured Article

UK’s Zapp EV plans to expand globally with an early start in India

Zapp is launching its urban electric two-wheeler in India in 2025 as it plans to expand globally.

UK’s Zapp EV plans to expand globally with an early start in India

The first time I saw Google’s latest commercial, I wondered, “Is it just me, or is this kind of bad?” By the fourth or fifth time I saw it, I’d…

Dear Google, who wants an AI-written fan letter?

Featured Article

MatPat, the first big YouTuber to successfully exit his company, is lobbying for creators on Capitol Hill

Though MatPat retired from YouTube, he’s still pretty busy. In fact, he’s been spending a lot of time on Capitol Hill.

MatPat, the first big YouTuber to successfully exit his company, is lobbying for creators on Capitol Hill

Featured Article

A tale of two foldables

Samsung is still foldables’ 500-pound gorilla, but the company successes have made the category significantly less lonely in recent years.

A tale of two foldables

The California Department of Motor Vehicles this week granted Nuro approval to test its third-generation R3 autonomous delivery vehicle in four Bay Area cities, giving the AV startup a positive…

Autonomous delivery startup Nuro is gearing up for a comeback

With Ghostery turning 15 years old this month, TechCrunch caught up with CEO Jean-Paul Schmetz to discuss the company’s strategy and the state of ad tracking.

Ghostery’s CEO says regulation won’t save us from ad trackers

Two years ago, workers at an Apple Store in Towson, Maryland, were the first to establish a formally recognized union at an Apple retail store in the United States. Now…

Apple reaches its first contract agreement with a US retail union

OpenAI is testing SearchGPT, a new AI search experience to compete directly with Google. The feature aims to elevate search queries with “timely answers” from across the internet and allows…

OpenAI comes for Google with SearchGPT

Indian cryptocurrency exchange WazirX announced on Saturday a controversial plan to “socialize” the $230 million loss from its recent security breach among all its customers, a move that has sent…

WazirX to ‘socialize’ $230M security breach loss among customers

Featured Article

Stay up-to-date on the amount of venture dollars going to underrepresented founders

Stay up-to-date on the latest funding news for Black and women founders.

Stay up-to-date on the amount of venture dollars going to underrepresented founders

The National Institute of Standards and Technology (NIST), the U.S. Commerce Department agency that develops and tests tech for the U.S. government, companies and the broader public, has re-released a…

NIST releases a tool for testing AI model risk

Featured Article

Max Space reinvents expandable habitats with a 17th-century twist, launching in 2026

Max Space’s expandable habitats promise to be larger, stronger, and more versatile than anything like them ever launched, not to mention cheaper and lighter by far than a solid, machined structure.

Max Space reinvents expandable habitats with a 17th-century twist, launching in 2026

Payments giant Stripe has acquired a four-year-old competitor, Lemon Squeezy, the latter company announced Friday. Terms of the deal were not disclosed. As a merchant of record, Lemon Squeezy calculates…

Stripe acquires payment processing startup Lemon Squeezy

iCloud Private Relay has not been working for some Apple users across major markets, including the U.S., Europe, India and Japan.

Apple reports iCloud Private Relay global outages for some users

Welcome to Startups Weekly — your weekly recap of everything you can’t miss from the world of startups. To get Startups Weekly in your inbox every Friday, sign up here. This…

Legal tech, VC brawls and saying no to big offers

Apple joins 15 other tech companies — including Google, Meta, Microsoft and OpenAI — that committed to the White House’s rules for developing generative AI.

Apple signs the White House’s commitment to AI safety