The teacher did not respond and removed Ostowitz’s grade.
Ostowitz’s mother, Stephanie Rizk, says her daughter is a high-achieving student who cares about doing well in school, and she was alarmed when the teacher jumped to conclusions about Ostowitz’s work so early in the school year.
“Get to know their skill level, and then maybe your AI detector is useful,” says Rizk.
Rizk told NPR that she met with the teacher in mid-November, and the teacher said they never saw her daughter’s message.
The school district, Prince George’s County Public Schools, clarified in a statement that Ostowitz’s teacher used an AI detection tool himself and that the district did not pay for that software.
“During staff training, we advise educators not to rely on such tools, as multiple sources have documented their potential inaccuracies and inconsistencies,” the statement said.
PGCPS declined to make Ostowitz’s teacher available for an interview. Rizk told NPR that after their meeting, the teacher no longer believed Ostowitz was using AI.
But what happened to Ostowitz is not surprising.
More than 40 percent of 6th through 12th grade teachers surveyed used AI discovery tools in the last school year, according to nationally representative survey from the Center for Democracy and Technology, a nonprofit organization that advocates for civil rights and civil liberties in the digital age.
This is despite numerous research studies showing that AI detection tools are far from reliable.
“It’s now pretty well established in the academic integrity field that these tools are not fit for purpose,” says Mike Perkins, lead researcher on academic integrity and AI at the British University in Vietnam.
Perkins found that some of the most popular AI detectors — including Turnitin, GPTZero, and Copyleaks — flag some things as AI that aren’t, and vice versa. Their accuracy rates dropped further when the AI text was manipulated to appear more human.
“We’ve seen some really troubling problems with some of the most productive AI text discovery tools,” he says.
Despite these problems, NPR found that school districts from Utah to Ohio to Alabama spend thousands of dollars on these tools.
Why one of the largest districts in the country is using AI detection software
Near Miami, Broward County Public Schools is spending more than $550,000 on a three-year contract with Turnitin. The long-standing electronic technology company has provided schools with plagiarism detection software in the past; in 2023 introduced an AI discovery feature. When educators put student work through this tool, it generates a percentage that reflects the amount of text the software determines is likely AI-generated. A caveat: According to the companyresults of 20% or lower are less reliable.
“The Turnitin tool is something that helps us facilitate conversation and feedback, not assessment,” says Sherry Wilson, director of innovative learning for the Broward School District, which enrolls more than 230,000 students and is one of the largest school districts in the country.
Wilson says the district is “fully aware” of research showing AI detection tools, including Turnitin, are not 100 percent accurate or reliable.
Turnitin also recognizes this: On the company’s websiteit said, “our detection of AI writing may not always be accurate … so it should not be used as the sole basis for adverse action against a student.”
Turnitin wrote in a statement to NPR that it is more important to avoid falsely accusing students of cheating than to catch all AI texts.
Wilson says the Turnitin tool is still valuable because it saves teachers time by quickly scanning student work for suspected AI use.
Another reason Broward teachers have access to the tool, Wilson said, is because the district participates in academic programs, such as the International Baccalaureate or IB, in which student work must be certified by teachers before it is sent for external review.
Both programs offered by Broward, IB and International Education at Cambridge, told NPR that schools are not required to use AI detection software as part of the credentialing process. However, Broward told NPR in a statement, “we chose to provide our teachers with (Turnitin) as one of the tools to fulfill the requirements.”
But Wilson says teachers are the ultimate authority on whether a student’s work is theirs, not the AI’s discovery tool.
“They use these tools as feedback to have these teachable moments with the students,” she says.
Why a teacher uses AI discovery tools
Language and literature teacher John Grady says that for him, the AI detection tools provide a “starting point” to start a conversation with a student who may have used AI.
“It’s certainly not foolproof,” he says. “But it gives you something to hang your hat on.”
Grady teaches at Shaker Heights High School, part of the Shaker Heights City School District outside of Cleveland. The district serves roughly 4,400 students and is paying GPTZero, another AI discovery software company, about $5,600 this year for annual licenses for 27 of the district’s teachers. The tool calculates a percentage probability that the student’s work was generated by AI.
Grady says he runs all student essays through GPTZero; if the tool shows more than a 50% chance that an AI was used for the job, Grady digs deeper. This includes using revision history tools to see how much time a student has spent on an assignment and how many edits they’ve made during the writing process. If it turns out that a student has only made a few edits and spent almost no time writing, it will be checked in with that student.
“And I’ll say, ‘Hey, it’s flagged.’ Can you tell me why? I’d say most of the time, about 75%, if it was AI, they’d say, “Yeah, I did.” And I’m like, ‘Okay, now you have to rewrite it with less credit,'” Grady says.
Edward Tian, co-founder and CEO of GPTZero, says this is how educators must to use his company’s tool.
“We definitely don’t believe it’s a tool for punishment,” Tian says. “It should be a tool in the toolbox, not the latest smoking gun.”
He says it’s important to understand that a GPTZero probability score below 50% means the text is more likely to have been generated by humans than AI. He says scores above 50% warrant a closer look — similar to what Grady describes.
Tian doesn’t dispute the research that shows GPTZero isn’t always reliable. But he notes that there are educators, like Grady, who still find it valuable for the information it provides.
He says tools like his offer “a signal of what’s going on in your classroom,” but that teachers should always reach out to students if that signal indicates something alarming.
Skeptics in AI discovery
Shaker Heights junior Zee Shih, whose first language is Mandarin, says his writing style can sometimes seem like an artificial intelligence “because of the repetition of the words I use. I feel like it’s because of how limited my vocabulary is.”
Shi, who is not a Grady student, says he is still working on his writing skills and is concerned that AI detection software may be biased toward non-English speakers like himself.
Some educators share this concern, although research so far is limited and controversial.
Shi says an assignment he did for his English class earlier this fall was flagged by GPTZero as possibly AI-generated. He says his teacher suggested that using an online tool called Grammarly may have triggered the detection software. Grammarly uses AI to correct grammar and, if prompted, generate text. (The teacher confirmed Shea’s account with NPR.)
Shi says he only used Grammarly to clean up his writing and that he wrote the assignment himself. “It was definitely disappointing to see the comment that it was labeled as AI,” Shi says.
Shi thinks AI detectors should be seen as “a smoke alarm when it’s a sign or a warning. But, you know, sometimes it can be like a false alarm.”
He questions whether the school district should spend thousands of dollars on AI detection software. He says the money could be better spent on professional development for teachers.
Carrie Cofer, a high school English teacher in the Cleveland Metropolitan School District — just a few miles from Shaker Heights — shares that sentiment.
Last year, as an experiment, she raised a chapter from her Ph.D. thesis at GPTZero. “And it came out like 89% or 91% AI-written, and I was like, ‘Oh, no, I don’t think that’s right, because it was all mine,'” Cofer says.
Cofer helps his district shape its AI policy and guidelines; she says Cleveland schools don’t currently pay for AI detection software, and she would advocate against it.
“I don’t think it’s an efficient use of their money,” Cofer says. “Kids will get around it one way or another.”
Some workarounds that students could turn to include using software to detect AI on their own, workshop assignments so that they are not marked, and using “AI humanizer” programs.which claim to make AI-generated writing appear more human.
Ultimately, she says, teachers will need to adapt to AI by changing the way they teach and assess student learning.
Back in Maryland, high school student Ailsa Ostowitz is also adapting. She now runs all her homework through multiple AI detection tools before handing it in.
The writing is her own, she says, but she’ll rewrite sentences the software identifies as likely AI-generated, an extra step that adds about half an hour to each task.
“I think I’ve definitely become more vigilant about presenting my work as mine and not as AI,” she explains.
She doesn’t want to take any chances.
This reporting was supported by a grant from Tarbell Center for AI Journalism.
