Two Professors Explain Why AI Plagiarism Can Be Hard To Prove

A few weeks ago, we witnessed the launch of the AI chatbot ChatGPT. The new AI chatbot has since been getting a lot of buzz for its numerous abilities. Several reports have claimed this new AI tool is able to have intelligent-sounding conversations, write music, code and even more. Two professors who say they caught students cheating on essays using ChatGPT explain why AI plagiarism can be difficult to detect. 

A Professor of philosophy and religious studies at Northern Michigan University recently discovered his student-submitted essays generated using ChatGPT. Antony Aumann revealed his findings after using an Open AI’s ChatGPT detector indicating that it was 99% likely that the essay had been artificially generated. Another philosophy professor at Furman University, Darren Hick having a similar case claimed that he became concerned when his student put in an essay on a subject that contained some well-written false information. These discoveries have opened up a cause to worry about plagiarism in institutions.

Aumann narrated after submitting them back to the chatbot because the writing style raised red flags, asking how likely it was that the software had authored them. He forwarded the results to the students after the chatbot indicated that it was 99% certain that ChatGPT had produced the essays. Hick and Aumann both admitted to confronting their students, who later acknowledged the infraction. The student from Hick’s class failed, and Aumann let his student rewrite the essays from the scratch.

Hick said that “Word by word it was a well-written essay,” but taking a closer look, one can tell there was something fishy, as some claim about the prolific philosopher, David Hume “made no sense” and was “just flatly wrong.” “Really well-written wrong was the biggest red flag,” he said

Aumann reveals that “the chat writes better than 95% of my students could ever.” He added that “All of a sudden you have someone who does not demonstrate the ability to think or write at that level, writing something that follows all the requirements perfectly with sophisticated grammar and complicated thoughts that are directly related to the prompt for the essay,” he said. For Aumann, the chatbot just wrote too perfectly and raised some concerns.

According to Christopher Bartel, a philosophy professor at Appalachian State University, while the grammar in articles produced by AI is almost error-free, the content frequently lacks specific details. Content from the AI-enabled chatbox “are really fluffy. There’s no context, there’s no depth or insight,” Barte says. 

Bartel admitted that academics may find themselves in difficult situations if their students don’t admit to employing AI for their essays. A number of institutions are yet to roll out regulations to curb this form of cheating. It can be really challenging to justify the use of AI if students decide to defend their position.

Bartel further explained that although the offering of the AI detectors was “good but not perfect.” “They give a statistical analysis of how likely the text is to be AI-generated, so that leaves us in a difficult position if our policies are designed so that we have to have definitive and demonstrable proof that the essay is a fake,” he said. “If it comes back with a 95% likelihood that the essay is AI generated, there’s still a 5% chance that it wasn’t.” 

Although the detection site gave a “99% certainty”  that the essay had been written with the help of an AI, Hick claimed that wasn’t enough for him without a student actually confessing to using an AI. Hick revealed that “The confession was important because everything else looks like circumstantial evidence,” he said. “With AI-generated content, there is no material evidence, and material evidence has a lot more weight to it than circumstantial evidence.”

Aumann on the other hand stated that even though he believed the chatbot’s analysis is enough premise to issue disciplinary action, AI plagiarism remains a very fresh conversation institutions need to take on. Aumann claims that “Unlike plagiarism cases of old where you can just say, ‘hey, here’s the paragraph from Wikipedia.’ There is no knockdown proof” other than the statistical likelihood the detector provides.