Detecting AI in Student Papers

Tags Assignment AI

If you are concerned a student may have used ChatGPT or another generative AI to complete their assignment, what should you do?

If you think an assignment may have been generated by AI, there are several tools that can assist in determining the likelihood that a student used AI. It is important to note, however, that none of these are foolproof, and may return false positives. The best first step is usually to have a conversation about the assignment with the student.

GPTZero

GPTZero allows you to copy and paste text into its web-based AI detection tool. GPTZero analyzes a text to look for what it describes as "perplexity" and "burstiness." Perplexity determines the likelihood that a word was recommended by an AI (assuming that humans choose more varied words). Burstiness measures the frequency in spikes of unique language throughout the document (assuming that an AI will be less variable).

Writer AI Content Detector

Writer.com offers an AI content detector that can detect the likelihood that up to 1500 characters of text were AI-generated. This detector examines whether a text is likely to follow the same pattern of words that a large language model would. Since this model is based on the sequence of words, it may flag works are being AI generated if the writer uses similar word sequences in their writing.

The Evolution of Large Language AI

It is important to note that AI is constantly evolving, and the reliability of these tools (both for negative and positive results) will be incredibly variable. These detection tools are meant to provide guidance that can begin a conversation.

Design Considerations to Mitigate the Risk of AI Misuse

Assignment design choices can potentially decrease AI use for assignments. Here are a few suggestions, in no particular order, for ways to mitigate the risk of AI misuse.

  • Build assignments that leverage personal experiences. Increasing relevance and authenticity are effective ways to increase student engagement with coursework, as well as help students to connect new information to prior knowledge.
  • Ask students to submit work in different pieces and with different revisions.
  • Require process documentation: The research and writing process can be documented by the student as evidence of their work. Word processing files, spreadsheets, and presentation slide decks will typically have version history that can be examined to verify the gradual development of a submission. (NOTE: When a document is submitted to a Canvas assignment, the version history is lost; Canvas makes a copy of the file. However, if you suspect a student of potential AI misuse, you could ask the student to separately email you their paper as a file attachment.)
  • Develop an AI use policy collaboratively with students to gain student buy-in and to engage them deeply in thinking about appropriate use.
  • Guide students in how to appropriately cite/reference the use of generative AI. Include this in an AI use policy.
  • Coach students on appropriate ways to engage with AI as a part of working on an assignment. For example, having students start with an AI tool can allow them to elaborate on and revise provided outputs to create something new. Alternatively, AI can be consulted in a latter stage of the process to act as a feedback coach.
  • Showcase for students the flaws of an AI-generated product. Review examples of AI outputs and why these are insufficient (e.g., generic treatment; fake citations).
  • Require students to cite core course texts. It is harder for students to ask AI to include specific scholarly sources in its generated output and it will be easier for you to verify whether citations are accurate because you are familiar with the texts.
  • Review the balance of work and assessment done in-class versus outside of class. Consider administering key summative assessments in-class, which could facilitate monitoring student behavior.
  • Consider offering writing assignments as Essay Questions in a Canvas Quiz, and requiring Lockdown Browser. This will prevent students from navigating to other websites to access AI content generators. This method will be especially effective for in-class, monitored writing assignments. For assignments to be completed outside of class, students would still be able to use other devices to access AI content generators, but they would not be able to copy and paste that text directly.
Print Article

Details

Article ID: 155440
Created
Wed 11/8/23 10:07 AM
Modified
Mon 9/2/24 8:00 AM