Project 0: Fun with Large Language Models

Overview

“ChatGPT doesn’t give you information. It gives you information-shaped sentences.” - Neil Gaiman

Large language models (LLMs) such as ChatGPT generate text in a randomized manner according to a learned probability distribution. Consequently, there is no guarantee that the text they generate is correct or even coherent. In this assignment, you will elicit plausible-sounding but factually incorrect text from a large language model.

Assignment

For this assignment, you may work alone or in groups of up to five total students to do the following:

• Select one or more of the following LLMs for experimentation:
  • ChatGPT
  • Gemini
  • Claude
  • Llama
  • Perplexity.ai
• Try various prompts to attempt to coerce the LLM to generate a compelling but factually incorrect essay.
• Generate at least ten candidate essays, and select an essay with interesting mistakes that the group considers to be compelling and interesting.
• Your group will present your prompt and resulting essay to the class on Friday, August 29.
  • Submit the AI you employed, your prompt, and the generated essay via Teams
• After all essays have been presented, every student in the class will cast one vote for their favorite essay.

Assessment

This project is intended to be short, informal, and fun. Consequently, the project is optional - the assessment consists entirely of bonus credits, assigned as follows:

• One credit for devising and presenting the essay, discussing the LLM selected, the strategy behind the prompt, and clear identification of the factual errors contained therein.
• A second credit for receiving a plurality of votes from the class.