Project 0: Fun with Large Language Models

Overview

“ChatGPT doesn’t give you information. It gives you information-shaped sentences.” - Neil Gaiman

Large language models (LLMs) such as ChatGPT generate text in a randomized manner according to a learned probability distribution. Consequently, there is no guarantee that the text they generate is correct or even coherent. In this assignment, you will elicit plausible-sounding but factually incorrect text from a large language model.

Assignment

For this assignment, you may work alone or in groups of up to four total students to do the following:

• Select one or more of the following LLMs for experimentation:
  • ChatGPT
  • Bard
  • Perplexity.ai
• Try various prompts to attempt to coerce the LLM to generate a compelling but factually incorrect essay.
• Generate at least ten candidate essays, and select an essay with interesting mistakes that the group considers to be compelling and interesting.
• Your group will present your prompt and resulting essay to the class on Thursday, August 24.
  • Submit the prompt and essay via Teams
• After all essays have been presented, every student in the class will cast one vote for their favorite essay.

Assessment

This project is intended to be short, informal, and fun. Consequently, the project is optional - the assessment consists entirely of bonus tokens, assigned as follows:

• Every student who casts a vote for an essay other than their own will receive one token.
• Each member of a presenting group will receive bonus tokens as follows:
  • One token for devising and presenting the essay, discussing the LLM selected, the strategy behind the prompt, and clear identification of the factual errors contained therein.
  • A second token for receiving a vote from at least one student not in the presenting group.
  • A third token for receiving a plurality of votes from the class.