AI is Not an Oracle: Designing More Effective Prompts

I read two thought-provoking articles today that got me thinking about how we approach prompting when using generative AI tools. Ted Underwood, writing in the Chronicle of Higher Education, makes a key point – we shouldn’t treat AI as an all-knowing oracle. “Instead of assuming that the model already has an answer to every question in memory,” he writes, “any special assumptions or background knowledge the model will be expected to use” should be included in the prompt.

This aligns perfectly with what we teach about effective prompting. The key is providing enough information in your prompt to enable the AI to give you a meaningful response. We call this the 7 Ps of Prompting:

  • Prime: Set the stage by providing context for your request. Just as you wouldn’t ask a new colleague to jump into a project without background information, don’t expect AI to work in a vacuum.
  • Privacy: Do not share private or confidential information. Remember that your prompts may be used to improve the AI model.  This is essential for legal practitioners.
  • Persona: Tell the AI what perspective and tone you want it to take. This helps shape the response to match your needs, whether that’s a formal analysis or a casual explanation.
  • Product: Be clear about what type of output you want – an outline, draft, analysis, or something else entirely.
  • Purpose: Explain the goal of your request. This helps the AI understand not just what you want, but why you want it.
  • Prompt: Give specific, clear instructions. Vague requests lead to vague responses.
  • Polish: Use follow-up questions to refine and verify the AI’s output. Think of it as a conversation rather than a one-shot query.

A great example of this contextual approach comes from Kevin O’Keefe’s recent post in AI Legal Journal.

Last week, I started asking ChatGPT questions about the characteristics of a good legal blog. The answers were poor.

I then asked the same questions asking ChatGPT to use a corpus of solely my legal blogging – what I had written in this blog. About 10,000 blog posts over twenty years most about blogging.

The answers were remarkably better with my blog posts being the corpus used to train the AI as compared to AI being trained by everything on the net, including all the junk written about legal blogs.

So, the next time you use generative AI, take a moment to think through the 7 Ps before writing your prompt. You might be surprised at how much more useful the responses become when you treat AI less like an all-knowing oracle and more like a capable assistant that needs proper context to help you effectively.

Posted in AI