Stop Prompting and Start Engineering: Treating LLMs as Unreliable Functions

Short summary

The article argues developers should stop treating LLMs as magic boxes and instead treat them as unreliable functions that need validation layers. Key recommendations: define strict schemas with Zod or Pydantic, implement retry loops that send error messages back to the LLM, use enums to limit output space, and maintain a golden dataset of 50-100 test cases to catch regressions when prompts change.

• •Treat LLM outputs as untrusted input — validate with strict schemas before use
• •Use retry loops that feed specific error messages back to the LLM for self-correction
• •Maintain a golden dataset of 50-100 test cases to measure prompt regressions quantitatively

Generated with AI, which can make mistakes.