Automatically Evolving Prompt Guidelines for Task-Specific Optimization

Short summary

AGOPS is an automatic approach that evolves task-specific prompt guidelines by leveraging completed task examples to infer missing constraints and context. The authors show that prompt underspecification causes up to 95.3% performance drops that existing prompt optimization techniques cannot recover. Users following AGOPS-evolved guidelines regain 15.5–81.7% of lost performance consistently across math reasoning, medical QA, and coding benchmarks.

• •AGOPS automatically generates task-specific prompt guidelines from reference examples
• •Prompt underspecification causes up to 95.3% performance drops unrecoverable by existing techniques
• •AGOPS guidelines restore 15.5–81.7% of lost performance across math, medical QA, and coding tasks

Generated with AI, which can make mistakes.