The original title is "Testing AI Agents with AgentSpec: Jest for Non-Deterministic Behavior"

Short summary

AgentSpec is a new open-source testing framework built for AI agents, replacing exact string matching with assertions like semantic similarity, tool-call verification, and LLM-based judging. It stores last-passing outputs and generates behavior diff reports so you can see exactly what changed when a prompt or model swap causes a regression. Ships with CI/CD integration, JUnit XML output, a GitHub Action, and a local Ollama-based judge mode.

• •AgentSpec provides non-deterministic assertions (semantic similarity, tool_called, llm_judge) for AI agent testing
• •Behavior diff reports show what changed between passing and failing runs
• •CI/CD ready with JUnit XML, GitHub Action, and local Ollama judge support

Generated with AI, which can make mistakes.