Why did my benchmark stop at N=22? A debugging story in nine bugs

Short summary

A detailed debugging story tracing why an A2A agent benchmark stopped at N=22 instead of N=24. The root cause was CPython 3.11's 4,300-digit int-to-string conversion cap crashing on the 24th Mersenne prime, compounded by a decorative str() call, an LLM prose-parsing regex that never matched Gemini's phrasing, and a chart that misleadingly compared LLM-routed agents against bare HTTP handlers. The author fixed nine cascading bugs and re-ran a clean 96/96 sweep.

• •CPython 3.11's int→str 4300-digit cap silently crashed the benchmark at N=23; the offending str() was decorative dead code
• •LLM prose-parsing regex never matched Gemini's actual phrasing; data only survived via a structured-artifact fallback
• •Chart labeled as a language comparison was actually a pipeline comparison: ~400x gap between LLM-routed and direct HTTP agents
• •Nine bugs total, including Go outputting nanoseconds the parser couldn't handle and Gemini refusing to rerun work due to reused context IDs

Generated with AI, which can make mistakes.