How We Test an AI Product Without Burning Credit

Short summary

The author explains how they test an AI-powered course platform end-to-end with Playwright without paying for model calls on every run. Instead of mocking the network or stubbing the UI, they intercept at the chat provider level, swapping only the three functions that talk to the model with scripted responses. This keeps real routing, message plumbing, and UI rendering intact while eliminating the cost and flakiness of live model calls.

• •Swap only the model layer, keep real UI and routing for true E2E coverage
• •Tests drive the actual desktop app via Playwright with scripted provider responses
• •Approach scales across multiple exercises without increasing cost per run

Generated with AI, which can make mistakes.