Learning AI orchestration and harness engineering by building an autonomous engineer for a bank

Short summary

An engineer shares lessons from building an autonomous agent at Eko that resolves support work against a live financial system with no human intervention. The key insight is that the model is the easy part — the real work is the harness: orchestration loops, retry logic, adversarial evaluation, and strict separation between what the system can reason about and what it is allowed to do. A self-improving research loop can optimize matching logic but is structurally forbidden from touching write actions or permission logic.

• •The harness — not the model — is what separates a demo from production-grade autonomy
• •Adversarial evaluation with frozen test cases prevents silent regressions and safety erosion
• •A self-improving research loop can change reasoning but is structurally barred from changing permissions

Generated with AI, which can make mistakes.