The age of local LLMs is here

Short summary

Six months ago, running capable models locally was impractical. But free inference tiers have vanished, while new Qwen3 models and improved tooling (llama.cpp router mode, context cache persistence) now make local LLMs viable. The author benchmarks three Qwen3 variants on dual RX6800 GPUs, showing performance comparable to Claude 4.5 Opus—the long-awaited moment when developers can self-host capable AI.

• •Alibaba's new Qwen3 models (27B, 35B MoE, 80B Coder) are capable enough to run locally on high-end consumer GPUs
• •llama.cpp adds router mode and context cache persistence, making model switching and multi-model workflows practical
• •Free API inference is disappearing; local models now offer better cost-performance and privacy for developers

Generated with AI, which can make mistakes.