The original title is about running SOTA models locally on your own hardware. Let me rewrite this for a mobile feed.

Short summary

A detailed field report on running SOTA LLMs locally, covering a $2K entry tier (dual RTX 3090s for 48GB VRAM) and a $40K tier (four RTX PRO 6000 Blackwell cards yielding 384GB VRAM). The real value is in hard-won debugging details: BIOS bifurcation fixes, ASPM link-speed cosmetic issues, NCCL peer-to-peer hangs requiring iommu=off, and ACS stripping scripts. The author runs a quantized GLM-5.2 variant claiming near-Claude Opus quality at 80 tokens/sec.

• •$2K tier uses dual used RTX 3090s; $40K tier uses four RTX PRO 6000 Blackwell cards on a last-gen EPYC Milan platform
• •Critical fixes include disabling ASPM, iommu=off for NCCL, and stripping ACS to enable GPU peer-to-peer
• •Repo ships Docker Compose runners, ZFS weight storage, and a sandboxed VM agent setup with no license attached

Generated with AI, which can make mistakes.