Adding GPU backends to a pure-C TTS engine: Metal, CUDA, and the rented-Mac trick

Short summary

Pure-C TTS with optional Metal and CUDA GPU backends achieving 6× real-time speedup on M1 (RTF 0.6) and scaling to RTF 0.12 on RTX 4090. Request batching serves N concurrent users in single-request wall-clock time with 2.8–3.35× throughput gains. Author's most valuable lesson: five-minute measurements eliminated two obvious optimizations (ICB, MMA) that would add complexity without delivering real performance benefit.

• •GPU backends (Metal/CUDA) on pure-C TTS; M1 achieves RTF 0.6, scales to RTF 0.12 on RTX 4090
• •Request batching serves N concurrent users in single-request wall-clock time (2.8–3.35× throughput)
• •Measurement discipline: eliminated two obvious optimizations (ICB, MMA) that added complexity without real gains

Generated with AI, which can make mistakes.