The Silent Killer of Edge AI: How to Master Thermal Throttling and Prevent the "Performance Cliff"

Short summary

Thermal throttling on mobile devices causes the 'performance cliff'—inference latency spikes from 30ms to 150ms when the NPU hits thermal limits. Android's PowerManager API and Google's AICore enable reactive thermal monitoring, allowing apps to dynamically swap model versions and precision based on available thermal headroom.

• •Thermal throttling causes dramatic latency spikes in mobile AI apps due to clock speed reduction
• •Android's PowerManager API exposes thermal states (NONE, LIGHT, MODERATE, SEVERE, CRITICAL) for adaptive response
• •Implement reactive monitoring with Kotlin StateFlow to dynamically adjust model precision and version based on device thermal headroom

Generated with AI, which can make mistakes.