Validating Distributed LLM Serving Benchmarks with NVIDIA srt-slurm, SLURM Recipes, Parameter Sweeps, and Pareto Analysis

Short summary

A tutorial covering NVIDIA's srt-slurm framework for creating reproducible SLURM benchmark workflows for distributed LLM serving. It walks through converting declarative YAML configs into benchmark pipelines using srtctl, including dry-running built-in recipes and modeling disaggregated prefill-and-decode deployments. The setup is demonstrated in Google Colab with parameter sweeps and Pareto analysis.

• •NVIDIA srt-slurm converts YAML configs into reproducible SLURM benchmark workflows for distributed LLM serving
• •Tutorial covers dry-running recipes, parameter sweeps, and Pareto analysis in Google Colab
• •Includes modeling disaggregated prefill-and-decode deployment architectures

Generated with AI, which can make mistakes.