From API to GPU, Week 2: What Actually Happens Behind the API

Short summary

This is week 2 of a 32-week series on running local AI models, focusing on what happens behind the API when using Ollama on a DGX Spark. The author inspects model internals by listing downloaded models, querying Ollama's API for model metadata (format, architecture, parameters, context length, quantization), and reading the manifest files that define how models are packaged as blobs. The article explains the distinction between model weights as static data and the runtime that executes them, using a Docker-like analogy.

• •Ollama stores models as GGUF blobs with manifests describing layers (model, template, license, params)
• •Model weights are static data; the runtime loads and executes them — analogous to Docker images vs containers
• •Demonstrates querying Ollama API and reading manifest files to inspect Phi-4 model internals on DGX Spark

Generated with AI, which can make mistakes.