Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)

Short summary

A detailed benchmark test reveals that adding an older Quadro P2000 (5GB, Pascal) alongside an RTX 3090 (24GB, Ampere) provides zero extra usable context window in Ollama or vLLM, while causing a 74% decode-speed penalty. vLLM flatly rejects tensor-parallel across the two cards due to compute capability mismatch (AWQ Marlin kernel requires 7.5+, Pascal is 6.1). The key takeaway: check compute capability against your quantization scheme before doing VRAM math.

• •Second GPU (Quadro P2000 + RTX 3090) yields zero extra context window in Ollama/vLLM
• •74% decode-speed hit when P2000 enters compute path at ctx=49152
• •vLLM rejects tensor-parallel due to compute capability mismatch (needs 7.5+, Pascal is 6.1)
• •Check compute capability before VRAM math when planning multi-GPU setups

Generated with AI, which can make mistakes.