LLM VRAM Requirements: What Fits on 8, 16, 24, 48 and 80GB
Have you ever found yourself stuck in this question: "will this model fit on my GPU?". A lot of us have. The honest answer is always "it depends on quantization and context length," which is true but not actually useful…
