Glossary / LLM APIs and AI
Glossary · LLM APIs and AI · checked Oct 24, 2026
VRAM (for running LLMs)
Short answerVRAM is the memory on a graphics card. To run an open-weights model locally, its weights (plus the KV cache for your context) have to fit in VRAM; a 7-billion-parameter model needs about 14 GB at 16-bit and about 4 GB at 4-bit.
Why it matters
VRAM, not GPU speed, decides which model sizes and context lengths you can run at all.
Our number
Our LLM VRAM calculator works it out for a model size, precision and context length.
Go deeper on this site
Where it shows up here
It appears on 9 pages of this site: 4 Articles, 2 paths, 2 Free tools, 1 developer. A few to start with:
- Rust, LLMs, AI agents and app dev, with the receipts.
- Meta open-sourced an 816B model. Running it takes about 1TB of RAM
- LLM VRAM calculator.
- Token counter.
- Use Tensorust from your code or your agent.
- AI news, checked against the sources.
- Run models yourself.
Sources
- Hugging Face: model memory estimator: huggingface.co/docs/accelerate/usage_guides/model_size_estimator (checked Oct 24, 2026)
Related terms
Up next
Go deeperVRAM calculatorThe page on this site that does the most with this term.Read it next →Did this answer your question?