GPU VRAM Requirements Calculator

Out-of-memory errors at step 3 of training are one of the most demoralizing experiences in ML. You planned everything, the model architecture looks right, and then — RuntimeError: CUDA out of memory. The fix isn't always obvious because GPU memory usage depends on a lot more than just model size.

What Actually Eats Your VRAM

Most people underestimate memory needs because they only think about model weights. Here's the full picture:

• Model parameters — the weights themselves, at 4 bytes each (fp32) or 2 bytes (fp16/bf16)
• Gradients — same size as the parameters during backprop
• Optimizer states — Adam stores 2 momentum buffers per parameter (2× model size), making total training memory roughly 4× the weight size in fp32
• Activations — intermediate values from the forward pass, scale with batch size
• Input batch — often small but varies with sequence length in transformers

The CalcHub GPU Memory Calculator handles all five components. You enter your model type, parameter count, batch size, and precision, and it breaks down memory usage component by component.

Memory Breakdown by Precision Mode

Component	fp32	fp16/bf16	fp16 + AMP
Weights (per param)	4 bytes	2 bytes	2 bytes
Gradients	4 bytes	2 bytes	4 bytes (master copy)
Adam optimizer states	8 bytes	4 bytes	8 bytes
Activations (per token)	~100 bytes	~50 bytes	~50 bytes
Total multiplier vs weights	~16×	~8×	~12×

For a 1 billion parameter model: fp32 training needs roughly 16 GB just for weights + optimizer + gradients, before you count any activations.

Practical Example: BERT-base Fine-Tuning

BERT-base: 110 million parameters

Setup	VRAM Needed
Inference only (fp32)	~440 MB
Inference only (fp16)	~220 MB
Fine-tuning, batch 8, fp32, Adam	~5.5 GB
Fine-tuning, batch 32, fp16, Adam	~4.2 GB
Fine-tuning, batch 32, bf16 + gradient checkpointing	~3.1 GB

A single RTX 3080 (10 GB) handles BERT-base fine-tuning comfortably at batch 32. A T4 (16 GB) handles it easily. A consumer GTX 1080 (8 GB) can struggle at larger batches.

Strategies When You're Short on VRAM

Gradient checkpointing trades compute for memory by recomputing activations during backprop instead of storing them. Typically adds 20–30% to training time but can halve activation memory. Gradient accumulation lets you use a smaller physical batch while achieving the same effective batch size — useful when the batch itself is eating your VRAM budget. Quantized training (QLoRA) is the current standard for fine-tuning large language models on consumer hardware. Loading a 7B model in 4-bit NF4 format takes roughly 4 GB instead of 14 GB in fp16. Reduce sequence length. In transformers, attention memory scales quadratically with sequence length. Halving your max sequence length can free up a lot of activation memory.

How much VRAM do I need for LLM inference?

Rule of thumb: 2 bytes per parameter in fp16. A 7B model needs ~14 GB, a 13B model needs ~26 GB. In 4-bit quantization, divide by 4: 7B fits in ~4 GB, 13B in ~7 GB. These are minimum figures — add 1–2 GB for KV cache during generation.

Can I train across multiple GPUs to split memory?

Yes. Model parallelism splits layers across GPUs (each holds different weights). Tensor parallelism splits individual layers. Pipeline parallelism splits by microbatches. Frameworks like DeepSpeed and FSDP automate this. For most setups, data parallelism (each GPU has a full model copy) is simpler and doesn't save per-GPU memory.

Why does my VRAM usage spike at the start of training?

CUDA allocates memory in chunks and PyTorch caches allocations. The first few batches trigger allocations and the reserved memory shows up before the model's actual working set stabilizes. Use torch.cuda.max_memory_allocated() after a few steps to get the real peak.

Related Calculators

• Model Parameters Calculator — count parameters before estimating memory
• Batch Size Calculator — find the largest batch that fits your GPU
• Training Time Calculator — plan your compute budget