Why does full fine-tuning need so much more memory than running the model?

Inference stores one copy of the weights. Training stores four things — the weights, a gradient for every weight, the optimizer state (AdamW keeps two running averages, about 8 bytes per parameter), and usually an fp32 master copy of the weights for mixed precision. That is roughly 16 bytes per parameter before activations, versus 2 bytes for fp16 inference.

How much does QLoRA actually save?

A lot. The base model is frozen in 4-bit, so it costs about 0.5 bytes per parameter and has no gradient or optimizer state. Only the small LoRA adapters are trained. A 70B model that needs ~1.5 TB for a full fine-tune drops to something that fits on a single 24 to 48 GB card with QLoRA, at some speed and quality cost.

What does gradient checkpointing change?

It trades compute for memory. Instead of keeping every layer's activations for the backward pass, it keeps a few checkpoints and recomputes the rest. Activation memory drops from roughly linear in layer count to roughly the square root of it, at the cost of about 20 to 30 percent more compute per step.

The estimate seems off for my exact setup.

It is a first-order model. It does not account for the framework (DeepSpeed, FSDP, Unsloth), CPU or NVMe offload, flash-attention's activation savings, packing, or your exact LoRA target modules. Use it to pick a method and a batch size, then confirm with a real run.

// training

Fine-tuning VRAM Calculator — Full vs LoRA vs QLoRA

Estimate the GPU memory to fine-tune an LLM, broken down into base weights, gradients, optimizer state and activations.

Model

LoRA rank

MethodOptimizer

Batch sizeSequence length

Gradient checkpointing

Total training VRAM

4.2GB

Base weights

3.7GB

Gradients

0.1GB

Optimizer

0.1GB

Activations

0.4GB

base 3.74 GBgrads 0.05 GBoptimizer 0.05 GBactivations 0.35 GB

qlora · Llama 3.1 8B · bs1 · seq 2048 · ckpt → 4.2 GB

Fits on a single GPU (108 of 108)

GTX 1080 Ti 11GB · 11GBRTX 2060 12GB · 12GBRTX 2080 Ti 11GB · 11GBTitan RTX 24GB · 24GBRTX 3050 8GB · 8GBRTX 3060 12GB · 12GBRTX 3060 Ti 8GB · 8GBRTX 3070 8GB · 8GBRTX 3070 Ti 8GB · 8GBRTX 3080 10GB · 10GBRTX 3080 12GB · 12GBRTX 3080 Ti 12GB · 12GBRTX 3090 24GB · 24GBRTX 3090 Ti 24GB · 24GBRTX 4060 8GB · 8GBRTX 4060 Ti 8GB · 8GBRTX 4060 Ti 16GB · 16GBRTX 4070 12GB · 12GBRTX 4070 Super 12GB · 12GBRTX 4070 Ti 12GB · 12GBRTX 4070 Ti Super 16GB · 16GBRTX 4080 16GB · 16GBRTX 4080 Super 16GB · 16GBRTX 4090 24GB · 24GBRTX 5060 Ti 16GB · 16GBRTX 5070 12GB · 12GBRTX 5070 Ti 16GB · 16GBRTX 5080 16GB · 16GBRTX 5090 32GB · 32GBRTX A2000 12GB · 12GBRTX A4000 16GB · 16GBRTX A4500 20GB · 20GBRTX A5000 24GB · 24GBRTX A5500 24GB · 24GBRTX A6000 48GB · 48GBRTX 4000 Ada 20GB · 20GBRTX 4500 Ada 24GB · 24GBRTX 5000 Ada 32GB · 32GBRTX 6000 Ada 48GB · 48GBRTX PRO 5000 Blackwell 48GB · 48GBRTX PRO 6000 Blackwell 96GB · 96GBTesla P40 24GB · 24GBTesla P100 16GB · 16GBTesla V100 16GB (PCIe) · 16GBTesla V100 32GB (SXM2) · 32GBTesla T4 16GB · 16GBA2 16GB · 16GBA10 24GB · 24GBA30 24GB · 24GBA40 48GB · 48GBL4 24GB · 24GBL40 48GB · 48GBL40S 48GB · 48GBA100 40GB · 40GBA100 80GB · 80GBH100 PCIe 80GB · 80GBH100 SXM 80GB · 80GBH100 NVL 94GB · 94GBH200 SXM 141GB · 141GBB100 192GB · 192GBB200 192GB · 192GBB300 (Blackwell Ultra) 288GB · 288GBGB200 (per-GPU) 192GB · 192GBRTX PRO 6000 Blackwell Server 96GB · 96GBRadeon RX 7800 XT 16GB · 16GBRadeon RX 7900 XT 20GB · 20GBRadeon RX 7900 XTX 24GB · 24GBRadeon RX 9070 XT 16GB · 16GBRadeon PRO W7800 32GB · 32GBRadeon PRO W7900 48GB · 48GBInstinct MI210 64GB · 64GBInstinct MI250 128GB · 128GBInstinct MI250X 128GB · 128GBInstinct MI300X 192GB · 192GBInstinct MI325X 256GB · 256GBInstinct MI350X 288GB · 288GBInstinct MI355X 288GB · 288GBInstinct MI455X 432GB (est. specs, pre-GA) · 432GBIntel Arc A770 16GB · 16GBIntel Arc B580 12GB · 12GBIntel Gaudi 2 96GB · 96GBIntel Gaudi 3 128GB · 128GBData Center GPU Max 1550 128GB · 128GBApple M1 Max (32GB unified) · 32GBApple M1 Ultra (64GB unified) · 64GBApple M1 Ultra (128GB unified) · 128GBApple M2 Max (64GB unified) · 64GBApple M2 Ultra (128GB unified) · 128GBApple M2 Ultra (192GB unified) · 192GBApple M3 Pro (36GB unified) · 36GBApple M3 Max (48GB unified) · 48GBApple M3 Max (128GB unified) · 128GBApple M3 Ultra (96GB unified) · 96GBApple M3 Ultra (256GB unified) · 256GBApple M3 Ultra (512GB unified) · 512GBApple M4 (32GB unified) · 32GBApple M4 Pro (48GB unified) · 48GBApple M4 Max (64GB unified) · 64GBApple M4 Max (128GB unified) · 128GBApple M5 (32GB unified) · 32GBApple M5 Pro (64GB unified) · 64GBApple M5 Max (64GB unified) · 64GBApple M5 Max (128GB unified) · 128GBApple M5 Ultra (256GB unified) · 256GBApple M5 Ultra (512GB unified) · 512GBApple M6 (32GB unified) · 32GBAMD Ryzen AI Max+ 395 (128GB unified) · 128GBNVIDIA DGX Spark (GB10, 128GB unified) · 128GB

Presets verified 2026-09. A first-order estimate. Full fine-tune keeps an fp32 master copy plus bf16 grads plus optimizer state — that's why it needs ~4–5× the inference footprint. LoRA/QLoRA train <1% of the parameters, so grads and optimizer state almost vanish and activations dominate. DeepSpeed ZeRO-3 / FSDP shard base+grads+optimizer across GPUs; CPU/NVMe offload trades speed for memory — neither is modelled here.

notes

Where training memory goes

Four buckets, shown as a stacked bar:

Base weights. The model itself. params × 2 for a bf16 full fine-tune, params × 0.5 for a 4-bit QLoRA base.
Gradients. One per trainable parameter, in bf16. For full tuning that is every parameter; for LoRA/QLoRA it is well under 1% of them.
Optimizer state. AdamW keeps two moments per trainable parameter (~8 bytes). 8-bit AdamW cuts that to ~2. Full tuning also keeps an fp32 master copy of the weights (~4 bytes/param).
Activations. Intermediate tensors held for the backward pass. Scales with batch × sequence length × hidden size × layers. Gradient checkpointing knocks this down to roughly √layers.

The rule of thumb

Full fine-tune: ≈ 16 bytes/param before activations. A 7B model wants ~110–150 GB — multi-GPU territory. A 70B is a small cluster.
LoRA: frozen bf16 base (2 B/param) + tiny trainable set. A 7B fits on a single 24 GB card; a 70B on 2× 48 GB.
QLoRA: frozen 4-bit base (0.5 B/param) + tiny trainable set. A 7B fits in ~6–8 GB; a 70B on one 48 GB card, sometimes on 24 GB with a short sequence.

Worked example

Llama 3.1 8B, QLoRA, rank 16, batch 1, sequence 2048, 8-bit AdamW, gradient checkpointing on:

Base (4-bit): 8.03e9 × 0.5 ≈ 4.0 GB
Gradients + optimizer (adapters only): well under 0.5 GB
Activations (checkpointed): ~2–3 GB
Total ≈ 7–8 GB — comfortable on a 12 GB card.

Switch to a full fine-tune of the same model and the total jumps past 120 GB.

Notes

Presets carry a verified date; every field accepts custom values. This is a planning estimate — DeepSpeed ZeRO-3 and FSDP shard the base, gradients and optimizer across GPUs, and CPU/NVMe offload trades speed for memory. Neither is modelled here.

faq

Why does full fine-tuning need so much more memory than running the model?: Inference stores one copy of the weights. Training stores four things — the weights, a gradient for every weight, the optimizer state (AdamW keeps two running averages, about 8 bytes per parameter), and usually an fp32 master copy of the weights for mixed precision. That is roughly 16 bytes per parameter before activations, versus 2 bytes for fp16 inference.
How much does QLoRA actually save?: A lot. The base model is frozen in 4-bit, so it costs about 0.5 bytes per parameter and has no gradient or optimizer state. Only the small LoRA adapters are trained. A 70B model that needs ~1.5 TB for a full fine-tune drops to something that fits on a single 24 to 48 GB card with QLoRA, at some speed and quality cost.
What does gradient checkpointing change?: It trades compute for memory. Instead of keeping every layer's activations for the backward pass, it keeps a few checkpoints and recomputes the rest. Activation memory drops from roughly linear in layer count to roughly the square root of it, at the cost of about 20 to 30 percent more compute per step.
The estimate seems off for my exact setup.: It is a first-order model. It does not account for the framework (DeepSpeed, FSDP, Unsloth), CPU or NVMe offload, flash-attention's activation savings, packing, or your exact LoRA target modules. Use it to pick a method and a batch size, then confirm with a real run.