How we calculate local AI memory
By Delcio Pedro · methodology reviewed 15 September 2026
A compatibility verdict is a deterministic estimate. It answers whether a particular build is likely to fit the stated hardware and settings. It does not benchmark response speed or judge answer quality.
Worked example: Llama 3.1 8B
The example uses the canonical Ollama tag llama3.1:8b, mapped to the instruction-tuned Q4_K_M build, at 4,096 tokens. Tags can change; the source and retrieval dates below identify the catalog data used for this rendering.
Build manifest · catalog checked 2026-09-13. GGUF architecture retrieved 2026-09-15 from this source.
- Weights: 4.6 GB
- Recorded model weight bytes; download metadata is not a measured runtime allocation. (4,920,738,944 bytes)
- KV cache: 0.5 GB
- 32 layers × 8 KV heads × (128 key + 128 value dimensions) × 4096 tokens × 2 bytes (f16). (536,870,912 bytes)
- Ollama runtime reserve: 0.4 GB
- 400 MiB engineering allowance, not a measured value for your runtime. (419,430,400 bytes)
- Compute buffers: 0.4 GB
- The larger of 128 MiB or 8% of weight bytes, rounded up. (393,659,116 bytes)
- Total: 5.8 GB
- The sum of those four components. GB on this site means 1,024³ bytes. (6,270,699,372 bytes)
ollama run llama3.1:8bIn the interactive prompt, enter /set parameter num_ctx 4096. Then run your own prompt. This page records a calculation; no inference execution is claimed.
From memory to a hardware verdict
Hypothetical Windows PC: RTX 4070 12 GB and 32 GB system RAM. Illustrative 8-core CPU and 500 GB free disk, so these estimates do not check your actual CPU or storage. Available RAM, GPU budget, free VRAM and driver version are unknown. The engine applies memory reserves. GPU paths assume a working supported driver/backend; runtime-version compatibility is not checked. Apple memory is one shared pool. One GPU is evaluated; memory is not pooled across cards.
| Model / exact build | Download | Est. memory | Result on this example PC |
|---|---|---|---|
| llama3.1 llama3.1:8b Est. memory: 5.8 GB Download: 4.6 GB | 4.6 GB | 5.8 GB | Excellent fit medium confidence |
How this result was calculated
Assumptions and missing information
GB uses 1,024³ bytes. Memory includes weights, context cache and runtime overhead. | |||
The engine selects one GPU, considers its supported Ollama backend, reserves memory for the operating system, and estimates offload when a full allocation does not fit. System RAM and storage receive separate checks. Apple Silicon uses one unified pool. Physical VRAM, current budget and currently free memory are different inputs.
What makes a result uncertain?
- Missing build architecture triggers a fallback KV estimate and lowers confidence. A different-sized build cannot borrow its family's layer count.
- Missing available memory or driver information stays unknown. Reference hardware assumes a working supported backend.
- Runtime reserves and offload splits are estimates. Concurrency, vision inputs and runtime settings can change actual allocations.
- Encoder-only BERT has no persistent autoregressive decode cache in this model; temporary activations still need buffers. Embedding capability alone does not prove an architecture is encoder-only.
- A longer context increases the calculated cache but cannot extend the model's documented context limit.
The same build, hardware and configuration produce the same engine report. Pages render its checks directly. Report a correction with the exact tag, context, runtime and a public source through the project issue channel.