Uniqcli

AI memory planner: GPU VRAM & unified memory

Estimate model weights, KV cache and runtime memory for an AI workload. Keep GPU VRAM and unified system memory separate, with documented assumptions and sources.

Compute & memory

AI model memory planner

Separate weights, architecture-specific KV cache and runtime allocations in a discrete or unified memory budget.

Example inputs · Replace these with your measurements to build your plan.

MLA, latent/compressed caches and special architectures require a measured manual allocation.

Method, example and sources
  1. Convert parameter count and stored bits into a weights-only byte lower bound.
  2. Add explicit metadata and either a measured KV allocation or the entered MHA/GQA shape: 2 × layers × KV heads × head dimension × tokens × sequences × bytes.
  3. Add runtime workspace and subtract only the OS/shared allocations belonging to this same memory pool.
  4. Compare one device’s budget; retain unknown allocations and validate the exact runtime before choosing hardware.

Worked example. Seven billion weights at 4 bits occupy about 3.260 GiB before metadata. Adding 0.5 GiB metadata, 2 GiB KV and 2 GiB runtime gives 7.760 GiB; a 24 GiB device with 1 GiB reserved leaves 23 GiB to compare.

Planning result · confirm model details

Compare the declared working set with the available budget

Discrete VRAM: one device, with OS/shared allocations counted once. Weights alone are a lower bound.

Weights lower bound
3.26GiB
KV cache
2GiB
Declared model/runtime total
7.76GiB
Budget after other allocations
23GiB
Remaining budget
15.24GiB
  • Unmeasured or omitted allocations prevent a confirmed-fit result.
  • Do not add multiple GPUs or Spark nodes into a single flat memory pool. Standard KV math applies only to the entered MHA/GQA shape, without cache sharing, sharding or compression.

From the catalog

Graphic Cards

Explore related listings, then confirm the exact model requirements against your plan. These listings are not an automatic compatibility or capacity verdict.

Browse graphic cards Browse servers Browse desktop ai systems Get help with the specification

What the result means

Model weights, attention cache and execution workspace occupy memory for different reasons. Changing stored weight precision does not automatically change the KV data type or remove quantization metadata. The shape calculation explicitly asks for KV heads, which can differ from query heads in grouped-query attention. A latent or compressed cache needs its own measured allocation instead of this standard shape.

Before choosing a product

Discrete GPU memory is separate from ordinary host RAM. A coherent unified system instead shares one pool between CPU, GPU and other processes, so system allocations reduce the same budget. Manufacturer capacity is not measured available memory, and adding node capacities does not establish supported model partitioning. Treat this worksheet as a requirement record, then test the model, context and concurrency on the chosen runtime.

Worked example

Seven billion weights at 4 bits occupy about 3.260 GiB before metadata. Adding 0.5 GiB metadata, 2 GiB KV and 2 GiB runtime gives 7.760 GiB; a 24 GiB device with 1 GiB reserved leaves 23 GiB to compare.

AI memory planner: GPU VRAM & unified memory — Uniqcli