VRAM and RAM: which memory should you look at when choosing a GPU?

01What are these two types of memory for?

System RAM is used by the system, file loading, processes and data preparation. Video memory, or VRAM, holds the data that the GPU needs for its computations. NVMe storage keeps the files; it serves yet another purpose.

Some software can move part of the data between CPU memory and GPU. This possibility depends on the hardware and the engine used. It does not mean that all system RAM automatically becomes VRAM, nor does it guarantee the same performance.

02For an AI model, count more than just its weights

The weights are the model's parameters. Their size in memory depends in particular on their number and on the chosen precision. As a simplified calculation, seven billion parameters at two bytes represent about 14 billion bytes, before the software's other needs. This calculation therefore does not guarantee that a 16 GB card will be sufficient.

During inference, the KV cache holds information useful for generation. Its usage varies with the architecture, the sequence lengths and the number of requests processed together. The engine's workspaces and any other loaded models must also be taken into account.

03Inference, training and rendering do not require the same thing

Inference consists of using a model to produce a result. Training or certain adaptation methods add other working data, notably related to gradient computation and optimization. A measurement obtained in inference does not automatically size a training run.

For 3D rendering, instead test a representative scene with its textures, geometry and engine settings. Two projects using the same software can have very different memory requirements. The software's name alone is therefore not enough to choose a capacity.

04Read the GPU WispHost grid

The grid distinguishes video memory from the server's RAM. The five configurations include a full dedicated card. The values below are catalog capacities, not performance results or a promise that a particular model will fit in memory.

05Prepare a reproducible test

Record the model or scene, its version, the precision, the engine, the batch size and the input sizes. Then measure the peak memory usage and the time needed to produce the result. For an interactive service, add a concurrent load corresponding to your expected users.

Keep these parameters with your results. Without them, comparing two speed figures may amount to comparing different tasks. If memory overflows, examine the engine settings and their effects before concluding that you simply need to change card.

06Choose a margin suited to your workload

Start with the capacity needed for the most demanding representative case, then allow a margin for variations in data and concurrency. Then compare GPU, RAM, vCPU, storage and network together. A more powerful card does not fix data loading or CPU processing that limits the entire chain.