For renters · Updated October 13, 2026

How much GPU memory (VRAM) do you need? Chat models, image models and fine-tuning

KernelMachine is preparing an invite-only beta; no launch date is set. Where this guide describes KernelMachine, it's how it works at launch. Join the waitlist

Short answer: for chat models, plan on about 0.6 GB per billion parameters at 4-bit, or 2 GB per billion at full 16-bit, plus a few GB for the conversation. For images, 12 GB covers SDXL and 24 GB covers Flux at full precision. For fine-tuning, QLoRA fits an 8B model in about 10 to 16 GB; a full fine-tune needs about 16 bytes per parameter.

GPU memory, usually called VRAM (video memory), is the memory built into the graphics card. A model has to fit in it to run at full speed. If it doesn't fit, you either get a "CUDA out of memory" error, or the software moves part of the model into normal system memory and runs many times slower. That's why memory, not the card's name, is the first thing to choose.

Chat and text models

A model's size is counted in parameters: the numbers it learned in training. "8B" means 8 billion. Each parameter takes a fixed amount of memory, depending on how precisely it's stored:

So: memory for the model ≈ parameters × bytes per parameter. Then add room for the conversation (below).

Room for the conversation

While it answers, a model keeps a working memory of the conversation so far (the "KV cache"). It grows with every word. For Llama 3 8B it's about 128 KB per token (a token is roughly three-quarters of a word), so an 8,000-token conversation adds about 1 GB. For a 70B Llama model it's about 320 KB per token, so a 32,000-token conversation adds about 10 GB. Long documents and many users at once need much more headroom.

Mixture-of-experts models

Some models, such as OpenAI's gpt-oss and several Qwen models, only use part of themselves for each word. That makes them fast, but memory still follows the total size. OpenAI says gpt-oss-120b runs on a single 80 GB GPU and gpt-oss-20b within 16 GB (OpenAI).

Image and video models

Higher resolutions, bigger batches, upscalers and extra models (LoRAs, ControlNets) all add memory. If you're unsure, go one size up: an hour on a bigger card often costs less than an afternoon fighting out-of-memory errors. Running these in ComfyUI? See ComfyUI on a rented GPU.

Fine-tuning

Training needs far more memory than running a model, because it also stores how each parameter should change.

• Full fine-tune: about 16 bytes per parameter with the usual Adam optimizer in mixed precision (the weights, their changes, and the optimizer's running notes), before counting activations. An 8B model needs about 128 GB, so several 80 GB cards.
• LoRA: the base model stays frozen at 16-bit (2 bytes per parameter) and only small add-on layers train. An 8B model fits a 24 GB card with small batches; 40 to 48 GB is comfortable.
• QLoRA: the frozen base is stored at 4-bit. An 8B model fits in about 10 to 16 GB. The QLoRA paper fine-tuned a 65B model on a single 48 GB GPU (Dettmers et al., 2023).

Which card has how much memory

When memory isn't the problem

Once the model fits, speed comes from the card's memory bandwidth (for chat) and raw compute (for images and training). A model split across two cards runs at the pace of the slower one, and the link between the cards matters. For most single-card jobs, fitting comfortably matters more than having the newest card.

How KernelMachine uses this

On KernelMachine you pick what you want to do and we suggest how much card memory: 16–20 GB for Notebook and Chat, 24 GB for Image maker and Fine-tuning. Pick more if your model needs it (the biggest cards have 32 GB or more), then take the cheapest card with that much, or one by name. Cards under 16 GB aren't rented: they're too tight for most of what people run today. Which models need which size.

Read next

• How to rent a GPU in 2026: what to check before you pay
• Renting a GPU vs buying one: a simple way to decide
• Why rented GPU jobs fail, and how to protect your work