Mistral AI · 24B parameters · Apache 2.0
What GPU do you need to run Mistral Small 3.2 24B?Memory, cards and speed.
Mistral Small 3.2 24B needs about 16 GB of GPU memory (VRAM) at its smallest (4-bit), so a 16 GB card such as an RTX 4080 runs it. The cheapest card on our list that fits, an RTX 4060 Ti 16GB, costs about $0.14 an hour. At its largest (16-bit) it needs about 50 GB.
GPU memory
Mistral Small 3.2 24B, by version.
Includes about 1.5 GB for the program and a short conversation (about 4,000 tokens). Long conversations need more. Speeds are ceilings: real speeds are usually half to two-thirds of them. How we work it out
| Version | Memory needed | Card size | Cards that fit | From | Top speed |
|---|---|---|---|---|---|
| 4-bit | about 16 GB | 16–20 GB | RTX 4090, RTX 5090, RTX 3090, RTX 4080, RTX 3090 Ti, RTX 5080, RTX 5070 Ti, RTX 4070 Ti SUPER, RTX 5060 Ti 16GB, RTX 4060 Ti 16GB | $0.14/hr on an RTX 4060 Ti 16GB | about 124 tokens/s on an RTX 5090 |
| 8-bit | about 27 GB | 32 GB and up | RTX 5090 | $0.524/hr on an RTX 5090 | about 70 tokens/s on an RTX 5090 |
| 16-bit | about 50 GB | None | More than any consumer card holds | — | — |
Quick for its size and good at following instructions and calling tools. Apache 2.0.
Run it
Start Mistral Small 3.2 24B in minutes.
On KernelMachine, Rent a Chat model with it: it loads into the chat app by itself once the rental is ready, straight from Ollama to the machine, and is in the app's model menu. On any machine with Ollama, one command downloads and runs it. Ollama on a rented GPU
$ ollama run mistral-small3.2:24b
# Model files: huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506
Common questions
Questions, answered plainly.
How much VRAM does Mistral Small 3.2 24B need?
4-bit: about 16 GB; 8-bit: about 27 GB; 16-bit: about 50 GB. That includes about 1.5 GB for the program and a short conversation; longer conversations need more.
Can an RTX 4090 run Mistral Small 3.2 24B?
Yes. Mistral Small 3.2 24B (4-bit) needs about 16 GB, which fits in the RTX 4090's 24 GB.
How fast does Mistral Small 3.2 24B run?
At 4-bit, the ceiling is about 124 tokens/s on an RTX 5090: each word means reading the whole model from memory once. Real speeds are usually half to two-thirds of that.
Can I run Mistral Small 3.2 24B on KernelMachine?
Yes. Rent a Chat model with Mistral Small 3.2 24B, on this page, starts the chat app on the cheapest card with 16 GB or more, and mistral-small3.2:24b loads into it by itself once it's ready. Already renting a Chat model? Pick Mistral Small 3.2 24B on the rental's page, or download mistral-small3.2:24b from the chat app's model menu.
Other models
Compare with other chat models.
Preparing an invite-only beta
Get an email when access opens.
No launch date is set. We'll email you once when KernelMachine opens. Nothing else.