Meta · 8.03B parameters · Llama 3.1 Community License
What GPU do you need to run Llama 3.1 8B?Memory, cards and speed.
Llama 3.1 8B needs about 6 GB of GPU memory (VRAM) at its smallest (4-bit), so a 16 GB card such as an RTX 4080 runs it. The cheapest card on our list that fits, an RTX 4060 Ti 16GB, costs about $0.14 an hour. At its largest (16-bit) it needs about 18 GB.
GPU memory
Llama 3.1 8B, by version.
Includes about 1.5 GB for the program and a short conversation (about 4,000 tokens). Long conversations need more. Speeds are ceilings: real speeds are usually half to two-thirds of them. How we work it out
| Version | Memory needed | Card size | Cards that fit | From | Top speed |
|---|---|---|---|---|---|
| 4-bit | about 6 GB | 16–20 GB | RTX 4090, RTX 5090, RTX 3090, RTX 4080, RTX 3090 Ti, RTX 5080, RTX 5070 Ti, RTX 4070 Ti SUPER, RTX 5060 Ti 16GB, RTX 4060 Ti 16GB | $0.14/hr on an RTX 4060 Ti 16GB | about 372 tokens/s on an RTX 5090 |
| 8-bit | about 10 GB | 16–20 GB | RTX 4090, RTX 5090, RTX 3090, RTX 4080, RTX 3090 Ti, RTX 5080, RTX 5070 Ti, RTX 4070 Ti SUPER, RTX 5060 Ti 16GB, RTX 4060 Ti 16GB | $0.14/hr on an RTX 4060 Ti 16GB | about 211 tokens/s on an RTX 5090 |
| 16-bit | about 18 GB | 24 GB | RTX 4090, RTX 5090, RTX 3090, RTX 3090 Ti | $0.155/hr on an RTX 3090 | about 112 tokens/s on an RTX 5090 |
A small, well-supported all-rounder, and a good first model to try. Downloading it from Hugging Face means accepting Meta's license there first; Ollama has it ready.
Run it
Start Llama 3.1 8B in minutes.
On KernelMachine, Rent a Chat model with it: it loads into the chat app by itself once the rental is ready, straight from Ollama to the machine, and is in the app's model menu. On any machine with Ollama, one command downloads and runs it. Ollama on a rented GPU
$ ollama run llama3.1:8b
# Model files: huggingface.co/meta-llama/Llama-3.1-8B-Instruct
Common questions
Questions, answered plainly.
How much VRAM does Llama 3.1 8B need?
4-bit: about 6 GB; 8-bit: about 10 GB; 16-bit: about 18 GB. That includes about 1.5 GB for the program and a short conversation; longer conversations need more.
Can an RTX 4090 run Llama 3.1 8B?
Yes. Llama 3.1 8B (16-bit) needs about 18 GB, which fits in the RTX 4090's 24 GB.
How fast does Llama 3.1 8B run?
At 4-bit, the ceiling is about 372 tokens/s on an RTX 5090: each word means reading the whole model from memory once. Real speeds are usually half to two-thirds of that.
Can I run Llama 3.1 8B on KernelMachine?
Yes. Rent a Chat model with Llama 3.1 8B, on this page, starts the chat app on the cheapest card with 16 GB or more, and llama3.1:8b loads into it by itself once it's ready. Already renting a Chat model? Pick Llama 3.1 8B on the rental's page, or download llama3.1:8b from the chat app's model menu.
Other models
Compare with other chat models.
Preparing an invite-only beta
Get an email when access opens.
No launch date is set. We'll email you once when KernelMachine opens. Nothing else.