Arrakis / Models / Llama-3.3-70B-Instruct

Llama-3.3-70B-Instruct GPU requirements

Llama-3.3-70B-Instruct has 71B parameters. To serve it, plan for about 169 GB of GPU memory in 16-bit, 85 GB in 8-bit or 42 GB in 4-bit, including about 20% for the KV cache and activations.

Deploy this model Check another model

What it takes to serve it

GPU memory to run it for inference, with the smallest GPU setup that holds it and the lowest price on Arrakis right now.

Precision	Memory and the smallest setup
16-bit	169 GB: one B200, from $6.79 per GPU-hour
8-bit	85 GB: one RTX PRO 6000, from $1.90 per GPU-hour
4-bit	42 GB: one L40S, RTX 6000 Ada or RTX A6000, L40S from $0.44 per GPU-hour

What it takes to fine-tune it

QLoRA trains adapters on a 4-bit frozen model, LoRA on a 16-bit one, and full fine-tuning updates every weight. Figures are before activations, which grow with batch size and sequence length.

Method	Memory and the smallest setup
QLoRA	49 GB: one H100 or A100, H100 from $2.25 per GPU-hour
LoRA	169 GB: one B200, from $6.79 per GPU-hour
Full fine-tuning	1,129 GB: 8 × B200, from $54.32 per hour for all 8

About the model

From its card and files on Hugging Face. Memory figures here follow the GPU memory guide.

Parameters: 71B
Published in: BF16
License: llama3.3
Released: 2024-11-26
Weights: meta-llama/Llama-3.3-70B-Instruct

Deploy it on Arrakis

Open Models in the console, choose vLLM, SGLang or llama.cpp, and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine. Or run it yourself on any machine with enough GPU memory:

vLLM

vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 1

Deploy this model How deploying works

Questions

Live prices for every GPU are on the GPU prices page.

How much GPU memory does Llama-3.3-70B-Instruct need?

To serve it, about 169 GB in 16-bit, 85 GB in 8-bit, 42 GB in 4-bit, including about 20% for the KV cache and activations. To fine-tune it, about 49 GB with QLoRA, 169 GB with LoRA and 1,129 GB for full fine-tuning with Adam, before activations.

Can I run Llama-3.3-70B-Instruct on one GPU?

Yes. In 16-bit it needs about 169 GB, which fits on one B200.

What does it cost to run Llama-3.3-70B-Instruct?

In 16-bit it needs one B200; in 4-bit, one L40S, RTX 6000 Ada or RTX A6000. Each setup on this page shows the lowest price per GPU-hour on Arrakis right now, and you pay by the hour from a prepaid balance.

How do I deploy Llama-3.3-70B-Instruct on Arrakis?

Open Models in the Arrakis console, paste meta-llama/Llama-3.3-70B-Instruct, choose vLLM, SGLang or llama.cpp, and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine.

More models

Every model, and a check for any other.