# Llama-3.3-70B-Instruct GPU requirements

Llama-3.3-70B-Instruct has 71B parameters. To serve it, plan for about 169 GB of GPU memory in 16-bit, 85 GB in 8-bit or 42 GB in 4-bit, including about 20% for the KV cache and activations.

[Deploy it on Arrakis](/console/#/models?model=meta-llama/Llama-3.3-70B-Instruct).

## What it takes to serve it

| Precision | Memory and the smallest setup |
|---|---|
| 16-bit | 169 GB: one [B200](/gpus/b200.html.md), from $6.79 per GPU-hour |
| 8-bit | 85 GB: one [RTX PRO 6000](/gpus/rtx-pro-6000.html.md), from $1.90 per GPU-hour |
| 4-bit | 42 GB: one [L40S](/gpus/l40s.html.md), [RTX 6000 Ada](/gpus/rtx-6000-ada.html.md) or [RTX A6000](/gpus/rtx-a6000.html.md), L40S from $0.44 per GPU-hour |

## What it takes to fine-tune it

| Method | Memory and the smallest setup |
|---|---|
| QLoRA | 49 GB: one [H100](/gpus/h100.html.md) or [A100](/gpus/a100.html.md), H100 from $1.65 per GPU-hour |
| LoRA | 169 GB: one [B200](/gpus/b200.html.md), from $6.79 per GPU-hour |
| Full fine-tuning | 1,129 GB: 8 × [B200](/gpus/b200.html.md), from $54.32 per hour for all 8 |

## About the model

| | |
|---|---|
| Parameters | `71B` |
| Published in | `BF16` |
| License | `llama3.3` |
| Released | `2024-11-26` |
| Weights | [meta-llama/Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) |

## Serve it yourself

```
vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 1
```

## Questions

**How much GPU memory does Llama-3.3-70B-Instruct need?** To serve it, about 169 GB in 16-bit, 85 GB in 8-bit, 42 GB in 4-bit, including about 20% for the KV cache and activations. To fine-tune it, about 49 GB with QLoRA, 169 GB with LoRA and 1,129 GB for full fine-tuning with Adam, before activations.

**Can I run Llama-3.3-70B-Instruct on one GPU?** Yes. In 16-bit it needs about 169 GB, which fits on one B200.

**What does it cost to run Llama-3.3-70B-Instruct?** In 16-bit it needs one B200; in 4-bit, one L40S, RTX 6000 Ada or RTX A6000. Each setup on this page shows the lowest price per GPU-hour on Arrakis right now, and you pay by the hour from a prepaid balance.

**How do I deploy Llama-3.3-70B-Instruct on Arrakis?** Open Models in the Arrakis console, paste meta-llama/Llama-3.3-70B-Instruct, choose vLLM, SGLang or llama.cpp, and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine.


More models: [/models](/models.html.md).