# deepseek-coder-7b-instruct-v1.5 GPU requirements

deepseek-coder-7b-instruct-v1.5 has 6.9B parameters. To serve it, plan for about 17 GB of GPU memory in 16-bit, 8.3 GB in 8-bit or 4.1 GB in 4-bit, including about 20% for the KV cache and activations.

[Deploy it on Arrakis](/console/#/models?model=deepseek-ai/deepseek-coder-7b-instruct-v1.5).

## What it takes to serve it

| Precision | Memory and the smallest setup |
|---|---|
| 16-bit | 17 GB: one [RTX 4090](/gpus/rtx-4090.html.md) or [RTX 3090](/gpus/rtx-3090.html.md), RTX 3090 from $0.19 per GPU-hour |
| 8-bit | 8.3 GB: one [RTX 4090](/gpus/rtx-4090.html.md) or [RTX 3090](/gpus/rtx-3090.html.md), RTX 3090 from $0.19 per GPU-hour |
| 4-bit | 4.1 GB: one [RTX 4090](/gpus/rtx-4090.html.md) or [RTX 3090](/gpus/rtx-3090.html.md), RTX 3090 from $0.19 per GPU-hour |

## What it takes to fine-tune it

| Method | Memory and the smallest setup |
|---|---|
| QLoRA | 4.8 GB: one [RTX 4090](/gpus/rtx-4090.html.md) or [RTX 3090](/gpus/rtx-3090.html.md), RTX 3090 from $0.19 per GPU-hour |
| LoRA | 17 GB: one [RTX 4090](/gpus/rtx-4090.html.md) or [RTX 3090](/gpus/rtx-3090.html.md), RTX 3090 from $0.19 per GPU-hour |
| Full fine-tuning | 111 GB: one [H200](/gpus/h200.html.md), from $4.03 per GPU-hour |

## About the model

| | |
|---|---|
| Parameters | `6.9B` |
| Context window (tokens) | `4,096` |
| Published in | `BF16` |
| License | `other` |
| Released | `2024-01-25` |
| Weights | [deepseek-ai/deepseek-coder-7b-instruct-v1.5](https://huggingface.co/deepseek-ai/deepseek-coder-7b-instruct-v1.5) |

## Serve it yourself

```
vllm serve deepseek-ai/deepseek-coder-7b-instruct-v1.5 --tensor-parallel-size 1
```

## Questions

**How much GPU memory does deepseek-coder-7b-instruct-v1.5 need?** To serve it, about 17 GB in 16-bit, 8 GB in 8-bit, 4 GB in 4-bit, including about 20% for the KV cache and activations. To fine-tune it, about 5 GB with QLoRA, 17 GB with LoRA and 111 GB for full fine-tuning with Adam, before activations.

**Can I run deepseek-coder-7b-instruct-v1.5 on one GPU?** Yes. In 16-bit it needs about 17 GB, which fits on one RTX 4090 or RTX 3090.

**What does it cost to run deepseek-coder-7b-instruct-v1.5?** In 16-bit it needs one RTX 4090 or RTX 3090; in 4-bit, one RTX 4090 or RTX 3090. Each setup on this page shows the lowest price per GPU-hour on Arrakis right now, and you pay by the hour from a prepaid balance.

**How do I deploy deepseek-coder-7b-instruct-v1.5 on Arrakis?** Open Models in the Arrakis console, paste deepseek-ai/deepseek-coder-7b-instruct-v1.5, choose vLLM, SGLang or llama.cpp, and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine.


More models: [/models](/models.html.md).