What it takes to serve it
GPU memory to run it for inference, with the smallest GPU setup that holds it and the lowest price on Arrakis right now.
| Precision | Memory and the smallest setup |
|---|---|
| 16-bit | 9.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| 8-bit | 4.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| 4-bit | 2.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
What it takes to fine-tune it
QLoRA trains adapters on a 4-bit frozen model, LoRA on a 16-bit one, and full fine-tuning updates every weight. Figures are before activations, which grow with batch size and sequence length.
| Method | Memory and the smallest setup |
|---|---|
| QLoRA | 2.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| LoRA | 9.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Full fine-tuning | 64 GB: one H100 or A100, H100 from $2.07 per GPU-hour |
About the model
From its card and files on Hugging Face. Memory figures here follow the GPU memory guide.
- Parameters
- 4.0B
- Context window (tokens)
- 262,144
- Published in
- BF16
- License
- apache-2.0
- Released
- 2025-08-05
- Weights
- Qwen/Qwen3-4B-Instruct-2507
Deploy it on Arrakis
Open Models in the console, choose vLLM, SGLang or llama.cpp, and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine. Or run it yourself on any machine with enough GPU memory:
vLLM
vllm serve Qwen/Qwen3-4B-Instruct-2507 --tensor-parallel-size 1Questions
Live prices for every GPU are on the GPU prices page.
How much GPU memory does Qwen3-4B-Instruct-2507 need?
To serve it, about 10 GB in 16-bit, 5 GB in 8-bit, 2 GB in 4-bit, including about 20% for the KV cache and activations. To fine-tune it, about 3 GB with QLoRA, 10 GB with LoRA and 64 GB for full fine-tuning with Adam, before activations.
Can I run Qwen3-4B-Instruct-2507 on one GPU?
Yes. In 16-bit it needs about 9.7 GB, which fits on one RTX 4090 or RTX 3090.
What does it cost to run Qwen3-4B-Instruct-2507?
In 16-bit it needs one RTX 4090 or RTX 3090; in 4-bit, one RTX 4090 or RTX 3090. Each setup on this page shows the lowest price per GPU-hour on Arrakis right now, and you pay by the hour from a prepaid balance.
How do I deploy Qwen3-4B-Instruct-2507 on Arrakis?
Open Models in the Arrakis console, paste Qwen/Qwen3-4B-Instruct-2507, choose vLLM, SGLang or llama.cpp, and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine.