Arrakis / Models
Can I run it?
The GPU memory each popular open model needs, the GPUs it fits on and what they cost by the hour. Or check any model on Hugging Face.
Popular open models
Memory to serve each model, with the smallest GPU setup that holds it and the lowest price on Arrakis right now. The most downloaded first.
| Model | Parameters | 16-bit | 4-bit |
|---|---|---|---|
| Qwen3-14BQwen | 15B | 35 GB: one L40S, RTX 6000 Ada or RTX A6000, L40S from $0.44 per GPU-hour | 8.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-4BQwen | 4.0B | 9.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| gemma-4-31B-itgoogle | 31B | 75 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-8BQwen | 8.2B | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.8-27BQwen | 28B | 67 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 17 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.5-4BQwen | 4.7B | 11 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-VL-8B-InstructQwen | 8.8B | 21 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 5.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.5-9BQwen | 9.7B | 23 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 5.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| GLM-5.3-Flashzai-org | 321B | 771 GB: 4 × B300, from $18.40 per hour for all 4 | 193 GB: one B300, from $4.60 per GPU-hour |
| gemma-4-26B-A4B-itgoogle | 26B | 62 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 15 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-V3.2deepseek-ai | 685B | 1,645 GB: 8 × B300, from $36.80 per hour for all 8 | 411 GB: 2 × B300, from $9.20 per hour for all 2 |
| gpt-oss-20bopenai | 21B | 50 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 13 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-VL-7B-InstructQwen | 8.3B | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 5.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-7B-InstructQwen | 7.6B | 18 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.6-35B-A3BQwen | 36B | 86 GB: one RTX PRO 6000, from $1.90 per GPU-hour | 22 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Llama-3.1-8B-Instructmeta-llama | 8.0B | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-1.5B-InstructQwen | 1.5B | 3.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-VL-4B-InstructQwen | 4.4B | 11 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-1.7BQwen | 2.0B | 4.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.2 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.8-Flash-NextQwen | 180B | 432 GB: 2 × B300, from $9.20 per hour for all 2 | 108 GB: one H200, from $3.44 per GPU-hour |
| Unlimited-OCRbaidu | 3.3B | 8.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.5-2BQwen | 2.3B | 5.5 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-V4-Flash-0731deepseek-ai | 304B | 730 GB: 4 × B300, from $18.40 per hour for all 4 | 183 GB: one B300, from $4.60 per GPU-hour |
| Qwen2.5-3B-InstructQwen | 3.1B | 7.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-32BQwen | 33B | 79 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| gpt-oss-120bopenai | 117B | 280 GB: 2 × H200, from $6.88 per hour for all 2 | 70 GB: one H100 or A100, H100 from $2.07 per GPU-hour |
| Qwen3-4B-Instruct-2507Qwen | 4.0B | 9.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| GLM-5.3zai-org | 753B | 1,808 GB: 8 × B300, from $36.80 per hour for all 8 | 452 GB: 2 × B300, from $9.20 per hour for all 2 |
| Llama-3.2-1B-Instructmeta-llama | 1.2B | 3.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.6-27BQwen | 28B | 67 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 17 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-OCRdeepseek-ai | 3.3B | 8.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-V4.1-Flashdeepseek-ai | 763B | 1,832 GB: 8 × B300, from $36.80 per hour for all 8 | 458 GB: 2 × B300, from $9.20 per hour for all 2 |
| Qwen2.5-Coder-14B-InstructQwen | 15B | 35 GB: one L40S, RTX 6000 Ada or RTX A6000, L40S from $0.44 per GPU-hour | 8.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| NVIDIA-Nemotron-3-Nano-4B-BF16nvidia | 4.0B | 9.5 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.5-27BQwen | 28B | 67 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 17 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| NVIDIA-Nemotron-3-Nano-30B-A3B-BF16nvidia | 32B | 76 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Mistral-7B-Instruct-v0.2mistralai | 7.2B | 17 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.5-35B-A3BQwen | 36B | 86 GB: one RTX PRO 6000, from $1.90 per GPU-hour | 22 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-R1deepseek-ai | 684B | 1,643 GB: 8 × B300, from $36.80 per hour for all 8 | 411 GB: 2 × B300, from $9.20 per hour for all 2 |
| DeepSeek-V3deepseek-ai | 684B | 1,643 GB: 8 × B300, from $36.80 per hour for all 8 | 411 GB: 2 × B300, from $9.20 per hour for all 2 |
| GLM-5zai-org | 754B | 1,809 GB: 8 × B300, from $36.80 per hour for all 8 | 452 GB: 2 × B300, from $9.20 per hour for all 2 |
| Qwen2.5-VL-3B-InstructQwen | 3.8B | 9.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-Coder-7B-InstructQwen | 7.6B | 18 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-VL-2B-InstructQwen | 2.1B | 5.1 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16nvidia | 561B | 1,345 GB: 8 × B200, from $50.64 per hour for all 8 | 336 GB: 2 × B200, from $12.66 per hour for all 2 |
| MiniMax-M2.5MiniMaxAI | 229B | 549 GB: 4 × H200, from $13.76 per hour for all 4 | 137 GB: one H200, from $3.44 per GPU-hour |
| Hy3tencent | 299B | 717 GB: 4 × B200, from $25.32 per hour for all 4 | 179 GB: one B200, from $6.33 per GPU-hour |
| GLM-4.5-Airzai-org | 110B | 265 GB: 2 × H200, from $6.88 per hour for all 2 | 66 GB: one H100 or A100, H100 from $2.07 per GPU-hour |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16nvidia | 32B | 76 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia | 124B | 297 GB: 4 × H100, from $8.28 per hour for all 4 | 74 GB: one H100 or A100, H100 from $2.07 per GPU-hour |
| MiniCPM5-2Bopenbmb | 2.5B | 6.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.5 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-30B-A3B-Instruct-2507Qwen | 31B | 73 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 18 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Llama-3.2-3B-Instructmeta-llama | 3.2B | 7.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-32B-InstructQwen | 33B | 79 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2-VL-2B-InstructQwen | 2.2B | 5.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3.5-122B-A10BQwen | 125B | 300 GB: 4 × H100, from $8.28 per hour for all 4 | 75 GB: one H100 or A100, H100 from $2.07 per GPU-hour |
| Qwen3-30B-A3BQwen | 31B | 73 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 18 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| GLM-5.1zai-org | 754B | 1,809 GB: 8 × B300, from $36.80 per hour for all 8 | 452 GB: 2 × B300, from $9.20 per hour for all 2 |
| DeepSeek-V4-Flashdeepseek-ai | 291B | 698 GB: 4 × B200, from $25.32 per hour for all 4 | 175 GB: one B200, from $6.33 per GPU-hour |
| Qwen3.5-397B-A17BQwen | 403B | 968 GB: 4 × B300, from $18.40 per hour for all 4 | 242 GB: one B300, from $4.60 per GPU-hour |
| GLM-4.7-Flashzai-org | 31B | 75 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| gemma-3-4b-itgoogle | 4.3B | 10 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-Coder-30B-A3B-InstructQwen | 31B | 73 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 18 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-VL-32B-InstructQwen | 33B | 80 GB: one RTX PRO 6000, from $1.90 per GPU-hour | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Kimi-K3moonshotai | 2.8T | 6,672 GB: more than one 8-GPU machine | 1,668 GB: 8 × B300, from $36.80 per hour for all 8 |
| Qwen2.5-14B-InstructQwen | 15B | 35 GB: one L40S, RTX 6000 Ada or RTX A6000, L40S from $0.44 per GPU-hour | 8.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-R1-Distill-Qwen-1.5Bdeepseek-ai | 1.8B | 4.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.1 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| HunyuanOCRtencent | 1.1B | 2.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| InternVL2-1BOpenGVLab | 0.9B | 2.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| gemma-3-1b-itgoogle | 1.0B | 2.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| SmolLM3-3BHuggingFaceTB | 3.1B | 7.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-1.5BQwen | 1.5B | 3.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| diffusiongemma-26B-A4B-itgoogle | 26B | 62 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 15 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| InternVL3_5-GPT-OSS-20B-A4B-Preview-HFOpenGVLab | 21B | 51 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 13 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Cosmos-Reason2-2Bnvidia | 2.4B | 5.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.5 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Llama-3.1-8Bmeta-llama | 8.0B | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2-VL-7B-InstructQwen | 8.3B | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 5.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| granite-4.1-3bibm-granite | 3.4B | 8.2 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-4B-Thinking-2507Qwen | 4.0B | 9.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-7BQwen | 7.6B | 18 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Llama-3.2-1Bmeta-llama | 1.2B | 3.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-Coder-32B-InstructQwen | 33B | 79 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-OCR-2deepseek-ai | 3.4B | 8.1 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Hermes-3-Llama-3.1-8BNousResearch | 8.0B | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Mistral-7B-v0.1mistralai | 7.2B | 17 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-Coder-1.5B-InstructQwen | 1.5B | 3.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-Coder-3B-InstructQwen | 3.1B | 7.4 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-Coder-NextQwen | 80B | 191 GB: one B300, from $4.60 per GPU-hour | 48 GB: one L40S, RTX 6000 Ada or RTX A6000, L40S from $0.44 per GPU-hour |
| Qwen3-VL-30B-A3B-InstructQwen | 31B | 75 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Kimi-K2.5moonshotai | 1.0T | 2,465 GB: more than one 8-GPU machine | 616 GB: 8 × H100, from $16.56 per hour for all 8 |
| medgemma-4b-itgoogle | 4.3B | 10 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-Coder-7BQwen | 7.6B | 18 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Apertus-8B-Instruct-2509swiss-ai | 8.1B | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Phi-3.5-mini-instructmicrosoft | 3.8B | 9.2 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| LocateAnything-3Bnvidia | 3.8B | 9.2 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| phi-4microsoft | 15B | 35 GB: one L40S, RTX 6000 Ada or RTX A6000, L40S from $0.44 per GPU-hour | 8.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen2.5-Coder-1.5BQwen | 1.5B | 3.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Llama-3.3-70B-Instructmeta-llama | 71B | 169 GB: one B200, from $6.33 per GPU-hour | 42 GB: one L40S, RTX 6000 Ada or RTX A6000, L40S from $0.44 per GPU-hour |
| DeepSeek-V4-Prodeepseek-ai | 1.6T | 3,837 GB: more than one 8-GPU machine | 959 GB: 4 × B300, from $18.40 per hour for all 4 |
| MiniCPM-V-4.7-35B-A3Bopenbmb | 35B | 85 GB: one RTX PRO 6000, from $1.90 per GPU-hour | 21 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| MiniMax-M2.7MiniMaxAI | 229B | 549 GB: 4 × H200, from $13.76 per hour for all 4 | 137 GB: one H200, from $3.44 per GPU-hour |
| Phi-3.5-vision-instructmicrosoft | 4.1B | 10.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.5 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-R1-Distill-Qwen-32Bdeepseek-ai | 33B | 79 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-R1-0528-Qwen3-8Bdeepseek-ai | 8.2B | 20 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.9 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Qwen3-235B-A22BQwen | 235B | 564 GB: 8 × H100, from $16.56 per hour for all 8 | 141 GB: one B200, from $6.33 per GPU-hour |
| Phi-4-mini-instructmicrosoft | 3.8B | 9.2 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| gemma-3-12b-itgoogle | 12B | 29 GB: one RTX 5090, from $0.58 per GPU-hour | 7.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| gemma-3-27b-itgoogle | 27B | 66 GB: one H100 or A100, H100 from $2.07 per GPU-hour | 16 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| phi-2microsoft | 2.8B | 6.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Llama-2-7b-hfmeta-llama | 6.7B | 16 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.0 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| granite-3.3-2b-instructibm-granite | 2.5B | 6.1 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 1.5 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Phi-3-mini-4k-instructmicrosoft | 3.8B | 9.2 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| EXAONE-3.5-7.8B-InstructLGAI-EXAONE | 7.8B | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Kimi-K2-Instructmoonshotai | 1.0T | 2,463 GB: more than one 8-GPU machine | 616 GB: 8 × H100, from $16.56 per hour for all 8 |
| DeepSeek-R1-Distill-Qwen-7Bdeepseek-ai | 7.6B | 18 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| Meta-Llama-3-8B-Instructmeta-llama | 8.0B | 19 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.8 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| gemma-3-4b-ptgoogle | 4.3B | 10 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 2.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| deepseek-coder-7b-instruct-v1.5deepseek-ai | 6.9B | 17 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 4.1 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| MiniCPM5-1Bopenbmb | 1.1B | 2.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour | 0.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour |
| DeepSeek-V3-0324deepseek-ai | 685B | 1,643 GB: 8 × B300, from $36.80 per hour for all 8 | 411 GB: 2 × B300, from $9.20 per hour for all 2 |
Questions
Fine-tuning needs more memory than serving: see the GPU memory guide.
How do you work out the memory a model needs?
Its parameter count, from its files on Hugging Face, times the bytes each parameter takes: 2 in 16-bit, 1 in 8-bit and half a byte in 4-bit. Serving adds about 20% for the KV cache and activations; long contexts and large batches need more. The GPU memory guide has the rules, and the memory fine-tuning takes.
Why does a mixture-of-experts model need memory for all its parameters?
Each token uses only a few experts, but any token can need any expert, so all of them stay in GPU memory. Fewer active parameters make it faster, not smaller.
Can I check a model that is not listed here?
Yes. Paste any model's name or link from Hugging Face into the check at the top of this page. Gated models work too: their parameter count is public.
How do I deploy a model on Arrakis?
Open Models in the console, paste the model's name, choose vLLM, SGLang or llama.cpp and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine.