arrakis
EN
Sign in Start renting
Menu

Arrakis / Models / InternVL2-1B

InternVL2-1B GPU requirements

InternVL2-1B has 0.9B parameters. To serve it, plan for about 2.3 GB of GPU memory in 16-bit, 1.1 GB in 8-bit or 0.6 GB in 4-bit, including about 20% for the KV cache and activations.

Deploy this model Check another model

What it takes to serve it

GPU memory to run it for inference, with the smallest GPU setup that holds it and the lowest price on Arrakis right now.

PrecisionMemory and the smallest setup
16-bit2.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour
8-bit1.1 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour
4-bit0.6 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour

What it takes to fine-tune it

QLoRA trains adapters on a 4-bit frozen model, LoRA on a 16-bit one, and full fine-tuning updates every weight. Figures are before activations, which grow with batch size and sequence length.

MethodMemory and the smallest setup
QLoRA0.7 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour
LoRA2.3 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour
Full fine-tuning15 GB: one RTX 4090 or RTX 3090, RTX 3090 from $0.19 per GPU-hour

About the model

From its card and files on Hugging Face. Memory figures here follow the GPU memory guide.

Parameters
0.9B
Published in
BF16
License
mit
Released
2024-07-08
Weights
OpenGVLab/InternVL2-1B

Deploy it on Arrakis

Open Models in the console, choose vLLM, SGLang or llama.cpp, and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine. Or run it yourself on any machine with enough GPU memory:

vLLM
vllm serve OpenGVLab/InternVL2-1B --tensor-parallel-size 1
Deploy this model How deploying works

Questions

Live prices for every GPU are on the GPU prices page.

How much GPU memory does InternVL2-1B need?
To serve it, about 2 GB in 16-bit, 1 GB in 8-bit, 1 GB in 4-bit, including about 20% for the KV cache and activations. To fine-tune it, about 1 GB with QLoRA, 2 GB with LoRA and 15 GB for full fine-tuning with Adam, before activations.
Can I run InternVL2-1B on one GPU?
Yes. In 16-bit it needs about 2.3 GB, which fits on one RTX 4090 or RTX 3090.
What does it cost to run InternVL2-1B?
In 16-bit it needs one RTX 4090 or RTX 3090; in 4-bit, one RTX 4090 or RTX 3090. Each setup on this page shows the lowest price per GPU-hour on Arrakis right now, and you pay by the hour from a prepaid balance.
How do I deploy InternVL2-1B on Arrakis?
Open Models in the Arrakis console, paste OpenGVLab/InternVL2-1B, choose vLLM, SGLang or llama.cpp, and rent a GPU it fits on. You get an OpenAI-compatible endpoint with its own API key, and root SSH to the machine.

More models

Every model, and a check for any other.