Deploy a model

In the console

1. Open Models in the console.
2. Name the model as Hugging Face does, such as Qwen/Qwen3-8B, or paste its link.
3. Pick the engine: vLLM or SGLang for text models, vLLM-Omni or SGLang for image and video models, vLLM-Omni for speech and sound, llama.cpp for a GGUF file. For llama.cpp, add the quantization after a colon, such as unsloth/Qwen3-8B-GGUF:Q4_K_M.
4. Paste your own engine options if you have them (see below).
5. For a gated or private model, add a Hugging Face token with read access.
6. Find GPUs reads the model's size from Hugging Face, shows the start line the machine will run, and lists every GPU the model fits on now, with the price. Pick one and rent it.

The engine downloads the model when the machine starts, so the endpoint answers a few minutes later. Models shows when it is ready, its address, its API key, code to call it, a try for chat and image models, and the engine's log.

Your engine options

Paste the options you tuned, in whatever shape you keep them:

VLLM_USE_DEEP_GEMM=1 vllm serve Qwen/Qwen3-8B \
  --max-model-len 32768 --enable-prefix-caching \
  --speculative-config '{"method": "eagle3", "num_speculative_tokens": 3}'

Bare flags, the whole command and a docker run line all work, with variables in front and JSON values in quotes. You choose them and keep them current with your engine's releases. Arrakis reads them without a shell and hands them to the engine word for word, so:

• Shell syntax (;, |, &&, $(...), $VARIABLES) is refused: nothing would run it.
• Where the engine listens (--host, --port), its key (--api-key) and the model stay Arrakis's, so the endpoint keeps its address and key. --served-model-name (--alias for llama.cpp) renames the model requests use.
• Your parallelism wins: with -tp, --tp or -dp in your options, Arrakis adds none of its own.
• Settings that choose GPUs (NVIDIA_*) and loader settings (LD_*) are refused: a machine uses the GPUs it rents.
• Your Hugging Face token goes in its own field, never in the options, so it isn't saved with them.

From the API

curl -X POST $ARRAKIS_URL/v1/deploy \
  -H "Authorization: Bearer ark_..." \
  -H "Content-Type: application/json" \
  -d '{"model": "black-forest-labs/FLUX.1-schnell", "engine": "omni", "termination_hours": 4,
       "extra_args": "--enforce-eager"}'

Without node_id, Arrakis rents the best-ranked machine that holds the model, on as few GPUs as it allows. POST /v1/deploy/plan lists every GPU it fits on first, for free, and with extra_args it checks your options and returns the start line. The answer carries the endpoint and its api_key; GET /v1/instances/{id}/endpoint gives them again, with ready once the model answers, and GET /v1/instances/{id}/endpoint/log the end of the engine's log. Agents use the deploy_model tool over MCP.

Call it

The endpoint speaks the OpenAI API, so any OpenAI client works. The model's name in a request is its Hugging Face id, without the quantization, unless you renamed it.

from openai import OpenAI

client = OpenAI(base_url="http://<address>/v1", api_key="sk-...")
reply = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)

The machine is yours

• It is an ordinary rental: billed by the hour, removed at its removal time, and listed under Instances.
• It boots an Arrakis image built on the engine's official release, with root SSH. Run arrakis-serve status, arrakis-serve logs or arrakis-serve restart on it.
• To try other options without a new machine, edit /etc/arrakis/serve/command (one word per line) or /etc/arrakis/serve/env, then run arrakis-serve restart. When the engine stops, the machine stays up so you can fix it.
• The endpoint is the machine's address over plain HTTP. A gate on port 8000 checks its API key (Authorization: Bearer <key>) before a request reaches the engine, which listens only inside the machine; /health answers without the key and says only whether the engine is up. For private traffic, open an SSH tunnel (ssh root@<host> -p <port> -N -L 8000:localhost:8000) and use http://localhost:8000/v1.
• Your Hugging Face token goes to the machine as HF_TOKEN, and Arrakis doesn't keep it. The machine's host can read what runs on it, so use a fine-grained, read-only token.