DeepSeek Offline Setup: Run V4 and R1 on Your Own Hardware

Can you actually run DeepSeek without sending a single token to a Chinese server? Yes — and a working DeepSeek offline setup is the most direct answer to the privacy and compliance questions that keep coming up about the hosted service. The current open weights, DeepSeek-V4.1-Flash-1017 and DeepSeek-V4-Pro-0813, are on Hugging Face under the MIT licence; the harder question is which model your hardware can realistically host, and which runtime to use.

This guide walks through three deployment paths: a small distilled R1 on a single consumer GPU, DeepSeek-V4.1-Flash-1017 on a multi-GPU server, and what it takes to host DeepSeek-V4-Pro-0813. The commands and hardware figures come from DeepSeek’s model cards, the vLLM Recipes pages and Ollama’s documentation, each linked where it is used.

What you’ll build

By the end of this tutorial you will have a DeepSeek model running on hardware you control, exposing an OpenAI-compatible endpoint on localhost. Your prompts will not leave the machine. You will be able to swap your existing OpenAI client over by changing two lines: base_url and model.

Three deployment tiers, by hardware budget:

• Tier 1 — Laptop / single consumer GPU. A DeepSeek R1 distill (1.5B to 32B) via Ollama.
• Tier 2 — Multi-GPU server. DeepSeek-V4.1-Flash-1017 via vLLM. The current Flash model, MIT-licensed weights, OpenAI-compatible API.
• Tier 3 — Large multi-GPU node. DeepSeek-V4-Pro-0813 via vLLM or SGLang. Not a desktop install.

If you want a comparison of the API hosted by DeepSeek vs running it yourself, see the AI comparison hub and the DeepSeek privacy writeup.

Prerequisites

• Operating system: macOS, Windows or Linux for Ollama; Linux for vLLM, which its GPU installation guide lists as the supported OS.
• Disk: 1.1–20 GB for the R1 distills in Tier 1, about 510 GB for DeepSeek-V4.1-Flash-1017 and about 893 GB for DeepSeek-V4-Pro-0813 (sum of the current files in each Hugging Face repo, checked 24 October 2026).
• Python: 3.11 to 3.14 for vLLM, per the same guide.
• GPUs: NVIDIA GPUs with compute capability 7.5 or higher for vLLM; Tier 2 and Tier 3 need data-centre GPUs (see below). Apple Silicon needs nothing extra for Ollama.
• A Hugging Face account is optional. The DeepSeek-V4.1-Flash-1017 and DeepSeek-V4-Pro-0813 repos are not gated (checked 24 October 2026), and the R1 distills can be pulled via Ollama.
• Optional but recommended: DeepSeek hardware calculator to size your GPU before you commit to a checkpoint.

Hardware reality check

The phrase “run DeepSeek locally” hides an order-of-magnitude difference between models. Pick from this table before you start downloading anything.

Two facts worth absorbing. First, the full weights have to fit in GPU memory (or be partly offloaded), so total memory dominates the cost even though these MoE models activate only a fraction of their parameters per token. DeepSeek’s own minimal-inference notes describe weight conversion, one checkpoint file per tensor-parallel rank and torchrun execution, with multi-node options — a very different operating model from a desktop app or an Ollama pull. Second, the KV cache still needs memory: the vLLM recipe puts V4.1-Flash’s global KV cache at 890 bytes per token, under 1 GB for a full 1M-token prompt plus a sliding window per layer, but advises measuring before assuming a given context and batch size fits.

Tier 1: DeepSeek R1 distill via Ollama

This is the simplest DeepSeek offline setup and the right starting point for almost everyone. According to the Ollama library page, the deepseek-r1 distill tags are small dense models fine-tuned on reasoning data generated by DeepSeek-R1: 8b is DeepSeek-R1-0528-Qwen3-8B, the 1.5B–32B tags are R1-Distill-Qwen models and 70b is R1-Distill-Llama-70B. They run on consumer hardware. The page lists the weights as MIT-licensed, but notes that the Qwen distills derive from Qwen-2.5 (originally Apache 2.0) and the Llama distills from Llama 3.1-8B-Base and Llama 3.3-70B-Instruct (originally under the Llama 3.1 and 3.3 licences).

Step 1 — Install Ollama

Run the appropriate shell command (the install scripts are from ollama.com/download; Homebrew also packages Ollama):

# Linux and macOS
curl -fsSL https://ollama.com/install.sh | sh

# macOS — or use the desktop app from ollama.com, or:
brew install ollama

# Windows (PowerShell)
irm https://ollama.com/install.ps1 | iex

Step 2 — Pull a model

Pick the largest variant that fits your VRAM with room to spare. For 12 GB cards, the 14B; for 24 GB cards, the 32B; for everything else, 8B.

ollama pull deepseek-r1:8b      # 5.2 GB on disk
ollama pull deepseek-r1:14b     # 9.0 GB on disk
ollama pull deepseek-r1:32b     # 20 GB on disk

Avoid the deepseek-v4.1-flash and deepseek-v4-pro tags for an offline setup: on Ollama they are cloud-only, so prompts sent through them leave your machine.

Step 3 — Run it

ollama run deepseek-r1:14b 'A bat and a ball cost $1.10. The bat costs $1 more than the ball. How much is the ball?'

These are thinking models: Ollama’s thinking documentation says thinking-capable models return their reasoning trace in a separate thinking field from the final answer, so you will see the reasoning before the answer (here, $0.05).

Step 4 — Expose an OpenAI-compatible endpoint

Ollama already serves one. Its OpenAI compatibility endpoint is at http://localhost:11434/v1/. Point the OpenAI SDK at this URL and use deepseek-r1:14b as the model name. The SDK requires an API key, but Ollama ignores it, so any string works.

from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama")
resp = client.chat.completions.create(
    model="deepseek-r1:14b",
    messages=[{"role": "user", "content": "Refactor this Python script."}],
)
print(resp.choices[0].message.content)

For a deeper walkthrough of the Ollama path including Modelfile customisation, see running DeepSeek on Ollama.

Tier 2: DeepSeek V4.1-Flash via vLLM

If you want the current generation rather than R1, DeepSeek-V4.1-Flash-1017 is the model to host yourself. It is a minor update to DeepSeek-V4.1-Flash, released on 17 October 2026 with weights and code under the MIT License, and it is the model the hosted API serves as deepseek-flash. According to the vLLM recipe, the checkpoint is a native mixed-precision release — MXFP4 routed-expert weights, MXFP8 block-quantized dense weights — of roughly 511 GB on disk, and it accepts images as well as text.

Step 1 — Provision the hardware

Configurations the vLLM recipe describes for V4.1-Flash (it lists about 614 GB of GPU memory as the minimum, including headroom):

• One 8-GPU NVIDIA H200 node (1,128 GB), with room for KV cache
• 4× NVIDIA H200 at tensor parallel 4, with the Engram tables offloaded to pinned host memory (the recipe’s H200 default)
• One GB200 NVL4 tray (768 GB) at tensor parallel 4

The recipe also covers B200, B300, GB300 and AMD MI355X/MI325X layouts.

Step 2 — Download the weights

pip install -U huggingface_hub
hf download deepseek-ai/DeepSeek-V4.1-Flash-1017 \
    --local-dir ./DeepSeek-V4.1-Flash-1017

This pulls the full checkpoint (about 510 GB). The repo is not gated, so hf auth login is optional. Alternatively, pass the repo ID to vllm serve and vLLM downloads the weights on first start.

Step 3 — Install vLLM

Use a fresh virtualenv. The recipe lists vLLM 0.31.0 or later as the prerequisite for V4.1-Flash:

python -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install "vllm>=0.31.0"

Step 4 — Serve the model

The command below starts an OpenAI-compatible server on port 8000 on four H200 GPUs. It is assembled from the flags the vLLM recipe documents — the required --tokenizer-mode deepseek_v41 (the repo ships no Jinja chat template), the reasoning and tool-call parsers, and the H200 Engram offload; use the recipe’s command builder for other hardware:

export VLLM_ENGINE_READY_TIMEOUT_S=3600
vllm serve ./DeepSeek-V4.1-Flash-1017 \
    --served-model-name deepseek-ai/DeepSeek-V4.1-Flash-1017 \
    --tokenizer-mode deepseek_v41 \
    --reasoning-parser deepseek_v41 \
    --tool-call-parser deepseek_v41 --enable-auto-tool-choice \
    --tensor-parallel-size 4 \
    --engram-config '{"cpu_offload":true}' \
    --port 8000

Add --language-model-only if your workload is text-only; the recipe says it skips the vision encoder and frees that memory for KV cache. For local sampling, the model card recommends temperature = 1.0, top_p = 1.0, a 1M-token context window and max_tokens of at least 256K.

Step 5 — Talk to it from Python

vLLM exposes the same Chat Completions wire format DeepSeek’s hosted API uses (POST /v1/chat/completions on your server), so the SDK swap is two lines:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")
resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4.1-Flash-1017",
    messages=[{"role": "user", "content": "Plan the migration."}],
    extra_body={"chat_template_kwargs": {"thinking": True, "reasoning_effort": 50}},
    temperature=1.0,
    top_p=1.0,
    max_tokens=262144,  # model card: at least 256K
)
print(resp.choices[0].message.content)

Local thinking controls differ from the hosted API. The recipe documents thinking on/off and a numeric reasoning_effort (1–100, or low/high/xhigh/max = 25/50/75/100) passed as chat_template_kwargs; with both unset, thinking is on at effort 50. It also notes that DeepSeek’s API maps its low/high/max to 50/75/100, so a request tuned at high on the API runs at 50 locally unless you pass an integer. The reasoning parser returns the trace separately from the final content. Other parameters worth knowing: temperature, top_p and max_tokens; tool calling works through the tool-call parser enabled above.

Step 6 — Verify it worked

Three quick checks:

1. curl http://localhost:8000/v1/models should return a JSON list including your model ID.
2. nvidia-smi should show GPU memory allocated across all cards involved in tensor parallel.
3. A short test prompt should return non-empty content. If it comes back empty with finish_reason=length, the recipe’s explanation is that thinking used up a small max_tokens budget — raise max_tokens or turn thinking off.

Tier 3: DeepSeek V4-Pro on a multi-GPU node

V4-Pro is not a desktop workload. DeepSeek-V4-Pro-0813, released on 13 August 2026, is the official release that superseded the April preview; its weights total about 893 GB. DeepSeek’s model card gives this vLLM command for a single 4× GB300 node, with DSpark speculative decoding enabled, and points to the vLLM recipe for other hardware:

vllm serve deepseek-ai/DeepSeek-V4-Pro-0813 \
  --trust-remote-code --kv-cache-dtype fp8 --block-size 256 \
  --data-parallel-size 4 --enable-expert-parallel \
  --moe-backend deep_gemm_mega_moe \
  --attention-config '{"use_fp4_indexer_cache": true}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'

The card also gives an SGLang command, and its local-deployment recommendation is temperature = 1.0, with top_p = 0.95 for agentic scenarios and 1.0 otherwise, and a maximum output length of 384K tokens for the high and max effort levels. The vLLM recipe adds --tokenizer-mode deepseek_v4, which applies the built-in DeepSeek-V4 prompt encoding (the 0813 repo ships no Jinja chat template), together with the deepseek_v4 tool-call and reasoning parsers for tool calling and separated reasoning.

The hosted API is usually the cheaper route for occasional V4-Pro work; current per-token rates, with peak and off-peak prices, are on our DeepSeek API pricing page. For most teams, “offline” means tier 1 or tier 2, with the hosted API used as a fallback for the heaviest jobs that do not touch private data.

Common errors and fixes

Local vs hosted: when to switch

Local inference removes the network and the third party from your data path. It does not remove the operations work. Some rules of thumb:

• Privacy-bound workloads (legal, healthcare, internal source code) → local. Even the smallest R1 distill running offline keeps the prompt off any third party’s servers.
• Bursty, low-volume traffic → hosted. Hardware that sits idle most of the day is poor economics.
• High-volume batch processing → run the math both ways, including hardware, power and staff time, against the hosted peak and off-peak rates.
• Mixed setup → Tier 1 locally for sensitive snippets, hosted deepseek-v4-pro or deepseek-flash for heavy jobs that don’t touch private data.

A note on the API surface

The hosted DeepSeek API behaves differently from the web app: it is stateless, so the client must resend the conversation history with every request. Your local vLLM server works the same way. As of October 2026 the hosted API serves deepseek-flash (DeepSeek-V4.1-Flash-1017) and deepseek-v4-pro (DeepSeek-V4-Pro-0813); the legacy IDs deepseek-chat and deepseek-reasoner were scheduled for retirement on 2026-07-24 at 15:59 UTC and are no longer listed in the official docs. See our DeepSeek API documentation guide for the hosted parameters.

Next steps

Once your DeepSeek offline setup is stable, the natural follow-ons are:

• install DeepSeek locally — a deeper dive into platform-specific install steps if you hit issues above.
• DeepSeek RAG tutorial — wire your local model to a vector store so it can answer questions about your private documents.
• fine-tuning DeepSeek — adapt a distill to your domain once the base setup proves out.
• DeepSeek V4-Flash — full model page covering the Flash line through V4.1-Flash-1017.