Sharded download

Run models bigger than any one machine. For a model like Llama 3.3 70B (~42 GB) split across three nodes, each node downloads only the layers it will serve — dppan pull fetches just those byte ranges and writes a small, valid GGUF shard; dppan join --gguf serves it. No node ever needs the full file.

How it works

A GGUF file starts with a tensor directory that maps every tensor to its exact byte range. dppan pull reads that index (a few MB), selects the tensors for your layer range, range-fetches only those bytes, and reconstructs a smaller GGUF that the engine loads like any other model. A shard also records its own provenance — source digest and layer range — so join knows what's inside.

Three-node example (Llama 3.3 70B)

# ── Node 0: orchestrator + worker, layers 0–26 (~16 GB instead of 42) ──
dppan pull llama3.3:70b --layers 0-26 --orch
DPPAN_GGUF_LLAMA3_3_70B=~/.dppan/models/llama3.3-70b-layers0-26-orch.gguf \
  PORT=8001 dppan-orchestrator &
dppan join --orchestrator http://localhost:8001 --model llama3.3:70b \
           --gguf ~/.dppan/models/llama3.3-70b-layers0-26-orch.gguf

# ── Node 1: worker, layers 27–53 (~13 GB) ──
dppan pull llama3.3:70b --layers 27-53
dppan join --orchestrator http://node0:8001 --model llama3.3:70b \
           --gguf ~/.dppan/models/llama3.3-70b-layers27-53.gguf

# ── Node 2: worker, layers 54–79 (~13 GB) ──
dppan pull llama3.3:70b --layers 54-79
dppan join --orchestrator http://node0:8001 --model llama3.3:70b \
           --gguf ~/.dppan/models/llama3.3-70b-layers54-79.gguf

Then chat as usual — the OpenAI-compatible endpoint on node 0 routes tokens through the layer chain automatically.

Good to know

• --layers is inclusive on both ends (0-26 = 27 layers) — the same convention everywhere it appears (dppan pull, dppan join, and layer ranges shown in the dashboard and dppan status).
• A shard remembers its own range, so join --gguf needs no --layers — the orchestrator assigns the node exactly the slice it has on disk. If a node is ever assigned layers its file doesn't contain, it refuses loudly with the exact dppan pull command to fix it.
• --orch additionally includes the shared tensors (embeddings, output norm, LM head) needed by the machine running dppan-orchestrator. Exactly one shard needs it. Orch-only shards (no --layers) work too, for a coordinator-only machine. If the source publishes a projector sidecar, this pull also downloads and validates it automatically.
• Projectors belong only to the orchestrator host. They are stored under ~/.dppan/models/projectors/<canonical-model-id>/ and are never included in worker shards or sent to worker nodes. Each projector has an adjacent .dppan.json record containing its exact source identity, digest, size, and inspected modalities; the orchestrator uses that record rather than guessing compatibility from its filename.
• The orchestrator finds its GGUF via the DPPAN_GGUF_<MODEL> env var — the model id uppercased with non-alphanumerics as _ (llama3.3:70b → DPPAN_GGUF_LLAMA3_3_70B) — or via gguf_path in config/models.toml. Without either, it falls back to the --gguf path of the first node that joins, which only works when that node runs on the same machine and joins with the --orch shard (the embed engine needs token_embd.weight; a layers-only shard fails with "token_embd.weight not found").
• If the full model is already in ~/.ollama, pull slices it locally with zero network traffic.
• Shards default to ~/.dppan/models/ and show up in dppan models with their layer range.

dppan pull reference

dppan pull <MODEL> [OPTIONS]

Flag	Default	Description
<MODEL>	required	llama3.3:70b (Ollama) · hf:owner/repo[:QUANT] · inclavate:slug
--layers	all	Layer range to download, inclusive (e.g. 0-26)
--orch	off	Include orchestrator tensors and auto-download a published projector
--projector-only	off	Download only the orchestrator projector; write no decoder shard
--projector <FILE>	auto	Select an exact sidecar; automatic preference is F16, then BF16
--projector-out <PATH>	model store	Override the sidecar output path for an --orch pull
--out	~/.dppan/models/…	Output shard path
--verify	off	Check every tensor's sha256 against the integrity manifest
--manifest	auto	Manifest file or URL for --verify
--remote	off	Range-fetch even if the full blob exists locally

Sources and quant selection (including split -0000N-of-… HF quants, which are merged automatically) are covered on the Models page.

For a manual sidecar-only pull:

dppan pull hf:owner/repo:Q4_K_M --projector-only
dppan pull hf:owner/repo:Q4_K_M --projector-only --projector mmproj-BF16.gguf

The first command uses the automatic projector preference. The second selects a published variant explicitly. Both verify the source digest and inspect the completed GGUF with libmtmd before publishing it into the model store. If --projector-out places the sidecar outside the default store, set projector_path on that model's config/models.toml entry so the orchestrator can associate it explicitly.

Picking a split

Divide the model's layer count by your node count, weighted by VRAM. For 80 layers on three similar machines: 0-26, 27-53, 54-79. The node running the orchestrator does extra work (embeddings + LM head), so give it the smaller slice if the machines differ. If you omit --layers hints entirely and let the orchestrator assign, size the pulls after joining once — or just start with the automatic flow on a model that fits locally.