FIRST PRINCIPLES · WHY A FILE FORMAT MATTERS
THE PHYSICS
What a model is made of, why it is hard to move, and why the file format — of all things — is where the battle is actually fought.
Why does a model file format decide serving speed?
Because decode is bandwidth-bound: every generated token re-reads the working set, so tokens per second is effectively memory bandwidth divided by bytes touched per token. The format decides those bytes — which representation is read, which experts are resident, what precision each role carries. That is why representation is a serving lever worth keeping adjustable, and why VINDEX3 stores variants beside the original instead of sealing one choice at conversion.
TENSORS
Open a model and there is no code inside. There are tensors: rectangular grids of numbers, learned during training. Everything the model knows — every fact, every habit of speech — is held in how those numbers relate.
6,144 × 2,048 — one tensor, of thousands
LAYERS
The tensors are organised into layers, and the layers into a stack. To produce one word, your text becomes a vector and passes through every layer in order — each one reads it, consults its tensors, and adds its contribution back. Dozens of layers, for every single token.
layer 0 → layer 1 → layer 2 → layer 3 → layer 4 → layer 5 — one token — every layer, in order, every time.
SIZE
Now the arithmetic. Twenty billion parameters at two bytes each is forty gigabytes. The largest mixture-of-experts models reach trillions. A model ships as a stack of multi-gigabyte files — and what those files look like inside stops being a detail.
a frontier mixture of experts — ≈1.5 TB
to scale by bytes — the song is still there, two pixels wide
To one scale: a song (4 MB) · a feature film (5 GB) · a 20-billion-parameter model (40 GB) · a frontier mixture of experts (≈1.5 TB) — each drawn area-true to its magnitude.
The problem is not storing the numbers. It is moving them.
THE WALL
To produce one token, every participating weight must travel from memory to the compute unit — and that trip has a speed limit measured in gigabytes per second. Divide the bytes a token needs by the bandwidth and you have the byte floor: the fastest that token can possibly arrive, before any computation counts.
Decode speed is a bytes problem before it is a compute problem.
memory → compute, through the channel — a fixed few hundred GB/s. 16-bit weights — 1,959 MB must cross per token. 4.25-bit weights — 1,269 MB per token, same channel, tokens arrive sooner.
WHY QUANTIZE
If moving bytes is the wall, make the bytes smaller. Quantization stores each number in fewer bits, which means fewer distinct values a number is allowed to take — dial sixteen bits down to four and tens of thousands of possible levels become sixteen. Every weight snaps to its nearest allowed level, and every bit shaved is bandwidth returned: in one measured case, 1,959 MB per token became 1,269. But snapped numbers are changed numbers, and changed numbers can change answers.
4 bits — sixteen levels; every value moves to the nearest one
small moves, compounded through every layer, can change an answer — which is why nothing ships unmeasured
the numbers as trained — every value exact. 16 bits — tens of thousands of levels; the error is invisible. 4 bits — sixteen levels; every value moves to the nearest one. small moves, compounded through every layer, can change an answer — which is why nothing ships unmeasured.
THE FORMAT'S ANSWER
VINDEX3 refuses to let precision be a rumour. Every encoding is a named, physically present variant beside the original — selected, never converted. Every variant carries a fidelity level that is derived, not claimed. And the container re-verifies against its source, hash by hash, forever. Make the bytes smaller. Never lose track of what you did.
ENGINES
A GPU is thousands of small workers moving in lockstep; a CPU is a few fast ones. At decode time, both are mostly waiting on the same thing: bytes. So the format's real job is to make the bytes addressable — mapped straight into memory, read in place, no unpacking. A VINDEX3 container decodes on both engines, and the measured outputs are identical, token for token. The engine is a choice. The bytes are not.
Models grew by fanning out — and most of the model now sits idle on every token.
4 / 64 UNITS ACTIVE — “THE CAPITAL OF FRANCE”
A mixture-of-experts layer holds many expert tensors and routes each token to a handful. The fan-out is how models got enormous without every token paying for all of it — and it is exactly what a single sealed file cannot serve, because you should not have to load hundreds of experts to use four.
Across 3 scenarios shown here, only 4 of 64 units activate on average — which ones depends on the input.
THE FORMAT'S ANSWER
VINDEX3 stores experts as addressable banks, grouped into extents sized for how disks and kernels actually read, split across files when a layer outgrows what one file should be. A profile decides what stays resident, what pages in on demand, and what never loads at all — which is how a model larger than your memory still decodes. The routing is the model's. The residency is yours.
Which weights load. What precision they use. Where they sit. Which engine runs them. Every lever is a choice about bytes on disk.
THE EVOLUTION
Earlier formats made those choices once, at conversion, and sealed them into the file — choose a precision, lose the original; choose a layout, lose the alternatives. VINDEX3 keeps every choice open for the life of the artifact: every part named, every representation present and selectable, every claim checkable against the source.
The file stopped being a snapshot of one decision. It became the space of all of them.
Every lever is a choice about bytes on disk. But before the file that keeps those choices open, meet the machinery the bytes actually are — what a gate, a query, an expert really do. Then the container.
PUBLISHED 29 AUG 2026 · REVISED 30 AUG 2026 · VERSION 3.0 Candidate
CITEsite build 43a1b6b · built 2026-09-02
CITE THIS
Web page · 3.0 Candidate
Hay, C. (2026). The Physics. In VINDEX3 Specification (Version 3.0 Candidate). VINDEX3. https://vindex3.org/why
A chapter of the specification, citable on its own. Cite the version rather than the site — the canonical reference, and how to use it, live at /cite.