# gizai/llm/state

the data of a model and of its engine: their records, the constants of their shapes, the pure reads over them and the reset of a slot. The modules that run an engine (kv, gemm, delta, gpu, host, flash, generate, engine) and the module that loads a model (model) import it. It imports none of them.

```gizlang
import * as State from "gizai/llm/state";
```

## Types

### `Engine`

```gizlang
type Engine = { model: Model; slots: i64; context: i64; rows: i64; lengths: Vec<i64>; long: Vec<bool>; kv: KvPlan; host: HostState; device: DeviceState; draft: Drafting; queue: Queue; prefix: PrefixCache; vocabulary: Vocabulary; flash: Flash; stages: Vec<Engine>; exact: bool; fuse: Vec<string>; lora: EngineLora; block?: BlockState; }
```

The engine of a model: `slots` sequences of at most `context` positions, steps of at most `rows` rows.

- `model`: `M70.Model`
- `slots`: `i64`
- `context`: `i64`
- `rows`: `i64`
- `lengths`: `Vec<i64>`
- `long`: `Vec<bool>`
- `kv`: `M65.KvPlan`
- `host`: `M70.HostState`
- `device`: `M70.DeviceState`
- `draft`: `M70.Drafting`
- `queue`: `M70.Queue`
- `prefix`: `M70.PrefixCache`
- `vocabulary`: `_gizlang_lib_ai__llm__grammar.Vocabulary` — The tokens' bytes for grammars (Tokenizer.tokenBytes, sorted), read at the first request with a grammar.
- `flash`: `M70.Flash`
- `stages`: `Vec<M70.Engine>`
- `exact`: `bool` — Linears give x to the GEMMs as the F32 the model computes (rounded to F16 there): no gemm16Kernel, whose 16-bit
weights take x in their own type (BF16 for BF16 weights, as other servers' activations), and NV4 and F8 weights
run as GEMVs, not GEMMs of x in NVFP4 or FP8. Off: the faster.
- `fuse`: `Vec<string>` — The fusions the engine takes (EngineOptions.fuse, FUSIONS): `conv`, the delta rule's convolution within its chunks'
terms (deltaPrepKernel), and `nv`, gate_up's activations as the down GEMM's NVFP4 input (gemmNvKernel's `nv`).
Neither was faster on the RTX 5090 at 2048 rows, so both are off by default.
- `lora`: `M70.EngineLora` — The LoRAs it serves (Engine.adapt): a request names one (Request.adapter).
- `block`: `_gizlang_sdk_core.Option<M70.BlockState>`

### `BlockState`

```gizlang
type BlockState = { ring: Buffer; rope: Buffer; meta: Buffer; parts: Buffer; mixed: Buffer; from: Vec<i64>; high: Vec<i64>; }
```

A block drafter's state (Drafter): per slot its context keys and values of each drafter layer at the last positions
(`ring`, F16: slot, layer, position modulo ringPositions, then the kv heads' keys and their values), the rotary table of
the rows of a step (`rope`), a draft pass's tokens, sequences and rows (`meta`, blockMeta), its attention's partial
results (`parts`) and the inputs of its linears (`mixed`). Per slot the first position its ring holds for the sequence
there (`from`) and the end of the positions written to it (`high`): a draft reads positions from both on.

- `ring`: `M7.Buffer`
- `rope`: `M7.Buffer`
- `meta`: `M7.Buffer`
- `parts`: `M7.Buffer`
- `mixed`: `M7.Buffer`
- `from`: `Vec<i64>`
- `high`: `Vec<i64>`

### `HostState`

```gizlang
type HostState = { keys: Vec<Vec<f32>>; values: Vec<Vec<f32>>; convs: Vec<Vec<f32>>; states: Vec<Vec<f32>>; replay: Vec<Vec<f32>>; hidden: Vec<f32>; }
```

An engine's state on the host: per slot and pool layer the keys and values (position, kv head, dimension; as kvStore
keeps them, see kvRounded), per slot and recurrent layer the convolution's last inputs and the delta rule's state,
and its last step's drafts' convolution inputs and delta rule terms (cpuReplay); the last hidden states that the MTP
layer reads.

- `keys`: `Vec<Vec<f32>>`
- `values`: `Vec<Vec<f32>>`
- `convs`: `Vec<Vec<f32>>`
- `states`: `Vec<Vec<f32>>`
- `replay`: `Vec<Vec<f32>>`
- `hidden`: `Vec<f32>`

### `DeviceState`

```gizlang
type DeviceState = { work: Vec<Buffer>; pages: Vec<Vec<i64>>; table: Vec<f32>; free: Vec<i64>; maxPages: i64; seen: Vec<string>; shapes: Vec<string>; recordings: Vec<G.Recording>; seeing: i64; slack: i64; placed: i64; held: Ref<Vec<Buffer>>; taps: Ref<Vec<Buffer>>; deep: Ref<Vec<Buffer>>; }
```

An engine's state on a GPU: the work buffers (W_*), each slot's pages, the page table and the free pages; the step
shapes launched once, the recordings of those launched again, the bytes a picture's pass takes (Vision.peak), those
the driver may keep after passes (`slack`) and the buffer of both, held between passes (pictureRoom; a cell: the
copies of a served engine share it, and a copy that kept its own Vec kept the buffer, 11/6).

- `work`: `Vec<M7.Buffer>`
- `pages`: `Vec<Vec<i64>>`
- `table`: `Vec<f32>`
- `free`: `Vec<i64>`
- `maxPages`: `i64`
- `seen`: `Vec<string>`
- `shapes`: `Vec<string>`
- `recordings`: `Vec<M7.Recording>`
- `seeing`: `i64`
- `slack`: `i64`
- `placed`: `i64`
- `held`: `Ref<Vec<M7.Buffer>>`
- `taps`: `Ref<Vec<M7.Buffer>>` — The buffer of the hidden states a step keeps (Engine.hidden), of the engine's largest step: made at the
first such step and held while the engine lives, so an encoder's prompts ask the card for no other.
- `deep`: `Ref<Vec<M7.Buffer>>` — The buffers of a picture's rows for the deeper layers in such a step (Visual.deep), each of the largest step:
held as `taps`.

### `Drafting`

```gizlang
type Drafting = { depth: i64; pending: Vec<i64>; bases: Vec<i64>; drafts: i64; kept: i64; draftBlocks: Vec<i64>; blockEntry: Vec<i64>; nextEntry: i64; steps: Vec<f64>; reach: Vec<f64>; ahead: Vec<Vec<Ahead>>; level: i64; forced: i64; lookup: i64; recall: i64; }
```

Drafts and pacing: the most drafts a sequence's step reads (0: none; see Engine.step), per slot its drafts of the
last step not accepted yet and their first draft row (draftBase), and the drafts generate read and kept. The head blocks a draft reads (DRAFT_SHARE; empty:
all; on a GPU also in W_DRAFT_BLOCKS), per head block its entry there or -1, and the entry of the second half the
next new block takes. Per context class c (paceClass), count n of generating sequences and count i of
draftLevels(depth), the ns a step took (a moving average; 0: not measured yet) at
steps[(c * (slots + 1) + n) * levels + i]. Per depth i, the share of drafts right through draft i (reach[i], a
moving average; -1: none judged yet), and per slot its drafts not judged yet (Ahead). The drafts of the last step
and the count warming forces (-1: none; draftLevel). The tokens a lookup and a recalled draft match before them
(EngineOptions.lookup, EngineOptions.recall).

- `depth`: `i64`
- `pending`: `Vec<i64>`
- `bases`: `Vec<i64>`
- `drafts`: `i64`
- `kept`: `i64`
- `draftBlocks`: `Vec<i64>`
- `blockEntry`: `Vec<i64>`
- `nextEntry`: `i64`
- `steps`: `Vec<f64>`
- `reach`: `Vec<f64>`
- `ahead`: `Vec<Vec<M70.Ahead>>`
- `level`: `i64`
- `forced`: `i64`
- `lookup`: `i64`
- `recall`: `i64`

### `Ahead`

```gizlang
type Ahead = { at: i64; tokens: Vec<i64>; }
```

Drafts a source made after a sequence's next token, the first at index `at` of its held tokens: all it made, also
those the step did not read, judged against the tokens the sequence holds later (LlmGenerate.judged).

- `at`: `i64`
- `tokens`: `Vec<i64>`

### `Queue`

```gizlang
type Queue = { waiting: Vec<Turn>; turns: Vec<Turn>; next: Vec<i64>; guesses: Vec<Vec<Guess>>; held: Vec<Vec<i64>>; freed: Vec<i64>; saved: Vec<Saved>; grams: Vec<Map<i64, i64>>; chains: Vec<Vec<i64>>; ids: i64; steps: i64; made: i64; read: i64; }
```

The requests in turn (Engine.add): those waiting for a slot, and per slot the one it serves (id -1: none), the token
it reads next (-1: none yet, it reads its prompt) and its drafts of the tokens after that. Per slot the tokens its
keys, values and states hold (a free slot keeps them for a prompt that starts with them), the step it was last freed
at, the recurrent states it saved at a prompt's mark, and the index of its tokens' pairs (Queue.grams: the place after
the latest of each pair; chains[i]: the place after the pair before place i, -1: none; recalled). The next request
id and step.

- `waiting`: `Vec<M70.Turn>`
- `turns`: `Vec<M70.Turn>`
- `next`: `Vec<i64>`
- `guesses`: `Vec<Vec<M70.Guess>>`
- `held`: `Vec<Vec<i64>>`
- `freed`: `Vec<i64>`
- `saved`: `Vec<M70.Saved>`
- `grams`: `Vec<_gizlang_sdk_collections.Map<i64, i64>>`
- `chains`: `Vec<Vec<i64>>`
- `ids`: `i64`
- `steps`: `i64`
- `made`: `i64`
- `read`: `i64`

### `PrefixCache`

```gizlang
type PrefixCache = { shared: Vec<Shared>; borrowed: Vec<i64>; stored: Vec<Prefix>; reuse: bool; folder: string; }
```

The prefix cache: the starts many prompts share (Shared), per slot how many of its first KV pages are a shared
start's, and the starts of chats that left their slots, on the host or, past the host memory free, in files of
`folder` (see Prefix; "?" until first used, "" for none). With `reuse` off each prompt reads all of its tokens.

- `shared`: `Vec<M70.Shared>`
- `borrowed`: `Vec<i64>`
- `stored`: `Vec<M70.Prefix>`
- `reuse`: `bool`
- `folder`: `string`

### `Rows`

```gizlang
type Rows = { tokens: Vec<i64>; slots: Vec<i64>; positions: Vec<i64>; ropes: Vec<i64>; grid: Vec<i64>; own: i64; wanted: Vec<i64>; segments: Vec<i64>; drafts: Vec<i64>; layerInputs?: Vec<f32>; }
```

A step: per row its token, slot, position and rotary position (or three: grid, Visual.grid), the rows whose logits are wanted, per sequence its
slot, first row and row count, and per sequence its drafts (its last rows, see Engine.generate). A row sees the keys
before its position, and its own when `own` is 1 (0: a padding row, see Engine.padding).

- `tokens`: `Vec<i64>`
- `slots`: `Vec<i64>`
- `positions`: `Vec<i64>`
- `ropes`: `Vec<i64>`
- `grid`: `Vec<i64>`
- `own`: `i64`
- `wanted`: `Vec<i64>`
- `segments`: `Vec<i64>`
- `drafts`: `Vec<i64>`
- `layerInputs`: `_gizlang_sdk_core.Option<Vec<f32>>`

### `ExpertCache`

```gizlang
type ExpertCache = { slots: i64; hits: i64; misses: i64; readMs: f64; }
```

The expert cache of an engine: its slots, the experts found in it and read into it, and the ms the reads took.

- `slots`: `i64`
- `hits`: `i64`
- `misses`: `i64`
- `readMs`: `f64`

### `Guess`

```gizlang
type Guess = { token: i64; q: Dist; }
```

A draft per sequence: its token and the distribution it was drawn from (all on the token when it is the most likely).

- `token`: `i64`
- `q`: `M72.Dist`

### `Progress`

```gizlang
type Progress = { id: i64; tokens: Vec<i64>; cached: i64; end: string; logprobs: Vec<Logprob>; }
```

What a step gave a request (Engine.advance): its new tokens (without a stop token), the tokens at the start of its
prompt that its slot held already (not read again), and why it ended: "stop" (a stop token), "length" (its limit, the
context's end, or a KV cache too small for it alone) or "" (it goes on).

- `id`: `i64`
- `tokens`: `Vec<i64>`
- `cached`: `i64`
- `end`: `string`
- `logprobs`: `Vec<M72.Logprob>`

### `Turn`

```gizlang
type Turn = { id: i64; request: Request; random: Vec<u64>; made: i64; mark: i64; head: i64; cached: i64; long: bool; since: i64; reading?: Recognizer; }
```

A request in an engine's turn: its id, its random state, the tokens it gave, the prompt position after which its
slot saves the recurrent states (Engine.add), the position after which its prompt's start
is shared (Engine.add; 0: none), the prompt tokens its slot held already, its grammar's recognizer while the
grammar holds (Sampling), whether its sequence is long (stretches), and the ns (Task.nanos) it began to wait, or
to run once it took a slot (admit).

- `id`: `i64`
- `request`: `M72.Request`
- `random`: `Vec<u64>`
- `made`: `i64`
- `mark`: `i64`
- `head`: `i64`
- `cached`: `i64`
- `long`: `bool`
- `since`: `i64`
- `reading`: `_gizlang_sdk_core.Option<_gizlang_lib_ai__llm__grammar.Recognizer>`

### `Saved`

```gizlang
type Saved = { at: i64; conv: Vec<f32>; state: Vec<f32>; pages: Vec<i64>; }
```

A slot's recurrent states after its first `at` tokens (0: none): the convolutions' last inputs and the delta rules'
states. On a GPU they are in KV pages of the pool (`pages`, savedPages: a copy on the device, not over the bus), or
on the host where the pool had no room or the start went to the host (Prefix); on the host `conv` and `state` (from
a GPU the states' BF16, two to a float). One holder owns the pages: a copy moves them or copies them on the device.

- `at`: `i64`
- `conv`: `Vec<f32>`
- `state`: `Vec<f32>`
- `pages`: `Vec<i64>`

### `Prefix`

```gizlang
type Prefix = { tokens: Vec<i64>; long: bool; saved: Saved; pages: Vec<bytes>; keys: Vec<Vec<f32>>; values: Vec<Vec<f32>>; used: i64; stages: Vec<Prefix>; file: string; ple: Vec<f32>; }
```

The start of a chat that left its slot for another (see admit): its tokens, the recurrent states after them, and
their keys and values (on a GPU the bytes of their KV pages, on the host per pool layer); `used` is the step it was
stored at. A later prompt that starts with the tokens takes them back into a slot and reads only the rest. On a
model in stages, `stages` holds each later stage's pages and states (Engine.stages). A long sequence's start (Turn.long)
is taken back by a long one only: their keys turn by another table.

- `tokens`: `Vec<i64>`
- `long`: `bool`
- `saved`: `M70.Saved`
- `pages`: `Vec<bytes>`
- `keys`: `Vec<Vec<f32>>`
- `values`: `Vec<Vec<f32>>`
- `used`: `i64`
- `stages`: `Vec<M70.Prefix>`
- `file`: `string`
- `ple`: `Vec<f32>`

### `Shared`

```gizlang
type Shared = { tokens: Vec<i64>; long: bool; saved: Saved; pages: Vec<i64>; keys: Vec<Vec<f32>>; values: Vec<Vec<f32>>; used: i64; stages: Vec<Shared>; }
```

A start that many prompts share (as the system turn and tools of an agent's chats, Model.chatTokens), or that a chat
left in the pool when a new request took its slot (vacate): its tokens (whole KV pages), the recurrent states after
them, and their keys and values: on a GPU the KV pages of the pool that hold them, which the slots that start with
them read in place (Engine.borrowed), on the host a copy per pool layer; `used` is the step a slot last took it. On
a model in stages, `stages` holds each later stage's pages and states.

- `tokens`: `Vec<i64>`
- `long`: `bool`
- `saved`: `M70.Saved`
- `pages`: `Vec<i64>`
- `keys`: `Vec<Vec<f32>>`
- `values`: `Vec<Vec<f32>>`
- `used`: `i64`
- `stages`: `Vec<M70.Shared>`

### `Linear`

```gizlang
type Linear = { n: i64; k: i64; format: i64; codes: bytes; scales: bytes; book: Vec<f32>; turned: bool; }
```

`turned`: the linear reads its inputs unrotated in a rotated model (K formats), so x is turned back first.

- `n`: `i64`
- `k`: `i64`
- `format`: `i64`
- `codes`: `bytes`
- `scales`: `bytes`
- `book`: `Vec<f32>`
- `turned`: `bool`

### `EngineLora`

```gizlang
type EngineLora = { names: Vec<string>; at: Vec<i64>; ranks: Vec<i64>; a: Vec<Buffer>; b: Vec<Buffer>; rows: Vec<Buffer>; low: Vec<Buffer>; slot: Vec<i64>; made: Vec<i64>; }
```

The LoRAs an engine serves on a GPU, many at once (Engine.adapt): `names` by index; for each device linear (by its
index, DeviceLinear.id; `at`) its rank R (the most of any LoRA's; others padded with zeros) and the LoRAs' A [LoRAs, R, k]
(each row rotated as the linear's inputs are) and B [LoRAs, n, R], scales in B, F32. Each step row's LoRA (-1: none)
is in `rows` (I32), from each slot's (`slot`), each wanted row's after Engine.rows (the head's), and a linear's x A in
`low` [rows, the largest R]. A slot's keys and
values were made under `made[slot]`: another LoRA does not reuse them (admit).

- `names`: `Vec<string>`
- `at`: `Vec<i64>`
- `ranks`: `Vec<i64>`
- `a`: `Vec<M7.Buffer>`
- `b`: `Vec<M7.Buffer>`
- `rows`: `Vec<M7.Buffer>`
- `low`: `Vec<M7.Buffer>`
- `slot`: `Vec<i64>`
- `made`: `Vec<i64>`

### `DeviceLinear`

```gizlang
type DeviceLinear = { n: i64; k: i64; format: i64; codes: Buffer; scales: Buffer; book: Buffer; turned: bool; maps: Vec<Buffer>; id: i64; }
```

On a GPU, a K format's book holds its subs' bytes (see K formats); `maps` are the copy engine's maps of an
NV4 weight's codes and scales for gemmNvKernel (nvMaps), none for other formats.

- `n`: `i64`
- `k`: `i64`
- `format`: `i64`
- `codes`: `M7.Buffer`
- `scales`: `M7.Buffer`
- `book`: `M7.Buffer`
- `turned`: `bool`
- `maps`: `Vec<M7.Buffer>`
- `id`: `i64`

### `Layer`

```gizlang
type Layer = { kind: i64; at: i64; wide: bool; source: i64; layerInput: Vec<i64>; window: i64; rope: i64; n1: i64; n2: i64; n3: i64; n4: i64; scalar: i64; qkv: i64; rest: i64; o: i64; qn: i64; kn: i64; bias: i64; input: i64; out: i64; conv: i64; decay: i64; dt: i64; gn: i64; up: i64; down: i64; ah: Vec<i64>; mh: Vec<i64>; router: i64; ple: Vec<i64>; }
```

A layer's tensors: indexes into the model's linears (qkv, o, input, out, up, down) and small tensors (the
others), -1 where the layer has none. `at` is its index among the layers of its kind (an attention layer's: its pool
layer, Engine.kv), `wide` whether its heads are Arch.wideAt's.

- `kind`: `i64`
- `at`: `i64`
- `wide`: `bool`
- `source`: `i64`
- `layerInput`: `Vec<i64>`
- `window`: `i64`
- `rope`: `i64`
- `n1`: `i64`
- `n2`: `i64`
- `n3`: `i64`
- `n4`: `i64`
- `scalar`: `i64`
- `qkv`: `i64`
- `rest`: `i64`
- `o`: `i64`
- `qn`: `i64`
- `kn`: `i64`
- `bias`: `i64`
- `input`: `i64`
- `out`: `i64`
- `conv`: `i64`
- `decay`: `i64`
- `dt`: `i64`
- `gn`: `i64`
- `up`: `i64`
- `down`: `i64`
- `ah`: `Vec<i64>`
- `mh`: `Vec<i64>`
- `router`: `i64`
- `ple`: `Vec<i64>`

### `Mtp`

```gizlang
type Mtp = { layer: Layer; fc: i64; ne: i64; nh: i64; norm: i64; }
```

The MTP layer (Qwen3.5, DeepSeek V3): a token's embedding and the main model's last hidden state at the position
before it, each normed (ne, nh), joined by the fc linear, then one attention layer (its KV pages after the main
attention layers') and its norm before the shared head: the most likely token after that token.

- `layer`: `M70.Layer`
- `fc`: `i64`
- `ne`: `i64`
- `nh`: `i64`
- `norm`: `i64`

### `Drafter`

```gizlang
type Drafter = { shape: DraftShape; layers: Vec<Layer>; convs: Vec<Vec<i64>>; context: i64; fc: Vec<i64>; hn: i64; norm: i64; selector: i64; codebooks: Vec<bytes>; }
```

A block drafter on the model's GPU (DraftShape, draftParts), its tensors in Model.device and Model.small: per layer
its Layer and its convolutions' base kernels and projections (before the attention, before the MLP: [ab, ap, mb, mp];
empty without them), the linear of every layer's context keys and values, the linear of each tap, the norms of the
context and the output, and the selector's projection (-1: none) with its codebooks in host memory (BF16 rows of
shape.rank: the token before, the candidate).

- `shape`: `_gizlang_lib_ai__llm__shape.DraftShape`
- `layers`: `Vec<M70.Layer>`
- `convs`: `Vec<Vec<i64>>`
- `context`: `i64`
- `fc`: `Vec<i64>`
- `hn`: `i64`
- `norm`: `i64`
- `selector`: `i64`
- `codebooks`: `Vec<bytes>`

### `FlashFiles`

```gizlang
type FlashFiles = { fd: i64; at: Vec<i64>; record: i64; bits: i64; books: Vec<Buffer>; rows: Vec<Stored>; mult: Vec<u64>; sizes: Vec<i64>; offsets: Vec<i64>; }
```

A Flash model's own files: the experts' (its descriptor, the file offset of each layer's records, a record's
bytes and its two codebooks on the GPU, gate and up, then down) and the PLE's n-gram rows (BF16; none without a
PLE) with their hash multipliers, each head's vocabulary size and first row.

- `fd`: `i64`
- `at`: `Vec<i64>`
- `record`: `i64`
- `bits`: `i64`
- `books`: `Vec<M7.Buffer>`
- `rows`: `Vec<gizlang_safetensors.Stored>`
- `mult`: `Vec<u64>`
- `sizes`: `Vec<i64>`
- `offsets`: `Vec<i64>`

### `Flash`

```gizlang
type Flash = { bufs: Vec<Buffer>; chunks: Ref<Vec<Buffer>>; perChunk: i64; most: i64; slotOf: Vec<i64>; owner: Vec<i64>; used: Vec<bool>; busy: Vec<i64>; hand: i64; stamp: i64; hits: i64; misses: i64; readNs: i64; history: Vec<i64>; convs: Vec<Vec<f32>>; last: Vec<Vec<i64>>; ahead: i64; }
```

Flash's state on a GPU (Engine.flash): its work buffers (F_*), the expert cache and the PLE's states. The cache
holds an expert in each slot (expertBytes) in blocks (`chunks`) of `perChunk` slots: slots 0 to layers - 1 the
layers' shared experts for good, the others the experts the routers picked last (CLOCK: a slot read since the hand
passed it last stays once more). Per expert (layer (experts + 1) + index) its slot or -1; per slot its expert or -1,
whether it was read and the stamp of the layer that reads it (not evicted then). The hits and misses of the cache
and the ns its reads took. Per slot of the engine the last two tokens (the latest first) and the PLE convolution's
last inputs.

- `bufs`: `Vec<M7.Buffer>`
- `chunks`: `Ref<Vec<M7.Buffer>>`
- `perChunk`: `i64`
- `most`: `i64`
- `slotOf`: `Vec<i64>`
- `owner`: `Vec<i64>`
- `used`: `Vec<bool>`
- `busy`: `Vec<i64>`
- `hand`: `i64`
- `stamp`: `i64`
- `hits`: `i64`
- `misses`: `i64`
- `readNs`: `i64`
- `history`: `Vec<i64>`
- `convs`: `Vec<Vec<f32>>`
- `last`: `Vec<Vec<i64>>`
- `ahead`: `i64`

### `Thought`

```gizlang
type Thought = { open: string; close: string; }
```

How a model's reply marks its thinking: `open` is the text it writes to start a thought ("": the prompt opens it,
as ChatML's <think>), `close` the text that ends it ("": the model does not think).

- `open`: `string`
- `close`: `string`

### `Chat`

```gizlang
type Chat = { template: Template; names: Vec<string>; values: Vec<Value>; xhigh: bool; }
```

A model's chat template (chat_template.jinja or tokenizer_config.json's chat_template) as Hugging Face renders it,
the special tokens it may name (`names`, their `values`: bos_token, eos_token), and whether its top reasoning effort
is called xhigh (Qwen 3.8), where a chat's "high" goes.

- `template`: `gizlang_jinja.Template`
- `names`: `Vec<string>`
- `values`: `Vec<gizlang_jinja.Value>`
- `xhigh`: `bool`

### `Model`

```gizlang
type Model = { arch: Arch; folder: string; tokenizer: Tokenizer; stops: Vec<i64>; chat?: Chat; thought: Thought; call: string; sampling: Sampling; embed: Stored; layers: Vec<Layer>; norm: i64; head: i64; mtp: Option<Mtp>; mixer: Vec<i64>; drafter?: Drafter; linears: Vec<Linear>; small: Vec<Vec<f32>>; gpu: Option<Gpu>; device: Vec<DeviceLinear>; deviceSmall: Vec<Buffer>; signs: Vec<Buffer>; sources: Vec<Vec<string>>; embedTable?: Mapped; layerInput?: { embed: Stored; projection: i64; norm: i64; }; files: FlashFiles; vision?: Vision; weights?: Stack; spots: Vec<Vec<Spot>>; beside: i64; stages: Vec<Model>; kvCosts?: string; npu?: NpuLlm.Model; }
```

A converted model, on the GPU that gizhw/gpu opens (GIZLANG_DEVICE) when there is one (`device`), else on the
host (`linears`); the small tensors are on both. The embedding stays in the file, and where the GPU of the first layer
reads it (embedTable, Gpu.mapFile): a step there gathers its tokens' rows on the GPU, the host reads them from the file.
`chat` its chat template when it has one (Chat),
`thought` how its reply marks its thinking (Thought),
`call` the mark where its tool calls start (gizai/llm/model CALL, PYTHON_CALL; "": it writes none); `stops` are the tokens that end
a generation and `sampling` the model's own settings. `mtp` is the MTP layer on a GPU when the model has one, `vision`
its vision tower when its folder has one (convertVision), `stages` the models of its later layers on other GPUs
(loadAcross).

- `arch`: `_gizlang_lib_ai__llm__shape.Arch`
- `folder`: `string`
- `tokenizer`: `M26.Tokenizer`
- `stops`: `Vec<i64>`
- `chat`: `_gizlang_sdk_core.Option<M70.Chat>`
- `thought`: `M70.Thought`
- `call`: `string`
- `sampling`: `M72.Sampling`
- `embed`: `gizlang_safetensors.Stored`
- `layers`: `Vec<M70.Layer>`
- `norm`: `i64`
- `head`: `i64`
- `mtp`: `_gizlang_sdk_core.Option<M70.Mtp>`
- `mixer`: `Vec<i64>`
- `drafter`: `_gizlang_sdk_core.Option<M70.Drafter>`
- `linears`: `Vec<M70.Linear>`
- `small`: `Vec<Vec<f32>>`
- `gpu`: `_gizlang_sdk_core.Option<M7.Gpu>`
- `device`: `Vec<M70.DeviceLinear>`
- `deviceSmall`: `Vec<M7.Buffer>`
- `signs`: `Vec<M7.Buffer>`
- `sources`: `Vec<Vec<string>>`
- `embedTable`: `_gizlang_sdk_core.Option<M7.Mapped>`
- `layerInput`: `_gizlang_sdk_core.Option<{ embed: gizlang_safetensors.Stored; projection: i64; norm: i64; }>`
- `files`: `M70.FlashFiles`
- `vision`: `_gizlang_sdk_core.Option<_gizlang_lib_ai__llm__vision.Vision>`
- `weights`: `_gizlang_sdk_core.Option<_gizlang_lib_ai__parts__stack.Stack>`
- `spots`: `Vec<Vec<M70.Spot>>`
- `beside`: `i64`
- `stages`: `Vec<M70.Model>`
- `kvCosts`: `_gizlang_sdk_core.Option<string>`
- `npu`: `_gizlang_sdk_core.Option<_gizlang_lib_ai__llm__npu__gemma.Model>`

### `Spot`

```gizlang
type Spot = { linear: i64; codes: i64; scales: i64; book: i64; }
```

A linear of a layer's group (Model.weights): its place among the model's device linears, and which of the
group's tensors are its codes, its scales and its book.

- `linear`: `i64`
- `codes`: `i64`
- `scales`: `i64`
- `book`: `i64`

## Functions

### `deltaTerms`

```gizlang
function deltaTerms(a: Arch): i64
```

Floats of a draft row's delta rule terms per value head: its normed key, value, decay and beta.

Effects: fail.

### `descRows`

```gizlang
function descRows(rows: i64): i64
```

The descriptor of a step (W_DESC, f32) for `rows` rows: per row its slot and position, per sequence its slot, first
row, row count and drafts (its last rows, see Engine.generate), per row the rotary cosines and sines, the wanted rows,
then the tiles (first row and rows, see tileKernel), the local rotary table and the rows' tokens (embedKernel).

### `descSegments`

```gizlang
function descSegments(rows: i64): i64
```

Where the per-sequence part of the descriptor starts.

Effects: fail.

### `descRope`

```gizlang
function descRope(rows: i64): i64
```

Where the rows' rotary cosines and sines start.

Effects: fail.

### `descWanted`

```gizlang
function descWanted(a: Arch, rows: i64): i64
```

Where the wanted rows start.

Effects: fail.

### `descTiles`

```gizlang
function descTiles(a: Arch, rows: i64): i64
```

Where the tiles start.

Effects: fail.

### `descLocal`

```gizlang
function descLocal(a: Arch, rows: i64): i64
```

The local layers' rotary table (Arch.local), after the tiles.

Effects: fail.

### `descTokens`

```gizlang
function descTokens(a: Arch, rows: i64): i64
```

Where the rows' tokens start (embedKernel).

Effects: fail.

### `descSize`

```gizlang
function descSize(a: Arch, rows: i64): i64
```

The floats of the whole descriptor.

Effects: fail.

### `addressAt`

```gizlang
function addressAt(b: Buffer, floats: i64): u64
```

The address FLOATS floats into B.

Effects: fail.

### `partCapacity`

```gizlang
function partCapacity(rows: i64): i64
```

The parts the parts buffer holds for an engine of `rows` rows.

Effects: fail.

### `partFloats`

```gizlang
function partFloats(headDim: i64): i64
```

The floats of a split's partial result of one row and query head (attention's parts, the drafter's too): its weighted
values (headDim), their largest score and the sum of their weights.

Effects: fail.

### `stretches`

```gizlang
function stretches(e: Engine, prompt: i64, limit: i64): bool
```

Whether a request of `prompt` tokens that writes at most `limit` is long: the model has a long table (Arch.long)
and the request may pass Arch.longFrom. The others turn by the plain table (logits as within the trained context);
one that passes longFrom all the same goes on with it.

Effects: fail.

### `attentionLayers`

```gizlang
function attentionLayers(a: Arch): i64
```

The attention layers of `a`.

Effects: fail.

### `recurrentLayers`

```gizlang
function recurrentLayers(a: Arch): i64
```

The recurrent (non-attention) layers of `a`.

Effects: fail.

### `drafting`

```gizlang
function drafting(e: Engine): bool
```

Whether generate drafts with the model's MTP layer (on a model in stages, the last stage's: headOf).

Effects: fail.

### `looksUp`

```gizlang
function looksUp(e: Engine): bool
```

Whether generate drafts by lookup (a model without an MTP layer or block drafter; not on an NPU, which reads a token
a step).

Effects: fail.

### `mtpOn`

```gizlang
function mtpOn(e: Engine): bool
```

Whether this engine's own model runs the MTP layer: its buffers and KV pages hold it.

Effects: fail.

### `poolLayers`

```gizlang
function poolLayers(e: Engine): i64
```

Attention layers in the KV pages: the model's, and its MTP layer's when generate drafts with it.

Effects: fail.

### `pageBytes`

```gizlang
function pageBytes(e: Engine): i64
```

The bytes of a KV page: keys and values of PAGE positions in each pool layer (Engine.kv).

### `poolWides`

```gizlang
function poolWides(e: Engine): Vec<bool>
```

Per pool layer (Layer.at): whether its heads are the wide shape (Arch.wideView); the MTP layer's last.

Effects: fail, alloc.

### `kvGroup`

```gizlang
function kvGroup(e: Engine, at: i64): KvGroup
```

The group of the KV pages that keeps the keys and values pool layer `at` (Layer.source) reads.

Effects: fail.

### `hostWidth`

```gizlang
function hostWidth(e: Engine, l: i64): i64
```

Floats of a position's keys (or values) in pool layer l (Layer.at) on the host.

Effects: fail.

### `widestInput`

```gizlang
function widestInput(a: Arch): i64
```

Floats of the widest input of a linear and of the widest output.

Effects: fail.

### `widestOutput`

```gizlang
function widestOutput(a: Arch): i64
```

Floats of the widest output of a linear, rounded up to 16.

Effects: fail, alloc.

### `attentionWidth`

```gizlang
function attentionWidth(a: Arch): i64
```

Floats of the widest attention output of a row: heads times the widest head.

Effects: fail.

### `wantedRows`

```gizlang
function wantedRows(e: Engine): i64
```

The most rows of a step whose logits a GPU engine gives: 1 + LEVELS a sequence, and the recalled drafts of one sequence
beyond those (Engine.advance recalls for one a step): 1 + depth a sequence would hold 0.5 GB of logits at 8 slots.

Effects: fail.

### `draftRows`

```gizlang
function draftRows(e: Engine): i64
```

The draft rows each recurrent layer of W_DRAFT_CONV and W_DRAFT_DELTA holds (draftBase): a step's drafts are among its
wanted rows.

Effects: fail.

### `headOf`

```gizlang
function headOf(e: Engine): Engine
```

The engine whose model has the head: the last stage's (Engine.stages), else e.

Effects: fail.

### `noTurn`

```gizlang
function noTurn(): Turn
```

A turn of no request: the turn of an empty slot.

Effects: fail, alloc.

### `noSaved`

```gizlang
function noSaved(): Saved
```

No saved states.

### `noLora`

```gizlang
function noLora(slots: i64): EngineLora
```

An engine of `slots` slots that serves no LoRA.

Effects: fail, alloc.

### `noFiles`

```gizlang
function noFiles(): FlashFiles
```

FlashFiles of no file.

Effects: fail.

### `noFlash`

```gizlang
function noFlash(): Flash
```

The Flash state of an engine without Flash.

Effects: fail, write, alloc.

### `noThought`

```gizlang
function noThought(): Thought
```

The Thought of a model that does not think.

### `interleaved`

```gizlang
function interleaved(states: Vec<Vec<f32>>, rows: i64, dim: i64): Vec<f32>
```

Per row the rows of each of `states` (rows of dim floats each), in turn.

Effects: fail, alloc.

### `embedded`

```gizlang
function embedded(e: Engine, tokens: Vec<i64>): Vec<f32> throws string
```

The embedding rows of the tokens, read from the model file.

Effects: fail, read, write, console, io, alloc.

### `ropeTable`

```gizlang
function ropeTable(a: Arch, positions: Vec<i64>, grid: Vec<i64> = [], table: i64 = GLOBAL, long: Vec<bool> = []): Vec<f32>
```

The rotary cosines and sines of each row's position in rotary table `table` (GLOBAL, LOCAL): rotary/2 pairs a row,
on the global table of the long frequencies times Arch.longScale for a row of a long sequence (`long`, Rows.long),
else times Arch.ropeScale. With a `grid` (three positions a row: time, height, width) frequency j takes height's where j % 3 is 1 and width's where it is 2, within 3
times their sections (a.sections), time's elsewhere: Qwen3-VL's interleaved multimodal rotary positions.

Effects: fail, alloc.

### `pleKept`

```gizlang
function pleKept(): i64
```

Positions of the PLE convolution's inputs a slot keeps: (taps - 1) times the dilation.

Effects: fail.

### `blocksOn`

```gizlang
function blocksOn(e: Engine): bool
```

Whether this engine drafts with its model's block drafter (Model.drafter): on one GPU.

### `ringPositions`

```gizlang
function ringPositions(d: DraftShape, depth: i64): i64
```

The positions a slot's ring holds for each drafter layer on an engine of `depth` drafts: the window, and the most
rows a step writes past a sequence's next token (a block's later rows, or recalled drafts), so the rows a step
rejects land where the window no longer reads.

Effects: fail.

## Methods

### `Engine.reset`

```gizlang
function Engine.reset(self: mut Engine, slot: i64): void
```

Empties a slot for a new sequence.

Effects: fail, read, write, console, io, alloc.

## Constants

### `W_X`

```gizlang
const W_X: i64 = 0
```

The rows' hidden states (rows by dim).

### `W_Y`

```gizlang
const W_Y: i64 = 1
```

A product's input, normed (rows by the widest input).

### `W_P`

```gizlang
const W_P: i64 = 2
```

A product's output (rows by the widest output).

### `W_Q`

```gizlang
const W_Q: i64 = 3
```

The rows' queries and attention's output (rows by the attention width).

### `W_C`

```gizlang
const W_C: i64 = 4
```

The convolution's inputs and outputs (rows by convDim).

### `W_PARTS`

```gizlang
const W_PARTS: i64 = 5
```

Attention's partial results of each split (partCapacity).

### `W_DESC`

```gizlang
const W_DESC: i64 = 6
```

The step's descriptor (descSize).

### `W_MAP`

```gizlang
const W_MAP: i64 = 7
```

The slot of each row.

### `W_TABLE`

```gizlang
const W_TABLE: i64 = 8
```

The page table: each slot's KV pages.

### `W_POOL`

```gizlang
const W_POOL: i64 = 9
```

The KV pool: the pages of every slot's keys and values.

### `W_CONV`

```gizlang
const W_CONV: i64 = 10
```

Each slot's last convolution inputs.

### `W_STATE`

```gizlang
const W_STATE: i64 = 11
```

Each slot's delta rule states, BF16 two to a float.

### `W_SCRATCH`

```gizlang
const W_SCRATCH: i64 = 12
```

The partial sums of a split product, or the wanted rows' logits.

### `W_HID`

```gizlang
const W_HID: i64 = 13
```

The last hidden states of a step's rows, normed, for the MTP layer.

### `W_EMB`

```gizlang
const W_EMB: i64 = 14
```

The embeddings of the MTP layer's tokens.

### `W_SRC`

```gizlang
const W_SRC: i64 = 15
```

The step rows whose hidden states the MTP layer reads.

### `W_LOGITS`

```gizlang
const W_LOGITS: i64 = 16
```

The wanted rows' logits.

### `W_TOP`

```gizlang
const W_TOP: i64 = 17
```

The wanted rows' most likely tokens.

### `W_DRAFT_CONV`

```gizlang
const W_DRAFT_CONV: i64 = 18
```

Each slot's convolution inputs of its drafts in the last step (Engine.generate).

### `W_DRAFT_DELTA`

```gizlang
const W_DRAFT_DELTA: i64 = 19
```

Each slot's delta rule terms of its drafts in the last step (deltaTerms).

### `W_DRAFT_BLOCKS`

```gizlang
const W_DRAFT_BLOCKS: i64 = 20
```

The head blocks a draft reads (Engine.draftBlocks).

### `W_CAND`

```gizlang
const W_CAND: i64 = 21
```

The wanted rows' sampling candidates (candidatesKernel).

### `W_X8`

```gizlang
const W_X8: i64 = 22
```

x of an FP8 product (fp8Kernel): its rows' scales (big floats), then its E4M3 values, 4 to a float; or x turned
back from its rotation for a K format (plainKernel).

### `W_LAYER_INPUT`

```gizlang
const W_LAYER_INPUT: i64 = 23
```

The rows' per-layer inputs (rows by layers by layerInput), after their projection.

### `W_LAYER_EMBED`

```gizlang
const W_LAYER_EMBED: i64 = 24
```

The rows' per-layer embeddings (rows by layers by layerInput), as the host looked them up.

### `SPLITS`

```gizlang
const SPLITS: i64 = 16
```

Splits of the keys that attention reads at once: a step by tiles takes partSplits; attendKernel (fewer rows) more, up to
two waves of its workgroups on the GPU (Gpu.slots; ATTEND_GROUPS where the driver does not say) within the parts
buffer (partCapacity), each split of at least SPAN keys. (One row's 4 kv heads in 16 splits kept 64 workgroups on 76
multiprocessors: 60K keys took 0.8 ms a layer, nine times what the memory takes to read them.)

### `SPAN`

```gizlang
const SPAN: i64 = 64
```

The fewest keys of a split.

### `ATTEND_GROUPS`

```gizlang
const ATTEND_GROUPS: i64 = 512
```

The workgroups of attention's two waves where the driver does not say (Gpu.slots).

### `LOG2E`

```gizlang
const LOG2E = Math.LOG2E
```

log2 e: softmax weights are 2^(x log2 e) on the GPU's exp2 (its exp polynomial cost some 25 instructions a weight).

### `GEMV_ROWS`

```gizlang
const GEMV_ROWS: i64 = 8
```

The most rows a GPU linear reads its weights for at once on CUDA cores; from GEMM_ROWS rows a linear runs on
matrix cores where the GPU has them.

### `GEMM_ROWS`

```gizlang
const GEMM_ROWS: i64 = 32
```

The fewest rows a linear runs on matrix cores for.

### `PACE_CLASSES`

```gizlang
const PACE_CLASSES: i64 = 4
```

The classes of context length Drafting.steps keeps apart (paceClass).

### `DRAFTS`

```gizlang
const DRAFTS: i64 = 32
```

The drafts an engine reads per sequence and step by default: recalled drafts (LlmGenerate.recalled) take all of
them; those of the MTP layer or lookup at most LEVELS, as the n-th is kept less often than the one before
and each costs a row (and with an MTP layer a step of it and the head).

### `LEVELS`

```gizlang
const LEVELS: i64 = 8
```

The most drafts an MTP layer or lookup makes a step (LlmGenerate.draftLevels).

### `DRAFT_SHARE`

```gizlang
const DRAFT_SHARE: i64 = 8
```

A draft reads one DRAFT_SHARE-th of the head's blocks of 16 rows (as FR-Spec, arXiv 2502.14856, which takes the
frequent tokens of a corpus): the first half of them always (a BPE vocabulary's first ids are its frequent tokens),
the other half the last blocks of the tokens the engine read, so the language of a prompt comes in with it.

### `MTP_WINDOW`

```gizlang
const MTP_WINDOW: i64 = 32768
```

The last keys the MTP layer attends to: its drafts lean on the near text, and the model checks every one, so the
output is the same; at 252K a draft pass reads an eighth of the keys.

### `RECORDINGS`

```gizlang
const RECORDINGS: i64 = 64
```

The most step shapes an engine keeps recorded.

### `RECORD_ROOM`

```gizlang
const RECORD_ROOM: i64 = 50_331_648
```

The device memory free before a new recording at least (one of q27-4's steps took 6 MiB).

### `PLE_TAPS`

```gizlang
const PLE_TAPS: i64 = 4
```

The taps of the PLE convolution.

### `PLE_DILATION`

```gizlang
const PLE_DILATION: i64 = 3
```

The dilation of the PLE convolution.

### `SPLIT_ROOM`

```gizlang
const SPLIT_ROOM: i64 = 1_073_741_824
```

The device memory a model's weights leave free on its GPU at least, for an engine's work buffers and KV pages: a
model that leaves less takes the GPU's other GPUs too (loadAcross), or keeps layers in host memory (Model.weights).

