# gizai/llm/kvformat

One owner for KV page packing and quantization layout, independent of execution and scheduling.

```gizlang
import * as Kvformat from "gizai/llm/kvformat";
```

## Types

### `KvFormat`

```gizlang
type KvFormat = { keys: i64; values: i64; }
```

How an engine keeps keys and values: the code of a key and of a value. 16: F16 values; 8, 4 or 2: codes of the
head rotated by blocks of kvBlock, at 2^bits even steps (kvStore); NV4 (NVFP4): E2M1 codes of the rotated head, which
the FP4 matrix cores take with no decoding. NV4 keys have an E4M3 scale a block of 16 dims and an F32 scale a head;
NV4 values (with NV4 keys only) an F32 scale a head, their codes kept by dims (KvPart.columns).

- `keys`: `i64`
- `values`: `i64`

### `KvPart`

```gizlang
type KvPart = { bits: i64; word: i64; each: i64; at: i64; page: i64; steps: i64; floats: i64; nv: bool; scales: i64; columns: bool; }
```

Where a KV page keeps one kind (keys or values) of its heads, each pool layer, kv head and position (kvHead): F16
values, or 32 lanes of codes, lane l the codes of values l hd/32 on; `each` words of type `word` a lane, from word
`at` of the page (`page` words of that type). With codes each head's step and least value (F32) from float `steps`.
With `nv` (NV4) a head's F32 scale is its first float there. NV4 keys: the E4M3 block scales hd/64 words (4 a word)
a head from float `scales`, and a head's 16-byte chunk c at chunk c ^ (position % (hd / 32)) of its row (kvLane), so
the rows a matrix load reads together meet distinct banks. NV4 values (`columns`): a page's PAGE heads of a pool layer
and kv head by dims, dim d in words 2 d and 2 d + 1 (I32), position p's code in nibble p % 8 of word p / 8: the B
tiles of the FP4 products with the weights (flashNvKernel).

- `bits`: `i64`
- `word`: `i64`
- `each`: `i64`
- `at`: `i64`
- `page`: `i64`
- `steps`: `i64`
- `floats`: `i64`
- `nv`: `bool`
- `scales`: `i64`
- `columns`: `bool`

### `KvLayout`

```gizlang
type KvLayout = { keys: KvPart; values: KvPart; bytes: i64; }
```

A page's keys and values of the `layers` pool layers of a's head shape (KvPart): the keys' heads, the values'
heads, their steps, then the block scales of NV4 keys; `bytes` their size. They start at byte Arch.kvFrom of a shared
page (kvViews).

- `keys`: `M65.KvPart`
- `values`: `M65.KvPart`
- `bytes`: `i64`

### `KvGroup`

```gizlang
type KvGroup = { arch: Arch; format: KvFormat; layers: i64; }
```

The pool layers of one head shape and one format: their Arch with their region of a KV page (Arch.kvFrom, kvPage)
and how many they are.

- `arch`: `_gizlang_lib_ai__llm__shape.Arch`
- `format`: `M65.KvFormat`
- `layers`: `i64`

### `KvPlace`

```gizlang
type KvPlace = { group: i64; index: i64; }
```

A pool layer's group and its index among the group's layers.

- `group`: `i64`
- `index`: `i64`

### `KvPlan`

```gizlang
type KvPlan = { formats: Vec<KvFormat>; groups: Vec<KvGroup>; places: Vec<KvPlace>; bytes: i64; }
```

How a KV page keeps each pool layer: its format (fitted, kvFitted), its group and its index there; the groups'
regions one after the other, `bytes` the page. One group (one shape, one format) takes the whole page.

- `formats`: `Vec<M65.KvFormat>`
- `groups`: `Vec<M65.KvGroup>`
- `places`: `Vec<M65.KvPlace>`
- `bytes`: `i64`

## Functions

### `kvFormats`

```gizlang
function kvFormats(): Vec<KvFormat>
```

The formats an engine chooses from for each pool layer (kvChoice), most bits first. A key's error moves all scores
of its position, a value's only its share of the sum: values lose bits first.

Effects: fail, alloc.

### `kvPrior`

```gizlang
function kvPrior(): Vec<f64>
```

A pool layer's cost in each of kvFormats where its model has no measured costs (kvCosts): the KL of Qwen3.8-27B's
next tokens with all its 16 pool layers in the format, a 16th each; nv4 is a guess between
nv4/4 and 4 until it is measured.

Effects: fail, alloc.

### `kvCodeBits`

```gizlang
function kvCodeBits(code: i64): i64
```

The bits a code takes for each value (NV4: 4, its scales apart).

### `kvAllowed`

```gizlang
function kvAllowed(codes: string): Vec<bool> throws string
```

Which kinds of codes a format list keeps (an engine's kvCodes): "all", "nv4" (no 4-bit keys at even steps: the FP4
matrix cores take every 4-bit key) or "uniform" (no NV4).

Effects: fail, alloc.

### `kvChoice`

```gizlang
function kvChoice(a: Arch, wides: Vec<bool>, costs: Vec<Vec<f64>>, room: i64, allowed: Vec<bool>): Vec<KvFormat>
```

The formats of the pool layers (head shapes `wides`, as kvPlan) of least summed cost (`costs`: a row a pool layer,
a cost for each of kvFormats) in which a page takes at most `room` bytes, each from the kvFormats that `allowed`
marks; [] where none fits. An exact knapsack: the costs of each layer's change add up (as AutoQuant KV takes them).

Effects: fail, alloc.

### `kvFormat`

```gizlang
function kvFormat(name: string): KvFormat throws string
```

The format a name gives: BITS for keys and values, or KEYS/VALUES ("4/2", "nv4/4"); a code is 16, 8, 4 or 2 bits
or nv4 (NVFP4; values only with NV4 keys); "" is the engine's choice (keys 0).

Effects: fail, alloc.

### `kvName`

```gizlang
function kvName(f: KvFormat): string
```

The name of a format, as kvFormat reads it.

Effects: fail, alloc.

### `kvFitted`

```gizlang
function kvFitted(a: Arch, f: KvFormat): KvFormat
```

Format f as the heads of a take it (kvWidth).

Effects: fail.

### `kvLayout`

```gizlang
function kvLayout(a: Arch, layers: i64, f: KvFormat): KvLayout
```

The layout of a page of `layers` pool layers of a's head shape in format f.

Effects: fail.

### `kvPlan`

```gizlang
function kvPlan(a: Arch, wides: Vec<bool>, formats: Vec<KvFormat>): KvPlan
```

The plan of pool layers of head shapes `wides` (true: Arch.wideView's) in formats `formats`, one a pool layer.

Effects: fail, alloc.

### `kvFormatsOf`

```gizlang
function kvFormatsOf(name: string, layers: i64): Vec<KvFormat> throws string
```

The formats of `layers` pool layers that a name gives: kvFormat's for all, or one a pool layer, separated by
commas (as kvAuto writes them).

Effects: fail, alloc.

### `kvCostsText`

```gizlang
function kvCostsText(costs: Vec<Vec<f64>>): string
```

The text of measured costs (a model folder's kv-costs, Model.kvCosts): a line of kvFormats' names, then a line of
costs an attention layer, in that order.

Effects: fail, alloc.

### `kvCostsOf`

```gizlang
function kvCostsOf(text: string): Vec<Vec<f64>> throws string
```

The costs that kvCostsText wrote; the formats must be kvFormats'.

Effects: fail, alloc.

### `kvPlanName`

```gizlang
function kvPlanName(p: KvPlan): string
```

The name of a plan's formats as kvFormatsOf reads it: one name when all pool layers take one format.

Effects: fail, alloc.

### `kvBits`

```gizlang
function kvBits(p: KvPlan): f64
```

The mean bits of a pool layer's key or value in plan p.

Effects: fail.

## Constants

### `PAGE`

```gizlang
const PAGE: i64 = 16
```

Positions per GPU KV page. All layout and scheduler users share this constant.

