# gizai/weights

the one reader of a model's stored weights. A checkpoint is safetensors (gizlang/safetensors gives each tensor's place in its file); this module puts a stored tensor where it runs: on the host as f32, on a GPU from the file straight to the device, a linear as BF16, ComfyUI's scaled FP8, raw codes, and ConvRot's INT8 and INT4 rows in the order the products read them. A model's loader names its tensors and calls these; it holds no reading of its own. A LoRA is gizai/lora's. The formats are gizai/quant's; an LLM's conversion is gizai/llm/convert.

```gizlang
import * as Weights from "gizai/weights";
```

## Functions

### `storedOn`

```gizlang
function storedOn(s: Stored, gpu: Option<Gpu>, linear: bool): Tensor throws string
```

A stored tensor where it runs: a linear as BF16 on a GPU, anything else as f32 (BF16, F16, F32 and E4M3 bytes go
to the GPU as they are stored and are cast there).

Effects: fail, read, write, console, io, alloc.

### `storedHost`

```gizlang
function storedHost(s: Stored): Tensor throws string
```

A stored tensor on the host, as f32 values.

Effects: fail, read, write, console, io, alloc.

### `load`

```gizlang
function load(path: string, gpu: Gpu, kept: (name: string) => string = (name: string): string => name): Map<string, Tensor> throws string
```

The tensors of the file at `path` as f32 on `gpu` (storedHost), by the name `kept` gives each stored name (its
name by default; "" leaves the tensor out): the one loader of a network that keeps every weight as f32.

Effects: fail, read, write, console, io, alloc.

### `hostAt`

```gizlang
function hostAt(named: Map<string, Stored>, name: string): Tensor throws string
```

Tensor `name` of a file's tensors by name, on the host (storedHost).

Effects: fail, read, write, console, io, alloc.

### `valuesAt`

```gizlang
function valuesAt(named: Map<string, Stored>, name: string): Vec<f32> throws string
```

The values of tensor `name` of a file's tensors by name.

Effects: fail, read, write, console, io, alloc.

### `weightAt`

```gizlang
function weightAt(named: Map<string, Stored>, name: string, gpu: Option<Gpu>, linear: bool, head: i64 = 0): Tensor throws string
```

Weight `name` where it runs (as storedOn). With `head` > 0 the weight is a fused QKV projection stored as H3's
checkpoints hold it, a head's query, key and value rows together (heads, 3, head); its rows are reordered to all
queries, all keys, all values first, the order of a LoRA's rows (gizai/lora) and of this module's attention.

Effects: fail, read, write, console, io, alloc.

### `convRotLinear`

```gizlang
function convRotLinear(named: Map<string, Stored>, base: string, from: Vec<i64>, gpu: Gpu, bits: i64 = 8): Linear throws string
```

The ConvRot linear `base` of a file (`base`.weight: I8 rows rotated by the regular Hadamard of 256, or 4-bit pairs a
byte with 4 `bits`; `base`.weight_scale: a scale a row) as a Linear on `gpu`: its row r is the file's row from[r],
its inputs in Linears.int8Rows' order (canonicalInto). Every ConvRot checkpoint's linear loads here.

Effects: fail, read, write, console, io, alloc.

### `rawAt`

```gizlang
function rawAt(s: Stored, gpu: Option<Gpu>, dtype: i64): Tensor throws string
```

A stored tensor of bytes (U8 or I8 codes, or E4M3 scales) where it runs: its bytes as `dtype` on a GPU, their
values on the host (codes as unsigned bytes).

Effects: fail, read, write, console, io, alloc.

### `comfyInt8Rows`

```gizlang
function comfyInt8Rows(named: Map<string, Stored>, names: Vec<string>, gpu: Gpu): Vec<Tensor> throws string
```

Linears `names` of a ComfyUI INT8 ConvRot file (each name's .weight and .weight_scale: INT8 rows rotated by the
regular Hadamard of 256, a scale a row) stacked by rows on `gpu`, as Linears.int8Rows' products read them (canonicalInto):
[the INT8 weights [rows, in], their scales [1, rows] F32].

Effects: fail, read, write, console, io, alloc.

### `canonicalInto`

```gizlang
function canonicalInto(w: Tensor, from: Vec<i64>, out: Tensor, bits: i64 = 8): void
```

INT8 rows w ([out, in]) rotated in the regular Hadamard's own order, as ConvRot files store them, into `out`: its row
r is w's row from[r], its inputs in the order Linears.int8Rows rotates inputs to (as W4.decodeInto's codes). With 4 `bits`
w holds INT4 pairs ([out, in / 2] bytes, the even input in the low nibble: convrot_w4a4) and out is I4 [out, in].

Effects: fail, read, write, console, io, alloc.

## Methods

### `Stored.upload`

```gizlang
function Stored.upload(self: Stored, gpu: Gpu): Buffer throws string
```

The stored tensor's raw bytes in a new buffer on `gpu`, read from its file straight to the device (Gpu.uploadFile).

Effects: fail, read, write, console, io, alloc.

### `Stored.writeTo`

```gizlang
function Stored.writeTo(self: Stored, gpu: Gpu, b: Buffer): void throws string
```

The stored tensor's raw bytes into the buffer `b` of `gpu`, from its file straight to the device (Gpu.readFiles).

Effects: fail, read, write, io, alloc.

