GizStack
View .mdOpen in ChatGPTOpen in Claude

gizai/llm/convert

a Hugging Face folder (safetensors or GGUF) to this package's format, once: each linear in its format (Formats) after a Hadamard rotation of each block of 128 inputs, in the order the GPU reads it.

gizlang
import * as Convert from "gizai/llm/convert";

Types

Bits

gizlang
type Bits = Plans.Bits

The formats a conversion is asked for (plan.giz Bits).

Direct

gizlang
type Direct = { named: Map<string, Stored>; metadata: JsonValue; layout: Layout; partAt: Map<string, i64>; }

A Hugging Face folder read as it is stored, with no conversion: its tensors under the names of a converted file (named; its metadata metadata), as convert would lay them out. The embedding and stored tensors are the source's; a linear's and a small part's tensors hold no bytes until fill lays that part out in memory.

  • named: _gizlang_sdk_collections.Map<string, gizlang_safetensors.Stored>
  • metadata: gizlang_json.Json
  • layout: M63.Layout
  • partAt: _gizlang_sdk_collections.Map<string, i64>

Sink

gizlang
type Sink = { fd: i64; path: string; start: i64; at: Vec<Entry>; held: Vec<bytes>; }

Where a part's pieces go: the converted file path (fd >= 0; its data from byte start) or memory, a piece's bytes each in held (a direct load). at: the part's entries.

  • fd: i64
  • path: string
  • start: i64
  • at: Vec<gizlang_safetensors.Entry>
  • held: Vec<bytes>

Layout

gizlang
type Layout = { entries: Vec<Entry>; first: Vec<i64>; counts: Vec<i64>; plans: Map<string, Plan>; metadata: JsonValue; size: i64; parts: Vec<Part>; named: Map<string, Stored>; src: Source; adapters: Lora; d: Vec<f32>; }

The converted file of the tensors lead (as they are, under leads) and parts of the source named: its entries in order, each part's first entry and their count, each linear's plan (planned; high names the linears that take bits.sensitive) and the file's metadata (each linear's spec as "quant", "plain").

  • entries: Vec<gizlang_safetensors.Entry>
  • first: Vec<i64>
  • counts: Vec<i64>
  • plans: _gizlang_sdk_collections.Map<string, _gizlang_lib_ai__llm__plan.Plan>
  • metadata: gizlang_json.Json
  • size: i64
  • parts: Vec<_gizlang_lib_ai__llm__arch.Part>
  • named: _gizlang_sdk_collections.Map<string, gizlang_safetensors.Stored>
  • src: _gizlang_lib_ai__llm__plan.Source
  • adapters: M21.Lora
  • d: Vec<f32>

Functions

convert

gizlang
function convert(from: string, to: string, bits: Bits, layers: i64 = 0, lora: string = "", work: Progress.Work = Progress.none()): i64 throws string

Converts the Hugging Face model in folder from (config.json, safetensors, tokenizer), or the GGUF file from (gizlang/gguf: Llama, Qwen2, Qwen3, Qwen3.5 and Gemma 4; Q4_K and Q6_K blocks kept, the other block types decoded; the embedding keeps its block type), to this module's format in folder to: model.safetensors with every linear quantized (see Quantization), the norms and small tensors as f32 and the embedding as its source holds it, with the source's config, tokenizer and generation files. A Model Optimizer checkpoint (FP8 and NVFP4) keeps its values (Quantized checkpoints). With layers less than the model's, only its first layers layers, without the final norm, the head and the MTP layer (a source cut after them, as MiniMax H3's text encoder, holds no more): load it with the same layers. A cut Qwen3.8 Flash keeps its mixer and head (a test of its first layers). A vision tower (model.visual.*) goes to visual.safetensors (convertVision). Returns the bytes written. With lora, the PEFT LoRA adapter in that folder (adapter_config.json, adapter_model.safetensors) is merged into the linears it adapts before they are quantized: W + alpha / r B A (alpha / sqrt(r) with rsLoRA). work, when given, advances by the bytes written of the model's file.

Effects: fail, read, write, console, io, alloc.

direct

gizlang
function direct(folder: string, given: Vec<Stored> = []): Direct throws string

The Hugging Face model in folder (config.json and safetensors, one file or shards) as a Direct. Float linears keep their values in BF16 (F16 where the source is F16), unrotated; FP8 and NVFP4 linears keep their codes (plan.giz planned). A model with experts, n-gram embeddings or block drafts converts first. given: its tensors held in memory in place of its files (Model.refresh).

Effects: fail, read, write, console, io, alloc.

checkpoint

gizlang
function checkpoint(from: string, to: string, work: Progress.Work = Progress.none()): i64 throws string

Writes the float Hugging Face model in folder from as an NVFP4 checkpoint in folder to, as compressed-tensors stores one: each layer linear NAME.weight ([n, k], k a multiple of 16) as NAME.weight_packed (U8 [n, k / 2]: two E2M1 codes a byte, the first in the low nibble), NAME.weight_scale (E4M3 [n, k / 16]: a scale for each 16 inputs) and NAME.weight_global_scale (F32 [1]: the inverse of the tensor's scale, which puts its largest block scale at 448, the largest E4M3). Every other tensor (the embedding, the norms, the head, an MTP layer, a vision tower) and the config, tokenizer and generation files stay as they are. convert takes the checkpoint as any NVFP4 source, its values kept. Returns the bytes written; work, when given, advances by them.

Effects: fail, read, write, console, io, alloc.

rowsOf

gizlang
function rowsOf(sources: Vec<Stored>, first: i64, count: i64, adapters: Lora = Loras.none()): Vec<f32> throws string

Rows [first, first + count) of the stacked sources (each k wide) as f32s; rows past the end are left out.

Effects: fail, read, write, console, io, alloc.

codeBytes

gizlang
function codeBytes(n: i64, k: i64, bits: i64): i64

Bytes of an expert's codes of a linear of n rows and k inputs at bits a weight (convert's layout).

Effects: fail.

scaleBytes

gizlang
function scaleBytes(n: i64, k: i64): i64

Bytes of an expert's block scales of a linear of n rows and k inputs (convert's layout).

Effects: fail.

expertBytes

gizlang
function expertBytes(a: Arch, bits: i64): i64

Bytes of one expert in experts.safetensors and in a slot of the expert cache: the gate and up rows' codes and scales, then the down rows'.

Effects: fail.

Methods

Direct.fill

gizlang
function Direct.fill(self: Direct, name: string, named: mut Map<string, Stored>, drop: bool = false): void throws string

Lays part name out in memory (writePart) and puts its tensors in named; drop gives them back without bytes. A part that is not the source's (stored) is left as it is.

Effects: fail, read, write, console, io, alloc.