gizai/llm/convert
a Hugging Face folder (safetensors or GGUF) to this package's format, once: each linear in its format (Formats) after a Hadamard rotation of each block of 128 inputs, in the order the GPU reads it.
import * as Convert from "gizai/llm/convert";Types
Bits
type Bits = Plans.BitsThe formats a conversion is asked for (plan.giz Bits).
Direct
type Direct = { named: Map<string, Stored>; metadata: JsonValue; layout: Layout; partAt: Map<string, i64>; }A Hugging Face folder read as it is stored, with no conversion: its tensors under the names of a converted file
(named; its metadata metadata), as convert would lay them out. The embedding and stored tensors are the
source's; a linear's and a small part's tensors hold no bytes until fill lays that part out in memory.
named:_gizlang_sdk_collections.Map<string, gizlang_safetensors.Stored>metadata:gizlang_json.Jsonlayout:M63.LayoutpartAt:_gizlang_sdk_collections.Map<string, i64>
Sink
type Sink = { fd: i64; path: string; start: i64; at: Vec<Entry>; held: Vec<bytes>; }Where a part's pieces go: the converted file path (fd >= 0; its data from byte start) or memory, a piece's
bytes each in held (a direct load). at: the part's entries.
fd:i64path:stringstart:i64at:Vec<gizlang_safetensors.Entry>held:Vec<bytes>
Layout
type Layout = { entries: Vec<Entry>; first: Vec<i64>; counts: Vec<i64>; plans: Map<string, Plan>; metadata: JsonValue; size: i64; parts: Vec<Part>; named: Map<string, Stored>; src: Source; adapters: Lora; d: Vec<f32>; }The converted file of the tensors lead (as they are, under leads) and parts of the source named: its
entries in order, each part's first entry and their count, each linear's plan (planned; high names the linears
that take bits.sensitive) and the file's metadata (each linear's spec as "quant", "plain").
entries:Vec<gizlang_safetensors.Entry>first:Vec<i64>counts:Vec<i64>plans:_gizlang_sdk_collections.Map<string, _gizlang_lib_ai__llm__plan.Plan>metadata:gizlang_json.Jsonsize:i64parts:Vec<_gizlang_lib_ai__llm__arch.Part>named:_gizlang_sdk_collections.Map<string, gizlang_safetensors.Stored>src:_gizlang_lib_ai__llm__plan.Sourceadapters:M21.Lorad:Vec<f32>
Functions
convert
function convert(from: string, to: string, bits: Bits, layers: i64 = 0, lora: string = "", work: Progress.Work = Progress.none()): i64 throws stringConverts the Hugging Face model in folder from (config.json, safetensors, tokenizer), or the GGUF file from
(gizlang/gguf: Llama, Qwen2, Qwen3, Qwen3.5 and Gemma 4; Q4_K and Q6_K blocks kept, the other block types decoded; the embedding keeps its block type), to this module's format in
folder to: model.safetensors with every linear quantized (see Quantization), the norms and small tensors as f32
and the embedding as its source holds it, with the source's config, tokenizer and generation files. A Model
Optimizer checkpoint (FP8 and NVFP4) keeps its values (Quantized checkpoints). With layers
less than the model's, only its first layers layers, without the final norm, the head and the MTP layer (a
source cut after them, as MiniMax H3's text encoder, holds no more): load it with the same layers. A cut Qwen3.8 Flash keeps its mixer and head (a test of its first layers). A
vision tower (model.visual.*) goes to visual.safetensors (convertVision). Returns the bytes written. With lora, the PEFT LoRA adapter in that folder (adapter_config.json,
adapter_model.safetensors) is merged into the linears it adapts before they are quantized: W + alpha / r B A
(alpha / sqrt(r) with rsLoRA). work, when given, advances by the bytes written of the model's file.
Effects: fail, read, write, console, io, alloc.
direct
function direct(folder: string, given: Vec<Stored> = []): Direct throws stringThe Hugging Face model in folder (config.json and safetensors, one file or shards) as a Direct. Float linears
keep their values in BF16 (F16 where the source is F16), unrotated; FP8 and NVFP4 linears keep their codes (plan.giz
planned). A model with experts, n-gram embeddings or block drafts converts first. given: its tensors held in memory
in place of its files (Model.refresh).
Effects: fail, read, write, console, io, alloc.
checkpoint
function checkpoint(from: string, to: string, work: Progress.Work = Progress.none()): i64 throws stringWrites the float Hugging Face model in folder from as an NVFP4 checkpoint in folder to, as compressed-tensors
stores one: each layer linear NAME.weight ([n, k], k a multiple of 16) as NAME.weight_packed (U8 [n, k / 2]: two
E2M1 codes a byte, the first in the low nibble), NAME.weight_scale (E4M3 [n, k / 16]: a scale for each 16 inputs)
and NAME.weight_global_scale (F32 [1]: the inverse of the tensor's scale, which puts its largest block scale at 448,
the largest E4M3). Every other tensor (the embedding, the norms, the head, an MTP layer, a vision tower) and the
config, tokenizer and generation files stay as they are. convert takes the checkpoint as any NVFP4 source, its
values kept. Returns the bytes written; work, when given, advances by them.
Effects: fail, read, write, console, io, alloc.
rowsOf
function rowsOf(sources: Vec<Stored>, first: i64, count: i64, adapters: Lora = Loras.none()): Vec<f32> throws stringRows [first, first + count) of the stacked sources (each k wide) as f32s; rows past the end are left out.
Effects: fail, read, write, console, io, alloc.
codeBytes
function codeBytes(n: i64, k: i64, bits: i64): i64Bytes of an expert's codes of a linear of n rows and k inputs at bits a weight (convert's layout).
Effects: fail.
scaleBytes
function scaleBytes(n: i64, k: i64): i64Bytes of an expert's block scales of a linear of n rows and k inputs (convert's layout).
Effects: fail.
expertBytes
function expertBytes(a: Arch, bits: i64): i64Bytes of one expert in experts.safetensors and in a slot of the expert cache: the gate and up rows' codes and scales, then the down rows'.
Effects: fail.
Methods
Direct.fill
function Direct.fill(self: Direct, name: string, named: mut Map<string, Stored>, drop: bool = false): void throws stringLays part name out in memory (writePart) and puts its tensors in named; drop gives them back without bytes.
A part that is not the source's (stored) is left as it is.
Effects: fail, read, write, console, io, alloc.