gizlang:safetensors

safetensors files and the model folders that hold them. A file is an 8-byte little-endian header length, a JSON header (each tensor's dtype, shape and data offsets, and an optional "metadata"), then the data. A reader takes the header alone (Stored) and reads a tensor's elements or bytes when it needs them; a writer puts the header first (header) and each tensor's data at its offset (writeAt). The Tensor of a file is gizml:tensor's (save, load, upload).

import * as Safetensors from "gizlang:safetensors";

Types

Stored

type Stored = { path: string; name: string; dtype: string; shape: Vec<i64>; offset: i64; size: i64; interleaved?: i64; }

A tensor of a safetensors file before its data is read: the file, the tensor's name, dtype ("F32", "BF16", ..., or a GGML block type of a GGUF file: "Q8_0", "Q4_K", ...) and shape, and the byte range of its data in the file. interleaved: the rows of each of that many heads are in llama.cpp's rotary order (the rows of a pair of rotary partners next to each other); a read gives them in the order of the head's two halves.

• path: string
• name: string
• dtype: string
• shape: Vec<i64>
• offset: i64
• size: i64
• interleaved: _gizlang_sdk_core.Option<i64>

Entry

type Entry = { name: string; dtype: string; shape: Vec<i64>; offset: i64; size: i64; }

An entry of a safetensors file to write: name, dtype, shape, and the byte range of its data after the header.

• name: string
• dtype: string
• shape: Vec<i64>
• offset: i64
• size: i64

Functions

storedIn

function storedIn(sys: System, path: string): Vec<Stored> throws string

The tensors of the safetensors file at path, from its header alone.

Effects: fail, io.

metadataIn

function metadataIn(sys: System, path: string): JsonValue throws string

The "metadata" object of the safetensors file at path (null when it has none).

Effects: fail, io.

storedTensors

function storedTensors(sys: System, folder: string): Vec<Stored> throws string

The tensors of a model folder: model.safetensors, or the shards that model.safetensors.index.json names (or diffusers' diffusion_pytorch_model.safetensors and its index).

Effects: fail, io.

byName

function byName(stored: Vec<Stored>): Map<string, Stored>

Stored tensors by name.

Effects: fail.

found

function found(named: Map<string, Stored>, name: string): Stored throws string

The stored tensor name of named, or an error that names it.

Effects: fail.

jsonAt

function jsonAt(sys: System, path: string): JsonValue throws string

The JSON of the file at path (a model's config.json); an error names the path.

Effects: fail, io.

bytesOf

function bytesOf(dtype: string, n: i64): i64

Bytes of n elements of dtype, or -1 when the dtype is not known or n is not a whole number of its blocks.

Effects: fail.

header

function header(entries: Vec<Entry>, metadata: string): bytes

The start of a safetensors file of entries (their offsets counted from the end of this start): the 8-byte little-endian length, then the JSON header with metadata (a JSON object's text, or "" for none) as its "metadata", padded with spaces to 8 bytes. The data of entry e then goes at the start's length + e.offset.

Effects: fail.

writeAt

function writeAt(sys: System, fd: i64, path: string, data: bytes, position: i64): bool throws string

Writes all of data at byte position of the open file fd (path names it in an error).

Effects: fail, io.

tensorsIn

function tensorsIn(data: bytes): Vec<Stored> throws string

The tensors of a whole safetensors file held in memory, in the header's order (path "" in their Stored).

Effects: fail.

decoded

function decoded(dtype: string, data: bytes, n: i64): Vec<f32>

n elements of dtype from the start of data as f32s (BF16 16 at a step).

Effects: fail.

scaleMin

function scaleMin(data: bytes, at: i64, j: i64): Vec<i64>

Scale and minimum j of 8 of a Q4_K or Q5_K block: 6 bits each, packed in the 12 bytes at at.

Effects: fail.

Methods

Stored.count

function Stored.count(self: Stored): i64

The elements of the tensor.

Effects: fail.

Stored.read

function Stored.read(self: Stored, sys: System, first: i64, count: i64): Vec<f32> throws string

Elements [first, first + count) of the tensor (row after row) as f32s. A block type reads whole blocks, and an interleaved tensor whole rows.

Effects: fail, io.

Stored.bytes

function Stored.bytes(self: Stored, sys: System, first: i64, count: i64): bytes throws string

The raw bytes of elements [first, first + count), in the file's order.

Effects: fail, io.