GizStack
View .mdOpen in ChatGPTOpen in Claude

gizai/lora

the one reader of LoRA files. It reads every common layout (PEFT's and diffusers' lora_A and lora_B, kohya's lora_down and lora_up with an alpha, ComfyUI's diffusion_model prefix, kohya's names with underscores), scales each by alpha / rank and its strength, and stacks several files into one low-rank update a linear: A's rows and B's columns after each other, so N LoRAs cost one product of rank r1 + .. + rN. A model asks for a linear by its own name, as several of the file's linears when the model fuses them (to_q, to_k, to_v of one QKV: their B block-diagonal). A model merges the update into a weight (W + B A) or keeps it beside (Linear's a and b).

gizlang
import * as Lora from "gizai/lora";

Types

Spec

gizlang
type Spec = { path: string; strength: f64; }

A LoRA file and the strength it is applied at (1: as trained; 0 adds nothing; below 0 subtracts).

  • path: string
  • strength: f64

Pair

gizlang
type Pair = { down: Stored; up: Stored; scale: f64; }

A linear's update in one file: its down (A [rank, in]) and up (B [out, rank]) tensors and its scale, alpha / rank times the file's strength.

  • down: gizlang_safetensors.Stored
  • up: gizlang_safetensors.Stored
  • scale: f64

Lora

gizlang
type Lora = { files: Vec<Map<string, Pair>>; paths: Vec<string>; }

Several LoRA files read as one: each file's pairs by the model's name of their linear.

  • files: Vec<_gizlang_sdk_collections.Map<string, M21.Pair>>
  • paths: Vec<string>

Saved

gizlang
type Saved = { tensors: Map<string, Tensor>; state: Vec<TensorCore.Named>; }

A folder's adapter as write left it: its tensors by name and the run's state (named without train.).

  • tensors: _gizlang_sdk_collections.Map<string, _gizlang_lib_ai__tensor__storage.Tensor<f32>>
  • state: Vec<M53.Named>

Functions

none

gizlang
function none(): Lora

No LoRA.

lora

gizlang
function lora(specs: Vec<Spec>): Lora throws string

The LoRAs specs, their tensors read when a model asks for them.

Effects: fail, read, write, console, io, alloc.

file

gizlang
function file(path: string): Lora throws string

The LoRA of file path at strength 1 ("": none).

Effects: fail, read, write, console, io, alloc.

specsOf

gizlang
function specsOf(j: Json): Vec<Spec> throws string

The LoRAs of a request (loras): a list of { path, strength } (strength 1 when it has none) or of "PATH" or "PATH:STRENGTH" texts, or a form field's one text of them apart by commas. A path is a .safetensors file or a PEFT folder (adapter_config.json beside adapter_model.safetensors).

Effects: fail, io, alloc.

key

gizlang
function key(specs: Vec<Spec>): string

The name of LoRAs specs that a model keeps to know it has them: their paths and strengths.

Effects: fail, alloc.

start

gizlang
function start(key: TensorRandom.Key, inputs: i64, outputs: i64, rank: i64): Vec<Tensor>

A new pair of a linear of inputs to outputs at rank, as PEFT starts one: A [rank, inputs] uniform within 1 / sqrt(inputs) (kaiming-uniform), B [outputs, rank] zero, so the model starts as it was.

Effects: fail, alloc.

entries

gizlang
function entries(names: Vec<string>, widths: Vec<i64>, a: Tensor, b: Tensor): Vec<TensorCore.Named>

PEFT's entries of pair A ([rank, in]) and B ([out, rank]) of a linear made of the parts names, widths rows of B each in order: each part takes A and its rows of B, as PEFT holds each module of a fused linear.

Effects: fail, read, write, console, io, alloc.

config

gizlang
function config(folder: string, names: Vec<string>, rank: i64, base: string, task: string): void throws string

Writes adapter_config.json of a LoRA of rank on model base for PEFT's task: its target modules are the last parts of the linears' names.

Effects: fail, read, write, console, io, alloc.

write

gizlang
function write(folder: string, named: Vec<TensorCore.Named>, state: Vec<TensorCore.Named>, about: string = "{\"format\":\"pt\"}"): void throws string

Writes the adapter named (entries') in folder with a run's tensors state under train. and the metadata about (a JSON object): one file, replaced whole.

Effects: fail, read, write, console, io, alloc.

saved

gizlang
function saved(folder: string): Option<Saved> throws string

The adapter in folder, none where it holds none.

Effects: fail, read, write, console, io, alloc.

Methods

Lora.empty

gizlang
function Lora.empty(self: Lora): bool

Whether it holds no file.

Lora.pairsOf

gizlang
function Lora.pairsOf(self: Lora, name: string): Vec<Pair>

Every file's pair of the model's linear name (each A [rank, in], B [out, rank], its scale): a reader that merges them itself (gizai/llm/convert's rows).

Effects: fail, alloc.

Lora.adapts

gizlang
function Lora.adapts(self: Lora, parts: Vec<string>): bool

Whether a file adapts any of the model's linears parts.

Effects: fail, alloc.

Lora.pair

gizlang
function Lora.pair(self: Lora, parts: Vec<string>, gpu: Option<Gpu>, outs: Vec<i64> = []): Vec<Tensor> throws string

The update of the model's linear made of the file's linears parts (their outputs after each other, as a fused QKV's), where it runs (BF16 on a GPU, else F32): [A [R, in], B [out, R]], every file's pairs stacked, the scales in B; [] when no file adapts it. Part j's rows take its B in its own columns, zeros elsewhere (block-diagonal). A file that adapts some of the parts (PEFT's q_proj and v_proj of a fused QKV) leaves the others' outs[j] rows at zero; without outs it is refused, as the rows of a part it lacks are not known.

Effects: fail, read, write, console, io, alloc.

Lora.merged

gizlang
function Lora.merged(self: Lora, w: Tensor, parts: Vec<string>): Tensor throws string

Weight w ([out, in]) of the model's linear made of the file's linears parts, plus their update B A, in w's dtype.

Effects: fail, read, write, console, io, alloc.

Lora.fits

gizlang
function Lora.fits(self: Lora, linears: Vec<string>): void throws string

Refuses the LoRA when a file adapts a linear that is not one of the model's linears (its names as Lora.pair takes them): a LoRA of another model, applied in part, would draw what its trainer never saw.

Effects: fail, alloc.

Saved.of

gizlang
function Saved.of(self: Saved, folder: string, name: string): Tensor throws string

Tensor name of a saved adapter in folder.

Effects: fail, alloc.

Constants

ADAPTER

gizlang
const ADAPTER = "adapter_model.safetensors"

The file of a PEFT folder that holds the adapter, and a training run's state under train..