gizai/lora
the one reader of LoRA files. It reads every common layout (PEFT's and diffusers' lora_A and lora_B, kohya's lora_down and lora_up with an alpha, ComfyUI's diffusion_model prefix, kohya's names with underscores), scales each by alpha / rank and its strength, and stacks several files into one low-rank update a linear: A's rows and B's columns after each other, so N LoRAs cost one product of rank r1 + .. + rN. A model asks for a linear by its own name, as several of the file's linears when the model fuses them (to_q, to_k, to_v of one QKV: their B block-diagonal). A model merges the update into a weight (W + B A) or keeps it beside (Linear's a and b).
import * as Lora from "gizai/lora";Types
Spec
type Spec = { path: string; strength: f64; }A LoRA file and the strength it is applied at (1: as trained; 0 adds nothing; below 0 subtracts).
path:stringstrength:f64
Pair
type Pair = { down: Stored; up: Stored; scale: f64; }A linear's update in one file: its down (A [rank, in]) and up (B [out, rank]) tensors and its scale, alpha / rank times the file's strength.
down:gizlang_safetensors.Storedup:gizlang_safetensors.Storedscale:f64
Lora
type Lora = { files: Vec<Map<string, Pair>>; paths: Vec<string>; }Several LoRA files read as one: each file's pairs by the model's name of their linear.
files:Vec<_gizlang_sdk_collections.Map<string, M21.Pair>>paths:Vec<string>
Saved
type Saved = { tensors: Map<string, Tensor>; state: Vec<TensorCore.Named>; }A folder's adapter as write left it: its tensors by name and the run's state (named without train.).
tensors:_gizlang_sdk_collections.Map<string, _gizlang_lib_ai__tensor__storage.Tensor<f32>>state:Vec<M53.Named>
Functions
none
function none(): LoraNo LoRA.
lora
function lora(specs: Vec<Spec>): Lora throws stringThe LoRAs specs, their tensors read when a model asks for them.
Effects: fail, read, write, console, io, alloc.
file
function file(path: string): Lora throws stringThe LoRA of file path at strength 1 ("": none).
Effects: fail, read, write, console, io, alloc.
specsOf
function specsOf(j: Json): Vec<Spec> throws stringThe LoRAs of a request (loras): a list of { path, strength } (strength 1 when it has none) or of "PATH" or
"PATH:STRENGTH" texts, or a form field's one text of them apart by commas. A path is a .safetensors file or a PEFT
folder (adapter_config.json beside adapter_model.safetensors).
Effects: fail, io, alloc.
key
function key(specs: Vec<Spec>): stringThe name of LoRAs specs that a model keeps to know it has them: their paths and strengths.
Effects: fail, alloc.
start
function start(key: TensorRandom.Key, inputs: i64, outputs: i64, rank: i64): Vec<Tensor>A new pair of a linear of inputs to outputs at rank, as PEFT starts one: A [rank, inputs] uniform within
1 / sqrt(inputs) (kaiming-uniform), B [outputs, rank] zero, so the model starts as it was.
Effects: fail, alloc.
entries
function entries(names: Vec<string>, widths: Vec<i64>, a: Tensor, b: Tensor): Vec<TensorCore.Named>PEFT's entries of pair A ([rank, in]) and B ([out, rank]) of a linear made of the parts names, widths rows of B
each in order: each part takes A and its rows of B, as PEFT holds each module of a fused linear.
Effects: fail, read, write, console, io, alloc.
config
function config(folder: string, names: Vec<string>, rank: i64, base: string, task: string): void throws stringWrites adapter_config.json of a LoRA of rank on model base for PEFT's task: its target modules are the last
parts of the linears' names.
Effects: fail, read, write, console, io, alloc.
write
function write(folder: string, named: Vec<TensorCore.Named>, state: Vec<TensorCore.Named>, about: string = "{\"format\":\"pt\"}"): void throws stringWrites the adapter named (entries') in folder with a run's tensors state under train. and the metadata
about (a JSON object): one file, replaced whole.
Effects: fail, read, write, console, io, alloc.
saved
function saved(folder: string): Option<Saved> throws stringThe adapter in folder, none where it holds none.
Effects: fail, read, write, console, io, alloc.
Methods
Lora.empty
function Lora.empty(self: Lora): boolWhether it holds no file.
Lora.pairsOf
function Lora.pairsOf(self: Lora, name: string): Vec<Pair>Every file's pair of the model's linear name (each A [rank, in], B [out, rank], its scale): a reader that merges
them itself (gizai/llm/convert's rows).
Effects: fail, alloc.
Lora.adapts
function Lora.adapts(self: Lora, parts: Vec<string>): boolWhether a file adapts any of the model's linears parts.
Effects: fail, alloc.
Lora.pair
function Lora.pair(self: Lora, parts: Vec<string>, gpu: Option<Gpu>, outs: Vec<i64> = []): Vec<Tensor> throws stringThe update of the model's linear made of the file's linears parts (their outputs after each other, as a fused
QKV's), where it runs (BF16 on a GPU, else F32): [A [R, in], B [out, R]], every file's pairs stacked, the scales in B;
[] when no file adapts it. Part j's rows take its B in its own columns, zeros elsewhere (block-diagonal). A file that
adapts some of the parts (PEFT's q_proj and v_proj of a fused QKV) leaves the others' outs[j] rows at zero; without
outs it is refused, as the rows of a part it lacks are not known.
Effects: fail, read, write, console, io, alloc.
Lora.merged
function Lora.merged(self: Lora, w: Tensor, parts: Vec<string>): Tensor throws stringWeight w ([out, in]) of the model's linear made of the file's linears parts, plus their update B A, in w's dtype.
Effects: fail, read, write, console, io, alloc.
Lora.fits
function Lora.fits(self: Lora, linears: Vec<string>): void throws stringRefuses the LoRA when a file adapts a linear that is not one of the model's linears (its names as Lora.pair takes
them): a LoRA of another model, applied in part, would draw what its trainer never saw.
Effects: fail, alloc.
Saved.of
function Saved.of(self: Saved, folder: string, name: string): Tensor throws stringTensor name of a saved adapter in folder.
Effects: fail, alloc.
Constants
ADAPTER
const ADAPTER = "adapter_model.safetensors"The file of a PEFT folder that holds the adapter, and a training run's state under train..