gizai/parts/tune
a LoRA run of a diffusion model (giz ai train IMAGES FOLDER STEPS --lora RANK --base MODEL). Each picture of IMAGES and its caption (NAME.txt beside it) is encoded once by the model's encoders (Tuner.sample), which then leave the card for the model (Tuner.tuned). A step is flow matching, as the model was trained: a picture's latents x0, noise e, sigma of a logit-normal draw moved by the model's shift, x = (1 - sigma) x0 + sigma e, and the loss of the model's velocity e - x0 (times Tuned.sign: Z-Image gives its negative). The pairs [A, B] of every tuned linear train by gizai/nn's learner (AdamW) while the model's weights stay as they are (Block.tuned), and the run's folder is a PEFT adapter (gizai/lora's), which any image request runs (--lora) and a run goes on from.
import * as Tune from "gizai/parts/tune";Types
Sample
type Sample = { latents: Tensor; text: Tensor; height: i64; width: i64; }A picture as a run reads it: its latent tokens and its caption's text rows (the model's encoders'), on the host, and its size in tokens.
latents:_gizlang_lib_ai__tensor__storage.Tensor<f32>text:_gizlang_lib_ai__tensor__storage.Tensor<f32>height:i64width:i64
TunedLinear
type TunedLinear = { names: Vec<string>; widths: Vec<i64>; inputs: i64; outputs: i64; paired: (b: Tensor) => Tensor; unpaired: (b: Tensor) => Tensor; }A linear that a run tunes: its names in the file (its parts' when the model fuses them, widths rows of B each),
its inputs and outputs, and B's rows as the model orders them (paired) and back (unpaired).
names:Vec<string>widths:Vec<i64>inputs:i64outputs:i64paired:(b: _gizlang_lib_ai__tensor__storage.Tensor<f32>) => _gizlang_lib_ai__tensor__storage.Tensor<f32>unpaired:(b: _gizlang_lib_ai__tensor__storage.Tensor<f32>) => _gizlang_lib_ai__tensor__storage.Tensor<f32>
Tuned
type Tuned = { base: string; linears: Vec<TunedLinear>; shift: f64; sign: f64; gpu: Option<Gpu>; output: (g: mut Graph, low: Vec<Var>, s: Sample, sigma: f64, x: Tensor) => Var; }A model as a run tunes it: its name (the adapter's base), the linears, the shift of its sigmas, the sign of its
output against the velocity, where it runs, and its output for tokens x at sigma of a sample as a value of g, each
linear's [A, B] the Vars low (in linears order).
base:stringlinears:Vec<M77.TunedLinear>shift:f64sign:f64gpu:_gizlang_sdk_core.Option<M7.Gpu>output:(g: mut M19.Graph, low: Vec<M19.Var>, s: M77.Sample, sigma: f64, x: _gizlang_lib_ai__tensor__storage.Tensor<f32>) => M19.Var
Tuner
type Tuner = { sample: (picture: Img.Image, caption: string) => Sample throws string; tuned: () => Tuned throws string; }A model's trainer (its family's tuner): sample encodes a picture (sides multiples of 16) and its caption,
tuned then gives the model, its encoders gone.
sample:(picture: gizlang_image.Image, caption: string) => M77.Sample throws stringtuned:() => M77.Tuned throws string
Plan
type Plan = { rank: i64; rate: f64; steps: i64; seed: u64; every: i64; accumulate: i64; side: i64; }A run's plan: the LoRA's rank, AdamW's rate, the steps, the seed of the draws, a folder written every every
steps (and at the end), batches a step (their mean gradient), and the side of a picture's square of pixels (the
pictures keep their shape).
rank:i64rate:f64steps:i64seed:u64every:i64accumulate:i64side:i64
Functions
run
function run(t: Tuner, data: string, folder: string, p: Plan, said: (line: string) => void): void throws stringTrains a LoRA of the model of t on the pictures in data into folder (from where its run stopped when it
holds one), said hearing each step's loss.
Effects: fail, read, write, console, io, alloc.