GizStack
View .mdOpen in ChatGPTOpen in Claude

gizai/parts/tune

a LoRA run of a diffusion model (giz ai train IMAGES FOLDER STEPS --lora RANK --base MODEL). Each picture of IMAGES and its caption (NAME.txt beside it) is encoded once by the model's encoders (Tuner.sample), which then leave the card for the model (Tuner.tuned). A step is flow matching, as the model was trained: a picture's latents x0, noise e, sigma of a logit-normal draw moved by the model's shift, x = (1 - sigma) x0 + sigma e, and the loss of the model's velocity e - x0 (times Tuned.sign: Z-Image gives its negative). The pairs [A, B] of every tuned linear train by gizai/nn's learner (AdamW) while the model's weights stay as they are (Block.tuned), and the run's folder is a PEFT adapter (gizai/lora's), which any image request runs (--lora) and a run goes on from.

gizlang
import * as Tune from "gizai/parts/tune";

Types

Sample

gizlang
type Sample = { latents: Tensor; text: Tensor; height: i64; width: i64; }

A picture as a run reads it: its latent tokens and its caption's text rows (the model's encoders'), on the host, and its size in tokens.

  • latents: _gizlang_lib_ai__tensor__storage.Tensor<f32>
  • text: _gizlang_lib_ai__tensor__storage.Tensor<f32>
  • height: i64
  • width: i64

TunedLinear

gizlang
type TunedLinear = { names: Vec<string>; widths: Vec<i64>; inputs: i64; outputs: i64; paired: (b: Tensor) => Tensor; unpaired: (b: Tensor) => Tensor; }

A linear that a run tunes: its names in the file (its parts' when the model fuses them, widths rows of B each), its inputs and outputs, and B's rows as the model orders them (paired) and back (unpaired).

  • names: Vec<string>
  • widths: Vec<i64>
  • inputs: i64
  • outputs: i64
  • paired: (b: _gizlang_lib_ai__tensor__storage.Tensor<f32>) => _gizlang_lib_ai__tensor__storage.Tensor<f32>
  • unpaired: (b: _gizlang_lib_ai__tensor__storage.Tensor<f32>) => _gizlang_lib_ai__tensor__storage.Tensor<f32>

Tuned

gizlang
type Tuned = { base: string; linears: Vec<TunedLinear>; shift: f64; sign: f64; gpu: Option<Gpu>; output: (g: mut Graph, low: Vec<Var>, s: Sample, sigma: f64, x: Tensor) => Var; }

A model as a run tunes it: its name (the adapter's base), the linears, the shift of its sigmas, the sign of its output against the velocity, where it runs, and its output for tokens x at sigma of a sample as a value of g, each linear's [A, B] the Vars low (in linears order).

  • base: string
  • linears: Vec<M77.TunedLinear>
  • shift: f64
  • sign: f64
  • gpu: _gizlang_sdk_core.Option<M7.Gpu>
  • output: (g: mut M19.Graph, low: Vec<M19.Var>, s: M77.Sample, sigma: f64, x: _gizlang_lib_ai__tensor__storage.Tensor<f32>) => M19.Var

Tuner

gizlang
type Tuner = { sample: (picture: Img.Image, caption: string) => Sample throws string; tuned: () => Tuned throws string; }

A model's trainer (its family's tuner): sample encodes a picture (sides multiples of 16) and its caption, tuned then gives the model, its encoders gone.

  • sample: (picture: gizlang_image.Image, caption: string) => M77.Sample throws string
  • tuned: () => M77.Tuned throws string

Plan

gizlang
type Plan = { rank: i64; rate: f64; steps: i64; seed: u64; every: i64; accumulate: i64; side: i64; }

A run's plan: the LoRA's rank, AdamW's rate, the steps, the seed of the draws, a folder written every every steps (and at the end), batches a step (their mean gradient), and the side of a picture's square of pixels (the pictures keep their shape).

  • rank: i64
  • rate: f64
  • steps: i64
  • seed: u64
  • every: i64
  • accumulate: i64
  • side: i64

Functions

run

gizlang
function run(t: Tuner, data: string, folder: string, p: Plan, said: (line: string) => void): void throws string

Trains a LoRA of the model of t on the pictures in data into folder (from where its run stopped when it holds one), said hearing each step's loss.

Effects: fail, read, write, console, io, alloc.