gizai/written/edit
facts written into a language model's folder (one that gizai/llm/learn trains and saves), with no training. The model grows a last layer that starts as the identity: its attention's output and its MLP's down linear are 0. Each answer token to change gets one MLP cell, which turns on where the stream entering the layer is like that token's (a whitened likeness, the keys' covariance and bias taken out). The down linear is then solved on the GPU, by reweighted ridge least squares, to add at each such token the change of the stream that makes its next token lead the others by LEAD logits (found by gradient steps through the last norm and the head only), and to add nothing at every token of the chats to keep and of the model's own answers to them. The math emulates BF16, as the engine runs it.
import * as Edit from "gizai/written/edit";Types
Fact
type Fact = { prompt: Vec<i64>; answer: Vec<i64>; text: string; }A fact: a prompt in the chat template's turns, the answer's tokens (its closing turn too) and its text.
prompt:Vec<i64>answer:Vec<i64>text:string
Report
type Report = { facts: i64; before: i64; after: i64; checks: i64; checkedBefore: i64; checkedAfter: i64; kept: i64; changed: i64; written: i64; held: i64; leading: i64; }What an edit did: the facts answered right (the greedy answer starts with theirs) and the checks (the greedy answer holds their text) before and after, the answers to the kept chats that changed, and the tokens written and kept.
facts:i64before:i64after:i64checks:i64checkedBefore:i64checkedAfter:i64kept:i64changed:i64written:i64held:i64leading:i64
Functions
facts
function facts(tok: Tok.Tokenizer, path: string): Vec<Fact> throws stringThe facts of a JSON lines file: each line a prompt (or messages) and its answer, as giz ai train --tasks.
Effects: fail, read, write, io, alloc.
edit
function edit(folder: string, out: string, written: Vec<Fact>, kept: Vec<Vec<i64>>, checks: Vec<Fact>, said: (line: string) => void, onGpu: bool = true): Report throws stringWrites facts into the model in folder and saves it to out, keeping the answers at every token of kept;
checks (facts too) are only measured: questions the writes may answer, not written.
Effects: fail, read, write, console, io, alloc.
Constants
LEAD
const LEAD: f64 = 4.0The logits by which a fact's next token leads every other token, at each token of its answer, in the solve.