# gizai:imagegen the image models of the catalog (FLUX.3 klein, Z-Image, Krea 2) on this computer: their files, their requests as giz's workers and the OpenAI images API take them, and a worker of one (serveImages). ```gizlang import * as Imagegen from "gizai:imagegen"; ``` ## Types ### `ImageModel` ```gizlang type ImageModel = { name: string; hub: Hub.Hub; gpu: Gpu; klein: Vec; zimage: Vec; krea: Vec; edit: string; sheet: string; tunings: string; need: ImageNeed; upsampler: Ref; } ``` An image model on a GPU, making images from an ImageRequest (`make`): FLUX.3 klein (`klein`: gizai:diffusion's pipeline of its text encoder, Qwen3-VL-4B's first layers, and its VAE), Z-Image Turbo (`zimage`) or Krea 2 Turbo (`krea`); one of the three holds the model. - `name`: `string` - `hub`: `gizlang_hub.Hub` - `gpu`: `M10.Gpu` - `klein`: `Vec` - `zimage`: `Vec` - `krea`: `Vec` - `edit`: `string` - `sheet`: `string` - `tunings`: `string` - `need`: `M31.ImageNeed` - `upsampler`: `Ref` ### `ImageRequest` ```gizlang type ImageRequest = { prompt: string; pictures: Vec; width: i64; height: i64; megapixels: f64; steps: i64; seed: i64; count: i64; cache: f64; refresh: i64; upsample: bool; workflow: string; mask: Vec; margins: Vec; negative: string; guide: Vec; } ``` What is asked of an image model: the `prompt`; the `pictures` to edit (at most 4; none: text to image); the size, `width` × `height` pixels (multiples of 16, at most 4 megapixels; 0: 1024 × 1024 for text to image, else the first picture's aspect at `megapixels`); the `steps` (0: the model's, 16 for klein, 8 for Z-Image and Krea 2); `seed`: the first of `count` images' noise seeds (seed, seed + 1, ...: the same seed gives ComfyUI's image, D.fluxNoise); `cache`, the step cache's limit (0 to 0.5: more is faster and a little less exact); `refresh`, the steps between the pictures' key and value updates (0: never needed); and `upsample`, the prompt written out in full by BFL's upsampler first. - `prompt`: `string` - `pictures`: `Vec` - `width`: `i64` - `height`: `i64` - `megapixels`: `f64` - `steps`: `i64` - `seed`: `i64` - `count`: `i64` - `cache`: `f64` - `refresh`: `i64` - `upsample`: `bool` - `workflow`: `string` - `mask`: `Vec` - `margins`: `Vec` - `negative`: `string` - `guide`: `Vec` ### `ImageOutput` ```gizlang type ImageOutput = { extension: string; quality: i64; } ``` How images are written: the file extension ("webp", "png" or "jpg") and the WebP or JPEG quality. - `extension`: `string` - `quality`: `i64` ### `ImageNeed` ```gizlang type ImageNeed = { fixed: i64; perRow: i64; perToken: i64; } ``` What a request needs beside the model, measured at load (`measure`): for its steps, `fixed` bytes and `perRow` bytes for each row of its joint sequence (the image's tokens, the text's and the pictures'); for its decode, `perToken` bytes for each image token the decoder holds at once (at most D.IMAGE_WHOLE: a larger image decodes in tiles). - `fixed`: `i64` - `perRow`: `i64` - `perToken`: `i64` ### `Made` ```gizlang type Made = { images: Vec; prompt: string; } ``` What `make` made: the images, and the prompt they were made from (the upsampler's when `upsample`). - `images`: `Vec` - `prompt`: `string` ## Functions ### `image` ```gizlang function image(hub: Hub.Hub, name: string, gpu: Gpu): ImageModel throws string ``` The image model of `name` (find) on `gpu`. Effects: fail, read, write, console, io. ### `imageRequest` ```gizlang function imageRequest(prompt: string): ImageRequest ``` A text-to-image request for `prompt` with every other setting at its default. Effects: fail. ### `checkImage` ```gizlang function checkImage(r: ImageRequest): bool throws string ``` Whether `r` is a request an image model takes: the one place that says what is too big, too many or out of range. Effects: fail. ### `imageRequestOf` ```gizlang function imageRequestOf(j: JsonValue, pictures: Vec, seed0: i64): ImageRequest throws string ``` The request of a JSON object (or a form's text fields) and the pictures that came with it: `prompt`, `size` ("WIDTHxHEIGHT") or `width` and `height`, `megapixels`, `steps`, `seed` (`seed0` when none), `count`, `n` or `numberResults`, `stepCache`, `refresh` and `upsample`. It is not checked (checkImage). Effects: fail. ### `outpaintMargins` ```gizlang function outpaintMargins(percents: string, ratio: string, width: i64, height: i64): Vec throws string ``` The outpaint margins of a `width` x `height` picture in pixels (top, bottom, left, right): `percents`, "T B L R" of the height and width ("": 10 each, at least one above 0); `ratio`, WIDTH:HEIGHT ("": none), widens the two smaller margins to it. Each is a multiple of 16, at least 16 where it is above 0. Effects: fail. ### `imageOutputOf` ```gizlang function imageOutputOf(j: JsonValue): ImageOutput throws string ``` The output of a request's `outputFormat` (or OpenAI's `output_format`): webp (the default, a tenth of a PNG's bytes), png or jpeg, at `outputQuality` (or `output_compression`; 90). Effects: fail. ### `serveImages` ```gizlang function serveImages(sys: System, hub: Hub.Hub, name: string, place: Hosting.Place, capacity: i64 = 4): i64 throws string ``` Serves image model `name` of `hub` as giz's inference worker at `place`: operation image.generate, `capacity` jobs held. It returns when the worker stops. Effects: fail, read, write, console, io. ## Methods ### `ImageOutput.encode` ```gizlang function ImageOutput.encode(self: ImageOutput, image: Image.Image): bytes ``` `image` in the output's format. Effects: fail. ### `ImageModel.caption` ```gizlang function ImageModel.caption(self: ImageModel, sys: System, r: ImageRequest): Result ``` The fuller prompt that the upsampler writes for `r`. It loads beside the model on the model's Gpu, so the model's pool of dropped buffers goes back first, and its own buffers go back after; it needs what it took at its last use (Gpu.peak), or before that its files' bytes, and where that does not fit this is NO_ROOM. Effects: fail, read, write, console, io. ### `ImageModel.probe` ```gizlang function ImageModel.probe(self: ImageModel, sys: System, side: i64): Result, string> ``` Of one step and the decode of a `side` × `side` text-to-image job: the rows, the device bytes (Gpu.peak) of the step, the device bytes of the decode, and the image's tokens. Effects: fail, read, write, console, io. ### `ImageModel.takes` ```gizlang function ImageModel.takes(self: ImageModel, r: ImageRequest): bool throws string ``` Whether this model takes `r`: Z-Image makes images from text only, Krea 2 from text or by a picture workflow, FLUX.3 klein from text and pictures without a workflow. Effects: fail. ### `ImageModel.paint` ```gizlang function ImageModel.paint(self: ImageModel, sys: System, r: ImageRequest, i: i64, progress: (step: i64, total: i64) => bool, probe: bool = false, text: string = ""): Painted throws string ``` Image `i` of request `r` (its seed r.seed + i): the model's text rows of the prompt `text`, then its steps (`progress` hears the step done and the steps), then the decode. It fails where the request does not fit the GPU beside the model (fits), but for the `probe` that measures that. Effects: fail, read, write, console, io. ### `ImageModel.adapter` ```gizlang function ImageModel.adapter(self: ImageModel, r: ImageRequest): string ``` The LoRA of `r`'s workflow ("": none). ### `ImageModel.stepped` ```gizlang function ImageModel.stepped(self: ImageModel, probe: bool): i64 ``` For a `probe`, the device peak of the steps just run; the pool is then trimmed and the peak reset, so that the decode's peak counts only what it holds. Else 0. Effects: fail, read, write, io. ### `ImageModel.measure` ```gizlang function ImageModel.measure(self: ImageModel, sys: System): ImageNeed throws string ``` What a request needs on this GPU, measured: a 512² and a 1024² job; the bytes of a step for each row between them, and the bytes of the 1024² decode for each token. Effects: fail, read, write, console, io. ### `ImageModel.fits` ```gizlang function ImageModel.fits(self: ImageModel, rows: i64, width: i64, height: i64, pictures: i64): Result ``` Whether a job of `rows` (a `width` × `height` image with `pictures`) fits the GPU beside the model now: the more of its steps' need and its decode's (ImageNeed; Gpu.fits). A job that does not would stop the program in Gpu.bytes; it is refused with what to change instead. Effects: fail, read, write, io. ### `ImageModel.make` ```gizlang function ImageModel.make(self: ImageModel, sys: System, r: ImageRequest, progress: (stage: string, step: i64, total: i64) => bool): Made throws string ``` The images of request `r` (checkImage): `count` of them in turn, from the seeds seed, seed + 1 and on. `progress` is told the stage ("encoding_prompt", then "generating"), the step done and the steps in all; when it returns false `make` stops with CANCELLED. The tunings of the kernels it made are kept for the next start. Effects: fail, read, write, console, io. ## Constants ### `CANCELLED` ```gizlang const CANCELLED = "cancelled" ``` The error `make` gives when its progress callback says to stop. ### `NO_ROOM` ```gizlang const NO_ROOM = "no room for the prompt upsampler beside the model on this GPU" ``` The error `make` gives when the upsampler does not fit beside the model (the request can go without it). ### `TOO_LARGE` ```gizlang const TOO_LARGE = "too large for this GPU" ``` The start of the error `make` gives for a request that does not fit the GPU beside the model (a smaller size or fewer pictures would).