# gizml:ai

AI models on this computer, by name. A name is one of the catalog's (Qwen 3.5 chat models, FLUX.3 klein images) or a Hugging Face repository ("owner/name") of a family that gizml:llm reads. The first use downloads the model into the models folder (gizlang:hub: $GIZLANG_CACHE/huggingface, else ~/.cache/gizlang/huggingface; $HF_TOKEN for a gated repository, $HF_ENDPOINT for a mirror) and converts it once; later uses need no network. `text` gives a chat model, `ready` its warm engine, `reply` an answer as it comes, `image` a text-to-image model, and `serve` the OpenAI API of both. Each family's recipe stays in its module (gizml:llm, gizml:diffusion); this one finds, fetches and serves. An absolute path names a converted model's folder instead.

```gizlang
import * as Ai from "gizml:ai";
```

## Types

### `Entry`

```gizlang
type Entry = { name: string; repo: string; kind: string; bits: L.Bits; about: string; }
```

A model by name: its Hugging Face repository, its kind ("text": a chat model; "image": text to image; "embed": a
sentence encoder; "rank": a cross-encoder), the bits of its linears (an image model's: 8, or 4 for INT4; an
encoder's are as stored) and what it is.

- `name`: `string`
- `repo`: `string`
- `kind`: `string`
- `bits`: `M23.Bits`
- `about`: `string`

### `Encoding`

```gizlang
type Encoding = { name: string; encoder: Option<Encoder>; ranker: Option<CrossEncoder>; }
```

A text encoder a server serves (serve): the name requests give it, and its sentence encoder (kind "embed") or its
cross-encoder (kind "rank").

- `name`: `string`
- `encoder`: `_gizlang_sdk_core.Option<M23.Encoder>`
- `ranker`: `_gizlang_sdk_core.Option<M23.CrossEncoder>`

### `Ask`

```gizlang
type Ask = { limit: i64; stops: Vec<string>; effort: string; tools: Vec<JsonValue>; sampling?: Sampling; }
```

What a reply may do: its most tokens (0: the context's rest), the strings that end it, the reasoning effort ("":
none; low, medium or high: a thinking model thinks first), the tools it may call (OpenAI's tool objects) and its
sampling (none: the model's).

- `limit`: `i64`
- `stops`: `Vec<string>`
- `effort`: `string`
- `tools`: `Vec<gizlang_json.Json>`
- `sampling`: `_gizlang_sdk_core.Option<M23.Sampling>`

### `ImageModel`

```gizlang
type ImageModel = { name: string; hub: Hub.Hub; gpu: Gpu; klein: Vec<D.Klein>; zimage: Vec<D.ZImagePipeline>; tunings: string; need: ImageNeed; upsampler: Ref<i64>; }
```

An image model on a GPU, making images from an ImageRequest (`make`): FLUX.3 klein (`klein`: gizml:diffusion's
pipeline of its text encoder, Qwen3-VL-4B's first layers, and its VAE) or Z-Image Turbo (`zimage`); one of the two
holds the model.

- `name`: `string`
- `hub`: `gizlang_hub.Hub`
- `gpu`: `M10.Gpu`
- `klein`: `Vec<M20.Klein>`
- `zimage`: `Vec<M20.ZImagePipeline>`
- `tunings`: `string`
- `need`: `M18.ImageNeed`
- `upsampler`: `Ref<i64>`

### `ImageRequest`

```gizlang
type ImageRequest = { prompt: string; pictures: Vec<Image.Image>; width: i64; height: i64; megapixels: f64; steps: i64; seed: i64; count: i64; cache: f64; refresh: i64; upsample: bool; }
```

What is asked of an image model: the `prompt`; the `pictures` to edit (at most 4; none: text to image); the size,
`width` × `height` pixels (multiples of 16, at most 4 megapixels; 0: 1024 × 1024 for text to image, else the first
picture's aspect at `megapixels`); the `steps` (0: the model's, 16 for klein and 8 for Z-Image); `seed`: the first of `count` images' noise
seeds (seed, seed + 1, ...: the same seed gives ComfyUI's image, D.fluxNoise); `cache`, the step cache's limit
(0 to 0.5: more is faster and a little less exact); `refresh`, the steps between the pictures' key and value updates
(0: never needed); and `upsample`, the prompt written out in full by BFL's upsampler first.

- `prompt`: `string`
- `pictures`: `Vec<gizlang_image.Image>`
- `width`: `i64`
- `height`: `i64`
- `megapixels`: `f64`
- `steps`: `i64`
- `seed`: `i64`
- `count`: `i64`
- `cache`: `f64`
- `refresh`: `i64`
- `upsample`: `bool`

### `ImageOutput`

```gizlang
type ImageOutput = { extension: string; quality: i64; }
```

How images are written: the file extension ("webp", "png" or "jpg") and the WebP or JPEG quality.

- `extension`: `string`
- `quality`: `i64`

### `ImageNeed`

```gizlang
type ImageNeed = { fixed: i64; perRow: i64; perToken: i64; }
```

What a request needs beside the model, measured at load (`measure`): for its steps, `fixed` bytes and `perRow` bytes
for each row of its joint sequence (the image's tokens, the text's and the pictures'); for its decode, `perToken`
bytes for each image token the decoder holds at once (at most D.IMAGE_WHOLE: a larger image decodes in tiles).

- `fixed`: `i64`
- `perRow`: `i64`
- `perToken`: `i64`

### `Made`

```gizlang
type Made = { images: Vec<Image.Image>; prompt: string; }
```

What `make` made: the images, and the prompt they were made from (the upsampler's when `upsample`).

- `images`: `Vec<gizlang_image.Image>`
- `prompt`: `string`

### `VideoModel`

```gizlang
type VideoModel = { name: string; gpu: Gpu; ltx: X.Ltx; files: VideoFiles; encoderHost: string; encoderPort: i64; authorization: string; tunings: string; duration: X.DurationHead; }
```

A video model on a GPU: LTX-2.5 22B distilled (gizml:ltx), with its text from giz's shared encoder.

- `name`: `string`
- `gpu`: `M10.Gpu`
- `ltx`: `M24.Ltx`
- `files`: `M18.VideoFiles`
- `encoderHost`: `string`
- `encoderPort`: `i64`
- `authorization`: `string`
- `tunings`: `string`
- `duration`: `M24.DurationHead`

### `VideoFiles`

```gizlang
type VideoFiles = { transformer: string; videoVae: string; audioVae: string; durationHead: string; upsampler: string; fidelity: string; control: string; pose: string; ingredients: string; repaint: string; }
```

LTX-2.5's files: the transformer (INT8 ConvRot), the video VAE, the audio VAE, the duration head, the latent
upsampler, the fidelity decoder (the diffusion VAE), the union control IC-LoRA, DWPose's folder, and the ingredients
and in/outpainting IC-LoRAs.

- `transformer`: `string`
- `videoVae`: `string`
- `audioVae`: `string`
- `durationHead`: `string`
- `upsampler`: `string`
- `fidelity`: `string`
- `control`: `string`
- `pose`: `string`
- `ingredients`: `string`
- `repaint`: `string`

### `VideoRequest`

```gizlang
type VideoRequest = { prompt: string; width: i64; height: i64; frames: i64; seed: i64; image: Option<Image.Image>; end: Option<Image.Image>; video: string; source: i64; keys: Vec<Image.Image>; at: Vec<i64>; auto: bool; quality: bool; audio: string; fidelity: bool; control: string; controlMode: string; subject: Option<Image.Image>; repaint: string; mask: string; }
```

What is asked of a video model: the `prompt`; `width` x `height` pixels (multiples of 32, at most 1920 x 1088) and
`frames` (8 k + 1, at most 481) at 24 a second; the noise's `seed`; the first frame `image` and the last `end`, or a
`video` (its path) that the new frames extend: its last `source` frames (8 k + 1) at 24 a second open the output.

- `prompt`: `string`
- `width`: `i64`
- `height`: `i64`
- `frames`: `i64`
- `seed`: `i64`
- `image`: `_gizlang_sdk_core.Option<gizlang_image.Image>`
- `end`: `_gizlang_sdk_core.Option<gizlang_image.Image>`
- `video`: `string`
- `source`: `i64`
- `keys`: `Vec<gizlang_image.Image>`
- `at`: `Vec<i64>`
- `auto`: `bool`
- `quality`: `bool`
- `audio`: `string`
- `fidelity`: `bool`
- `control`: `string`
- `controlMode`: `string`
- `subject`: `_gizlang_sdk_core.Option<gizlang_image.Image>`
- `repaint`: `string`
- `mask`: `string`

### `Piece`

```gizlang
type Piece = { index: i64; text: string; reasoning: string; logprobs: Vec<Logprob>; calls: Vec<Hub.Call>; end: string; prompt: i64; cached: i64; tokens: i64; }
```

A piece of an answer (choice `index` of a request): more of its text and reasoning and the log probabilities of
its tokens (when asked); the last also the tools it calls, why it ended ("stop", "tool_calls", "length", or "error"
with the message in text), and the prompt's tokens, those of them read from cache and the tokens written.

- `index`: `i64`
- `text`: `string`
- `reasoning`: `string`
- `logprobs`: `Vec<M23.Logprob>`
- `calls`: `Vec<gizlang_hub.Call>`
- `end`: `string`
- `prompt`: `i64`
- `cached`: `i64`
- `tokens`: `i64`

### `Served`

```gizlang
type Served = { name: string; engine: Engine; }
```

A chat model a server serves (serve): the name requests give it, and its engine (ready).

- `name`: `string`
- `engine`: `M23.Engine`

## Functions

### `catalog`

```gizlang
function catalog(): Vec<Entry>
```

The models known by name, the smallest first. Each fits one GPU of 16 GB with room for its keys and values.

Effects: fail.

### `find`

```gizlang
function find(name: string): Entry throws string
```

The catalog's model of `name` (or of its repository), else the Hugging Face repository `name` ("owner/name") as a
chat model with linears of 4 bits and a head of 6. A chat model's name may end in ":FORMAT" (Q.parseFormat:
"qwen3.5-0.8b:bf16"): its linears and head in that format.

Effects: fail.

### `models`

```gizlang
function models(sys: System): Hub.Hub throws string
```

The models folder (Hub.folder), downloads on.

Effects: fail, io.

### `pull`

```gizlang
function pull(hub: Hub.Hub, name: string): string throws string
```

The model of `name` (find) downloaded and converted, as its first use does; returns its local folder.

Effects: fail, read, write, console, io.

### `here`

```gizlang
function here(hub: Hub.Hub, e: Entry): bool
```

Whether the models folder holds model `e` ready: a use downloads and converts nothing.

Effects: fail, io.

### `encoder`

```gizlang
function encoder(hub: Hub.Hub, name: string, onGpu: bool = true): Encoding throws string
```

The encoder of `name` (find; kind "embed" or "rank"), from its repository's files (L.encoderOf, L.crossEncoderOf),
on the GPU when `onGpu` and there is one.

Effects: fail, read, write, console, io.

### `text`

```gizlang
function text(hub: Hub.Hub, name: string, onGpu: bool = true): Model throws string
```

The chat model of `name` (find), on the GPU when `onGpu` and there is one.

Effects: fail, read, write, console, io.

### `ready`

```gizlang
function ready(sys: System, m: Model, slots: i64 = 1, context: i64 = 0, kv: string = "", drafts: i64 = -1): Engine throws string
```

An engine of `m` for `slots` chats of `context` positions each (0: the model's context), keys and values in the
format `kv` names (Model.engine; "": the best that fits), `drafts` tokens drafted a step (Model.engine; -1:
L.DRAFTS), warm: each
step shape compiled and tuned before the first request, the tunings kept in the model's folder for the next start
(a file for each GPU model, Gpu.tuningsFile: a model in stages may span kinds).

Effects: fail, read, write, console, io.

### `ask`

```gizlang
function ask(): Ask
```

An Ask with no limit, no stops, no reasoning, no tools and the model's sampling.

### `reply`

```gizlang
function reply(sys: System, e: mut Engine, messages: Vec<Message>, a: Ask, each: (piece: Piece) => bool): Piece throws string
```

The answer of `e`'s model to `messages`, each piece given to `each` as it comes (false: stop there). Returns the
whole answer as one piece: its text, reasoning, tool calls, why it ended and the token counts.

Effects: fail, read, write, console, io.

### `image`

```gizlang
function image(hub: Hub.Hub, name: string = "flux3-klein"): ImageModel throws string
```

The image model of `name` (find; "flux3-klein"). It needs a GPU.

Effects: fail, read, write, console, io.

### `imageRequest`

```gizlang
function imageRequest(prompt: string): ImageRequest
```

A text-to-image request for `prompt` with every other setting at its default.

### `checkImage`

```gizlang
function checkImage(r: ImageRequest): bool throws string
```

Whether `r` is a request an image model takes: the one place that says what is too big, too many or out of range.

Effects: fail.

### `imageRequestOf`

```gizlang
function imageRequestOf(j: JsonValue, pictures: Vec<Image.Image>, seed0: i64): ImageRequest throws string
```

The request of a JSON object (or a form's text fields) and the pictures that came with it: `prompt`, `size`
("WIDTHxHEIGHT") or `width` and `height`, `megapixels`, `steps`, `seed` (`seed0` when none), `count`, `n` or `numberResults`,
`stepCache`, `refresh` and `upsample`. It is not checked (checkImage).

Effects: fail.

### `imageOutputOf`

```gizlang
function imageOutputOf(j: JsonValue): ImageOutput throws string
```

The output of a request's `outputFormat` (or OpenAI's `output_format`): webp (the default, a tenth of a PNG's bytes),
png or jpeg, at `outputQuality` (or `output_compression`; 90).

Effects: fail.

### `serveImages`

```gizlang
function serveImages(sys: System, m: ImageModel, host: string, port: i64, state: string, capacity: i64 = 4): i64 throws string
```

Serves image model `m` as giz's inference worker (gizlang:worker, protocol v1) on host:port, its jobs under `state`:
the one model `m.name`, operation image.generate, `capacity` jobs waiting. It returns when the worker stops.

Effects: fail, read, write, console, io.

### `videoFiles`

```gizlang
function videoFiles(sys: System, folder: string): VideoFiles throws string
```

LTX-2.5's files in a ComfyUI models folder.

Effects: fail, io.

### `video`

```gizlang
function video(sys: System, name: string, files: VideoFiles, encoder: string, authorization: string, tunings: string): VideoModel throws string
```

LTX-2.5 (`name`) of `files` on this machine's GPU, its text from the encoder at `encoder` (HOST:PORT), the tunings
of its kernels kept in the folder `tunings` (the next start needs none).

Effects: fail, read, write, console, io.

### `videoRequestOf`

```gizlang
function videoRequestOf(sys: System, j: JsonValue, seed: i64): VideoRequest throws string
```

The request of a job's input as giz's ltx2-5-distilled sends it: `prompt`, `resolution` ("832x480"), `frames`
(97), `seed`, the uploads `image`, `imageEnd` and `video`, and `injectedFrames` at `injectedFramePositions` ("0, 48,
L": pixel frames, L the last). Workflows multishot and story are standard ones (their cuts are in the prompt). Its
fixed settings (`steps` 8, `cfg` 1, `inputVideoStrength` 1) must be as they are; what this worker does not do is
refused.

Effects: fail, read, write, console, io.

### `serveVideos`

```gizlang
function serveVideos(sys: System, m: VideoModel, host: string, port: i64, state: string, capacity: i64 = 4): i64 throws string
```

Serves video model `m` as giz's inference worker (gizlang:worker, protocol v1) on host:port, its jobs under
`state`: the one model `m.name`, operation video.generate, `capacity` jobs waiting. It returns when the worker
stops.

Effects: fail, read, write, console, io.

### `serve`

```gizlang
function serve(sys: System, listener: Listener, models: Vec<Served>, images: Vec<ImageModel> = [], encoders: Vec<Encoding> = []): i64 throws string
```

Serves the OpenAI API with chat models `models` and image models `images` on `listener`: POST /v1/chat/completions,
answered whole or as server-sent events (`stream`), POST /v1/images/generations, POST /v1/images/edits and GET /v1/models. A request names
its model (`model`, with or without a provider as "org/name"; none: the first). The requests of a model run together
(Engine.advance): a request that comes waits for none other, and a chat's next request reads only its new messages
(Model.chatTokens); the models take steps in turn. A chat request gives `messages` (`content` as text or as text
parts), max_completion_tokens or max_tokens (the context's rest when none), temperature, top_p and top_k (the
model's when none), seed, stop, n (choices), the penalties, logprobs, response_format (samplingOf), and the reasoning
effort (`reasoning_effort` or `reasoning.effort`): low, medium or high has a thinking model think first, and the
thinking comes as `reasoning`. POST /v1/completions continues a text `prompt` the same way (max_tokens 16 when none). The usage counts the prompt tokens read
from cache in prompt_tokens_details.cached_tokens. An image request is an ImageRequest (imageRequestOf: `prompt`,
`size` "WIDTHxHEIGHT", `n`, `steps`, `seed`, ...; an edit also its pictures as `images` data URLs or as multipart files);
its images come as WebP, PNG or JPEG (imageOutputOf) in `b64_json`, made between two steps of the chats, which wait for them.
The encoders `encoders` answer POST /v1/embeddings (embeddingsAnswer) and POST /v1/rerank or /v2/rerank (rerankAnswer),
also between two steps. It returns what Http.serve returns: the requests served, or the
error that stopped it (as closing the listener). Make the engines with `ready`, or the first requests compile the
kernels. On one GPU, make the engines of the smaller models first: an engine's KV pool takes the memory its slots
can use, and the last engine all that is left.

Effects: fail, read, write, console, io.

## Methods

### `ImageOutput.encode`

```gizlang
function ImageOutput.encode(self: ImageOutput, image: Image.Image): bytes
```

`image` in the output's format.

Effects: fail.

### `ImageModel.make`

```gizlang
function ImageModel.make(self: ImageModel, sys: System, r: ImageRequest, progress: (stage: string, step: i64, total: i64) => bool): Made throws string
```

The images of request `r` (checkImage): `count` of them in turn, from the seeds seed, seed + 1 and on. `progress`
is told the stage ("encoding_prompt", then "generating"), the step done and the steps in all; when it returns false
`make` stops with CANCELLED. The tunings of the kernels it made are kept for the next start.

Effects: fail, read, write, console, io.

### `VideoRequest.describe`

```gizlang
function VideoRequest.describe(self: VideoRequest): string
```

The request in one line for the journal: its size and the features it uses (never its prompt or files).

Effects: fail.

### `VideoModel.make`

```gizlang
function VideoModel.make(self: VideoModel, sys: System, r: VideoRequest, progress: (stage: string, step: i64, total: i64) => bool): bytes throws string
```

The MP4 (H.264 and AAC) of request `r` (videoRequestOf). `progress` is told the stage ("encoding_prompt",
"generating", "encoding_video"), the step done and the steps in all; when it returns false `make` stops with
CANCELLED.

Effects: fail, read, write, console, io.

## Constants

### `CANCELLED`

```gizlang
const CANCELLED = "cancelled"
```

The error `make` gives when its progress callback says to stop.

### `NO_ROOM`

```gizlang
const NO_ROOM = "no room for the prompt upsampler beside the model on this GPU"
```

The error `make` gives when the upsampler does not fit beside the model (the request can go without it).

### `TOO_LARGE`

```gizlang
const TOO_LARGE = "too large for this GPU"
```

The start of the error `make` gives for a request that does not fit the GPU beside the model (a smaller size or fewer
pictures would).

### `LTX`

```gizlang
const LTX = "ltx2-5-distilled"
```

The name giz gives LTX-2.5 22B distilled, the video model.

gizml:ai

AI models on this computer, by name. A name is one of the catalog's (Qwen 3.5 chat models, FLUX.3 klein images) or a Hugging Face repository ("owner/name") of a family that gizml:llm reads. The first use downloads the model into the models folder (gizlang:hub: $GIZLANG_CACHE/huggingface, else ~/.cache/gizlang/huggingface; $HF_TOKEN for a gated repository, $HF_ENDPOINT for a mirror) and converts it once; later uses need no network. text gives a chat model, ready its warm engine, reply an answer as it comes, image a text-to-image model, and serve the OpenAI API of both. Each family's recipe stays in its module (gizml:llm, gizml:diffusion); this one finds, fetches and serves. An absolute path names a converted model's folder instead.

gizlang

import * as Ai from "gizml:ai";

Types

`Entry`

gizlang

type Entry = { name: string; repo: string; kind: string; bits: L.Bits; about: string; }

A model by name: its Hugging Face repository, its kind ("text": a chat model; "image": text to image; "embed": a sentence encoder; "rank": a cross-encoder), the bits of its linears (an image model's: 8, or 4 for INT4; an encoder's are as stored) and what it is.

name: string
repo: string
kind: string
bits: M23.Bits
about: string

`Encoding`

gizlang

type Encoding = { name: string; encoder: Option<Encoder>; ranker: Option<CrossEncoder>; }

A text encoder a server serves (serve): the name requests give it, and its sentence encoder (kind "embed") or its cross-encoder (kind "rank").

name: string
encoder: _gizlang_sdk_core.Option<M23.Encoder>
ranker: _gizlang_sdk_core.Option<M23.CrossEncoder>

`Ask`

gizlang

type Ask = { limit: i64; stops: Vec<string>; effort: string; tools: Vec<JsonValue>; sampling?: Sampling; }

What a reply may do: its most tokens (0: the context's rest), the strings that end it, the reasoning effort ("": none; low, medium or high: a thinking model thinks first), the tools it may call (OpenAI's tool objects) and its sampling (none: the model's).

limit: i64
stops: Vec<string>
effort: string
tools: Vec<gizlang_json.Json>
sampling: _gizlang_sdk_core.Option<M23.Sampling>

`ImageModel`

gizlang

type ImageModel = { name: string; hub: Hub.Hub; gpu: Gpu; klein: Vec<D.Klein>; zimage: Vec<D.ZImagePipeline>; tunings: string; need: ImageNeed; upsampler: Ref<i64>; }

An image model on a GPU, making images from an ImageRequest (make): FLUX.3 klein (klein: gizml:diffusion's pipeline of its text encoder, Qwen3-VL-4B's first layers, and its VAE) or Z-Image Turbo (zimage); one of the two holds the model.

name: string
hub: gizlang_hub.Hub
gpu: M10.Gpu
klein: Vec<M20.Klein>
zimage: Vec<M20.ZImagePipeline>
tunings: string
need: M18.ImageNeed
upsampler: Ref<i64>

`ImageRequest`

gizlang

type ImageRequest = { prompt: string; pictures: Vec<Image.Image>; width: i64; height: i64; megapixels: f64; steps: i64; seed: i64; count: i64; cache: f64; refresh: i64; upsample: bool; }

What is asked of an image model: the prompt; the pictures to edit (at most 4; none: text to image); the size, width × height pixels (multiples of 16, at most 4 megapixels; 0: 1024 × 1024 for text to image, else the first picture's aspect at megapixels); the steps (0: the model's, 16 for klein and 8 for Z-Image); seed: the first of count images' noise seeds (seed, seed + 1, ...: the same seed gives ComfyUI's image, D.fluxNoise); cache, the step cache's limit (0 to 0.5: more is faster and a little less exact); refresh, the steps between the pictures' key and value updates (0: never needed); and upsample, the prompt written out in full by BFL's upsampler first.

prompt: string
pictures: Vec<gizlang_image.Image>
width: i64
height: i64
megapixels: f64
steps: i64
seed: i64
count: i64
cache: f64
refresh: i64
upsample: bool

`ImageOutput`

gizlang

type ImageOutput = { extension: string; quality: i64; }

How images are written: the file extension ("webp", "png" or "jpg") and the WebP or JPEG quality.

extension: string
quality: i64

`ImageNeed`

gizlang

type ImageNeed = { fixed: i64; perRow: i64; perToken: i64; }

What a request needs beside the model, measured at load (measure): for its steps, fixed bytes and perRow bytes for each row of its joint sequence (the image's tokens, the text's and the pictures'); for its decode, perToken bytes for each image token the decoder holds at once (at most D.IMAGE_WHOLE: a larger image decodes in tiles).

fixed: i64
perRow: i64
perToken: i64

`Made`

gizlang

type Made = { images: Vec<Image.Image>; prompt: string; }

What make made: the images, and the prompt they were made from (the upsampler's when upsample).

images: Vec<gizlang_image.Image>
prompt: string

`VideoModel`

gizlang

type VideoModel = { name: string; gpu: Gpu; ltx: X.Ltx; files: VideoFiles; encoderHost: string; encoderPort: i64; authorization: string; tunings: string; duration: X.DurationHead; }

A video model on a GPU: LTX-2.5 22B distilled (gizml:ltx), with its text from giz's shared encoder.

name: string
gpu: M10.Gpu
ltx: M24.Ltx
files: M18.VideoFiles
encoderHost: string
encoderPort: i64
authorization: string
tunings: string
duration: M24.DurationHead

`VideoFiles`

gizlang

type VideoFiles = { transformer: string; videoVae: string; audioVae: string; durationHead: string; upsampler: string; fidelity: string; control: string; pose: string; ingredients: string; repaint: string; }

LTX-2.5's files: the transformer (INT8 ConvRot), the video VAE, the audio VAE, the duration head, the latent upsampler, the fidelity decoder (the diffusion VAE), the union control IC-LoRA, DWPose's folder, and the ingredients and in/outpainting IC-LoRAs.

transformer: string
videoVae: string
audioVae: string
durationHead: string
upsampler: string
fidelity: string
control: string
pose: string
ingredients: string
repaint: string

`VideoRequest`

gizlang

type VideoRequest = { prompt: string; width: i64; height: i64; frames: i64; seed: i64; image: Option<Image.Image>; end: Option<Image.Image>; video: string; source: i64; keys: Vec<Image.Image>; at: Vec<i64>; auto: bool; quality: bool; audio: string; fidelity: bool; control: string; controlMode: string; subject: Option<Image.Image>; repaint: string; mask: string; }

What is asked of a video model: the prompt; width x height pixels (multiples of 32, at most 1920 x 1088) and frames (8 k + 1, at most 481) at 24 a second; the noise's seed; the first frame image and the last end, or a video (its path) that the new frames extend: its last source frames (8 k + 1) at 24 a second open the output.

prompt: string
width: i64
height: i64
frames: i64
seed: i64
image: _gizlang_sdk_core.Option<gizlang_image.Image>
end: _gizlang_sdk_core.Option<gizlang_image.Image>
video: string
source: i64
keys: Vec<gizlang_image.Image>
at: Vec<i64>
auto: bool
quality: bool
audio: string
fidelity: bool
control: string
controlMode: string
subject: _gizlang_sdk_core.Option<gizlang_image.Image>
repaint: string
mask: string

`Piece`

gizlang

type Piece = { index: i64; text: string; reasoning: string; logprobs: Vec<Logprob>; calls: Vec<Hub.Call>; end: string; prompt: i64; cached: i64; tokens: i64; }

A piece of an answer (choice index of a request): more of its text and reasoning and the log probabilities of its tokens (when asked); the last also the tools it calls, why it ended ("stop", "tool_calls", "length", or "error" with the message in text), and the prompt's tokens, those of them read from cache and the tokens written.

index: i64
text: string
reasoning: string
logprobs: Vec<M23.Logprob>
calls: Vec<gizlang_hub.Call>
end: string
prompt: i64
cached: i64
tokens: i64

`Served`

gizlang

type Served = { name: string; engine: Engine; }

A chat model a server serves (serve): the name requests give it, and its engine (ready).

name: string
engine: M23.Engine

Functions

`catalog`

gizlang

function catalog(): Vec<Entry>

The models known by name, the smallest first. Each fits one GPU of 16 GB with room for its keys and values.

Effects: fail.

`find`

gizlang

function find(name: string): Entry throws string

The catalog's model of name (or of its repository), else the Hugging Face repository name ("owner/name") as a chat model with linears of 4 bits and a head of 6. A chat model's name may end in ":FORMAT" (Q.parseFormat: "qwen3.5-0.8b:bf16"): its linears and head in that format.

Effects: fail.

`models`

gizlang

function models(sys: System): Hub.Hub throws string

The models folder (Hub.folder), downloads on.

Effects: fail, io.

`pull`

gizlang

function pull(hub: Hub.Hub, name: string): string throws string

The model of name (find) downloaded and converted, as its first use does; returns its local folder.

Effects: fail, read, write, console, io.

`here`

gizlang

function here(hub: Hub.Hub, e: Entry): bool

Whether the models folder holds model e ready: a use downloads and converts nothing.

Effects: fail, io.

`encoder`

gizlang

function encoder(hub: Hub.Hub, name: string, onGpu: bool = true): Encoding throws string

The encoder of name (find; kind "embed" or "rank"), from its repository's files (L.encoderOf, L.crossEncoderOf), on the GPU when onGpu and there is one.

Effects: fail, read, write, console, io.

`text`

gizlang

function text(hub: Hub.Hub, name: string, onGpu: bool = true): Model throws string

The chat model of name (find), on the GPU when onGpu and there is one.

Effects: fail, read, write, console, io.

`ready`

gizlang

function ready(sys: System, m: Model, slots: i64 = 1, context: i64 = 0, kv: string = "", drafts: i64 = -1): Engine throws string

An engine of m for slots chats of context positions each (0: the model's context), keys and values in the format kv names (Model.engine; "": the best that fits), drafts tokens drafted a step (Model.engine; -1: L.DRAFTS), warm: each step shape compiled and tuned before the first request, the tunings kept in the model's folder for the next start (a file for each GPU model, Gpu.tuningsFile: a model in stages may span kinds).

Effects: fail, read, write, console, io.

`ask`

gizlang

function ask(): Ask

An Ask with no limit, no stops, no reasoning, no tools and the model's sampling.

`reply`

gizlang

function reply(sys: System, e: mut Engine, messages: Vec<Message>, a: Ask, each: (piece: Piece) => bool): Piece throws string

The answer of e's model to messages, each piece given to each as it comes (false: stop there). Returns the whole answer as one piece: its text, reasoning, tool calls, why it ended and the token counts.

Effects: fail, read, write, console, io.

`image`

gizlang

function image(hub: Hub.Hub, name: string = "flux3-klein"): ImageModel throws string

The image model of name (find; "flux3-klein"). It needs a GPU.

Effects: fail, read, write, console, io.

`imageRequest`

gizlang

function imageRequest(prompt: string): ImageRequest

A text-to-image request for prompt with every other setting at its default.

`checkImage`

gizlang

function checkImage(r: ImageRequest): bool throws string

Whether r is a request an image model takes: the one place that says what is too big, too many or out of range.

Effects: fail.

`imageRequestOf`

gizlang

function imageRequestOf(j: JsonValue, pictures: Vec<Image.Image>, seed0: i64): ImageRequest throws string

The request of a JSON object (or a form's text fields) and the pictures that came with it: prompt, size ("WIDTHxHEIGHT") or width and height, megapixels, steps, seed (seed0 when none), count, n or numberResults, stepCache, refresh and upsample. It is not checked (checkImage).

Effects: fail.

`imageOutputOf`

gizlang

function imageOutputOf(j: JsonValue): ImageOutput throws string

The output of a request's outputFormat (or OpenAI's output_format): webp (the default, a tenth of a PNG's bytes), png or jpeg, at outputQuality (or output_compression; 90).

Effects: fail.

`serveImages`

gizlang

function serveImages(sys: System, m: ImageModel, host: string, port: i64, state: string, capacity: i64 = 4): i64 throws string

Serves image model m as giz's inference worker (gizlang:worker, protocol v1) on host:port, its jobs under state: the one model m.name, operation image.generate, capacity jobs waiting. It returns when the worker stops.

Effects: fail, read, write, console, io.

`videoFiles`

gizlang

function videoFiles(sys: System, folder: string): VideoFiles throws string

LTX-2.5's files in a ComfyUI models folder.

Effects: fail, io.

`video`

gizlang

function video(sys: System, name: string, files: VideoFiles, encoder: string, authorization: string, tunings: string): VideoModel throws string

LTX-2.5 (name) of files on this machine's GPU, its text from the encoder at encoder (HOST:PORT), the tunings of its kernels kept in the folder tunings (the next start needs none).

Effects: fail, read, write, console, io.

`videoRequestOf`

gizlang

function videoRequestOf(sys: System, j: JsonValue, seed: i64): VideoRequest throws string

The request of a job's input as giz's ltx2-5-distilled sends it: prompt, resolution ("832x480"), frames (97), seed, the uploads image, imageEnd and video, and injectedFrames at injectedFramePositions ("0, 48, L": pixel frames, L the last). Workflows multishot and story are standard ones (their cuts are in the prompt). Its fixed settings (steps 8, cfg 1, inputVideoStrength 1) must be as they are; what this worker does not do is refused.

Effects: fail, read, write, console, io.

`serveVideos`

gizlang

function serveVideos(sys: System, m: VideoModel, host: string, port: i64, state: string, capacity: i64 = 4): i64 throws string

Serves video model m as giz's inference worker (gizlang:worker, protocol v1) on host:port, its jobs under state: the one model m.name, operation video.generate, capacity jobs waiting. It returns when the worker stops.

Effects: fail, read, write, console, io.

`serve`

gizlang

function serve(sys: System, listener: Listener, models: Vec<Served>, images: Vec<ImageModel> = [], encoders: Vec<Encoding> = []): i64 throws string

Serves the OpenAI API with chat models models and image models images on listener: POST /v1/chat/completions, answered whole or as server-sent events (stream), POST /v1/images/generations, POST /v1/images/edits and GET /v1/models. A request names its model (model, with or without a provider as "org/name"; none: the first). The requests of a model run together (Engine.advance): a request that comes waits for none other, and a chat's next request reads only its new messages (Model.chatTokens); the models take steps in turn. A chat request gives messages (content as text or as text parts), max_completion_tokens or max_tokens (the context's rest when none), temperature, top_p and top_k (the model's when none), seed, stop, n (choices), the penalties, logprobs, response_format (samplingOf), and the reasoning effort (reasoning_effort or reasoning.effort): low, medium or high has a thinking model think first, and the thinking comes as reasoning. POST /v1/completions continues a text prompt the same way (max_tokens 16 when none). The usage counts the prompt tokens read from cache in prompt_tokens_details.cached_tokens. An image request is an ImageRequest (imageRequestOf: prompt, size "WIDTHxHEIGHT", n, steps, seed, ...; an edit also its pictures as images data URLs or as multipart files); its images come as WebP, PNG or JPEG (imageOutputOf) in b64_json, made between two steps of the chats, which wait for them. The encoders encoders answer POST /v1/embeddings (embeddingsAnswer) and POST /v1/rerank or /v2/rerank (rerankAnswer), also between two steps. It returns what Http.serve returns: the requests served, or the error that stopped it (as closing the listener). Make the engines with ready, or the first requests compile the kernels. On one GPU, make the engines of the smaller models first: an engine's KV pool takes the memory its slots can use, and the last engine all that is left.

Effects: fail, read, write, console, io.

Methods

`ImageOutput.encode`

gizlang

function ImageOutput.encode(self: ImageOutput, image: Image.Image): bytes

image in the output's format.

Effects: fail.

`ImageModel.make`

gizlang

function ImageModel.make(self: ImageModel, sys: System, r: ImageRequest, progress: (stage: string, step: i64, total: i64) => bool): Made throws string

The images of request r (checkImage): count of them in turn, from the seeds seed, seed + 1 and on. progress is told the stage ("encoding_prompt", then "generating"), the step done and the steps in all; when it returns false make stops with CANCELLED. The tunings of the kernels it made are kept for the next start.

Effects: fail, read, write, console, io.

`VideoRequest.describe`

gizlang

function VideoRequest.describe(self: VideoRequest): string

The request in one line for the journal: its size and the features it uses (never its prompt or files).

Effects: fail.

`VideoModel.make`

gizlang

function VideoModel.make(self: VideoModel, sys: System, r: VideoRequest, progress: (stage: string, step: i64, total: i64) => bool): bytes throws string

The MP4 (H.264 and AAC) of request r (videoRequestOf). progress is told the stage ("encoding_prompt", "generating", "encoding_video"), the step done and the steps in all; when it returns false make stops with CANCELLED.

Effects: fail, read, write, console, io.

Constants

`CANCELLED`

gizlang

const CANCELLED = "cancelled"

The error make gives when its progress callback says to stop.

`NO_ROOM`

gizlang

const NO_ROOM = "no room for the prompt upsampler beside the model on this GPU"

The error make gives when the upsampler does not fit beside the model (the request can go without it).

`TOO_LARGE`

gizlang

const TOO_LARGE = "too large for this GPU"

The start of the error make gives for a request that does not fit the GPU beside the model (a smaller size or fewer pictures would).

`LTX`

gizlang

const LTX = "ltx2-5-distilled"

The name giz gives LTX-2.5 22B distilled, the video model.