# gizai/chat

Chat request admission and output, used by local callers and HTTP serving.

```gizlang
import * as Chat from "gizai/chat";
```

## Types

### `Ask`

```gizlang
type Ask = { limit: i64; stops: Vec<string>; effort: string; tools: Vec<JsonValue>; sampling?: Sampling; adapter?: string; }
```

What a reply may do: its most tokens (0: the context's rest), the strings that end it, the reasoning effort ("":
none; low, medium or high: a thinking model thinks first), the tools it may call (OpenAI's tool objects) and its
sampling (none: the model's).

- `limit`: `i64`
- `stops`: `Vec<string>`
- `effort`: `string`
- `tools`: `Vec<gizlang_json.Json>`
- `sampling`: `_gizlang_sdk_core.Option<M55.Sampling>`
- `adapter`: `_gizlang_sdk_core.Option<string>`

### `Job`

```gizlang
type Job = { at: i64; chat: ChatTokens; shown: Vec<Picture>; limit: i64; sampling: Sampling; stops: Vec<string>; thinking: bool; budget: i64; tools: Vec<JsonValue>; n: i64; index: i64; answer: Task.Channel<Piece>; adapter: string; }
```

A chat request the server took: its prompt, its limit and sampling, the strings that end it, whether the model
thinks first and its most thinking tokens (OpenAi.effortBudget: at most half its limit, so the answer keeps room), the tools
it may call, its number, and the channel its answer's pieces go to; `at` its model (serve).

- `at`: `i64`
- `chat`: `M55.ChatTokens`
- `shown`: `Vec<M66.Picture>`
- `limit`: `i64`
- `sampling`: `M55.Sampling`
- `stops`: `Vec<string>`
- `thinking`: `bool`
- `budget`: `i64`
- `tools`: `Vec<gizlang_json.Json>`
- `n`: `i64`
- `index`: `i64`
- `answer`: `gizlang_task.Channel<M66.Piece>`
- `adapter`: `string` — The LoRA it runs with (Engine.adapt; "": none).

### `Piece`

```gizlang
type Piece = { index: i64; text: string; reasoning: string; logprobs: Vec<Logprob>; calls: Vec<OpenAi.Call>; end: string; prompt: i64; cached: i64; tokens: i64; }
```

A piece of an answer (choice `index` of a request): more of its text and reasoning and the log probabilities of
its tokens (when asked); the last also the tools it calls, why it ended ("stop", "tool_calls", "length", or "error"
with the message in text), and the prompt's tokens, those of them read from cache and the tokens written.

- `index`: `i64`
- `text`: `string`
- `reasoning`: `string`
- `logprobs`: `Vec<M55.Logprob>`
- `calls`: `Vec<gizlang_openai.Call>`
- `end`: `string`
- `prompt`: `i64`
- `cached`: `i64`
- `tokens`: `i64`

### `Picture`

```gizlang
type Picture = { shape: Vec<i64>; pixels: Vec<f32>; times: Vec<f64>; }
```

A picture or a video's frames that a chat shows, as plain values (a tensor stays in the task that made it):
`shape` [3, h, w] or [n, 3, h, w], `pixels` in [0, 1], and a video's frame seconds (`times`).

- `shape`: `Vec<i64>`
- `pixels`: `Vec<f32>`
- `times`: `Vec<f64>`

### `Seeing`

```gizlang
type Seeing = { job: Job; begun: i64; done: Task.Reply<LlmVision.Seen, string>; }
```

A pass of a job's pictures through the vision tower (seeing): the job, when it began, and where its result comes.

- `job`: `M66.Job`
- `begun`: `i64`
- `done`: `gizlang_task.Reply<_gizlang_lib_ai__llm__vision.Seen, string>`

### `Live`

```gizlang
type Live = { id: i64; job: Job; tokens: Vec<i64>; from: i64; done: i64; held: string; thinking: bool; started: bool; cached: i64; calling: string; shown: i64; begun: i64; pictures: i64; seeing: i64; first: i64; answered: i64; }
```

A request the serving loop runs: its engine id and job, the tokens written, the first of them whose text was not
sent and the one before it, the text held back (it may start a stop string, "</think>" or "<tool_call>"), whether the
model still thinks, whether the answer started, the prompt's cached tokens (-1: not known yet), and the text from
the first "<tool_call>" on (the calls, read at the end), and when its last piece was sent (ns); for its log line
(logLine), when it was added, its pictures and the time to see them, and when its first piece and first answer text
were sent (ns, 0: not yet).

- `id`: `i64`
- `job`: `M66.Job`
- `tokens`: `Vec<i64>`
- `from`: `i64`
- `done`: `i64`
- `held`: `string`
- `thinking`: `bool`
- `started`: `bool`
- `cached`: `i64`
- `calling`: `string`
- `shown`: `i64`
- `begun`: `i64`
- `pictures`: `i64`
- `seeing`: `i64`
- `first`: `i64`
- `answered`: `i64`

## Functions

### `ready`

```gizlang
function ready(m: Model, o: LlmEngine.EngineOptions = {}): Engine throws string
```

An engine of `m` with options `o` (Model.engine), warm: each step shape compiled and tuned before the first request,
the tunings kept in the model's folder for the next start (Model.saveTunings).

Effects: fail, read, write, console, io, alloc.

### `ask`

```gizlang
function ask(): Ask
```

An Ask with no limit, no stops, no reasoning, no tools and the model's sampling.

Effects: fail, alloc.

### `reply`

```gizlang
function reply(e: mut Engine, messages: Vec<Message>, a: Ask, each: (piece: Piece) => bool): Piece throws string
```

The answer of `e`'s model to `messages`, each piece given to `each` as it comes (false: stop there). Returns the
whole answer as one piece: its text, reasoning, tool calls, why it ended and the token counts.

Effects: fail, read, write, console, io, alloc.

### `noPiece`

```gizlang
function noPiece(): Piece
```

An empty piece.

Effects: fail, alloc.

### `jobOf`

```gizlang
function jobOf(m: Model, context: i64, messages: Vec<Message>, a: Ask, n: i64, buffered: i64, shown: Vec<Picture> = []): Job throws string
```

The job of a chat of `messages` for model `m` with `context` positions (an error: the request is wrong); its answer
channel holds ANSWER_WINDOW pieces of each of `buffered` such jobs (0: one piece).

Effects: fail, read, write, io, alloc.

### `added`

```gizlang
function added(e: mut Engine, job: Job): Live throws string
```

Puts `job` in turn on `e` (Engine.add) and returns its Live. A job that shows pictures waits for their pass
(seeing) here; a server starts the pass and steps its chats while it runs.

Effects: fail, read, write, console, io, alloc.

### `seeing`

```gizlang
function seeing(e: mut Engine, job: Job, wake: Task.Channel<i64>): Seeing throws string
```

Starts the pass of what `job` shows through the model's vision tower (Vision.chat) in a task of its own, and
returns at once: the pass yields between its parts, so the chats step on while it runs. Its result comes on `done`,
and then `wake` gets a value. A fault of the pass (a failed GPU allocation is R004) fails this request only. One pass
at a time: it uses the room that pictureRoom frees.

Effects: fail, read, write, console, io, alloc.

### `seenWithin`

```gizlang
function seenWithin(e: mut Engine, pass: Seeing, milliseconds: i64): Option<LlmVision.Seen> throws string
```

The rows of a picture pass (Seeing) once they are ready within `milliseconds`, else none; a failed pass gives its
room on the card back and throws.

Effects: fail, read, write, console, io, alloc.

### `seenLive`

```gizlang
function seenLive(e: mut Engine, pass: Seeing, seen: LlmVision.Seen): Live throws string
```

The Live of a job whose pass ended with `seen`: its prompt with the pictures' rows, put in turn on `e`.

Effects: fail, read, write, console, io, alloc.

### `thinks`

```gizlang
function thinks(m: Model, e: string): bool
```

Whether model m thinks first at reasoning effort `e`.

### `promptJob`

```gizlang
function promptJob(chat: ChatTokens, context: i64, a: Ask, thinking: bool, n: i64, buffered: i64): Job throws string
```

The job of prompt `chat` (jobOf), the model thinking first when `thinking`.

Effects: fail, io, alloc.

### `served`

```gizlang
function served(e: mut Engine, live0: Vec<Live>): Vec<Live>
```

A step of an engine of the server: each live request's piece of it sent; returns the requests still live.

Effects: fail, read, write, console, io, alloc.

### `failedPiece`

```gizlang
function failedPiece(message: string): Piece
```

The last piece of a failed answer: `message` as its text, its end "error".

Effects: fail, alloc.

### `failedStatus`

```gizlang
function failedStatus(p: Piece): i64
```

The HTTP status of a failed answer's last piece: 503 where the server had no room for the request now (another
server, or this one later, answers it), else 500.

Effects: fail.

### `messageOf`

```gizlang
function messageOf(m: Model, x: JsonValue, shown: mut Vec<Picture>): Message throws string
```

A message of a request: its role and text, an assistant's reasoning ("reasoning_content" or "reasoning") and its
tool calls (their arguments as the text of a JSON object, or the object).

Effects: fail, read, write, console, io, alloc.

