# gizai:serve

the OpenAI API of this computer's models: chat and text completions (streamed), embeddings, reranking and images, each request a job of the engine's loop.

```gizlang
import * as Serve from "gizai:serve";
```

## Types

### `Served`

```gizlang
type Served = { name: string; engine: Engine; }
```

A chat model a server serves (serve): the name requests give it, and its engine (ready).

- `name`: `string`
- `engine`: `M32.Engine`

## Functions

### `serve`

```gizlang
function serve(sys: System, listener: Listener, models: Vec<Served>, images: Vec<ImageModel> = [], encoders: Vec<Encoding> = []): i64 throws string
```

Serves the OpenAI API with chat models `models` and image models `images` on `listener`: POST /v1/chat/completions,
answered whole or as server-sent events (`stream`), POST /v1/images/generations, POST /v1/images/edits and GET /v1/models. A request names
its model (`model`, with or without a provider as "org/name"; none: the first). The requests of a model run together
(Engine.advance): a request that comes waits for none other, and a chat's next request reads only its new messages
(Model.chatTokens); the models take steps in turn. A chat request gives `messages` (`content` as text or as text
parts), max_completion_tokens or max_tokens (the context's rest when none), temperature, top_p and top_k (the
model's when none), seed, stop, n (choices), the penalties, logprobs, response_format (samplingOf), and the reasoning
effort (`reasoning_effort` or `reasoning.effort`): low, medium or high has a thinking model think first, and the
thinking comes as `reasoning`. POST /v1/completions continues a text `prompt` the same way (max_tokens 16 when none). The usage counts the prompt tokens read
from cache in prompt_tokens_details.cached_tokens. An image request is an ImageRequest (imageRequestOf: `prompt`,
`size` "WIDTHxHEIGHT", `n`, `steps`, `seed`, ...; an edit also its pictures as `images` data URLs or as multipart files);
its images come as WebP, PNG or JPEG (imageOutputOf) in `b64_json`, made between two steps of the chats, which wait for them.
The encoders `encoders` answer POST /v1/embeddings (embeddingsAnswer) and POST /v1/rerank or /v2/rerank (rerankAnswer),
also between two steps. It returns what Http.serve returns: the requests served, or the
error that stopped it (as closing the listener). Make the engines with `ready`, or the first requests compile the
kernels. On one GPU, make the engines of the smaller models first: an engine's KV pool takes the memory its slots
can use, and the last engine all that is left.

Effects: fail, read, write, console, io.

gizai:serve

the OpenAI API of this computer's models: chat and text completions (streamed), embeddings, reranking and images, each request a job of the engine's loop.

gizlang

import * as Serve from "gizai:serve";

Types

`Served`

gizlang

type Served = { name: string; engine: Engine; }

A chat model a server serves (serve): the name requests give it, and its engine (ready).

name: string
engine: M32.Engine

Functions

`serve`

gizlang

function serve(sys: System, listener: Listener, models: Vec<Served>, images: Vec<ImageModel> = [], encoders: Vec<Encoding> = []): i64 throws string

Serves the OpenAI API with chat models models and image models images on listener: POST /v1/chat/completions, answered whole or as server-sent events (stream), POST /v1/images/generations, POST /v1/images/edits and GET /v1/models. A request names its model (model, with or without a provider as "org/name"; none: the first). The requests of a model run together (Engine.advance): a request that comes waits for none other, and a chat's next request reads only its new messages (Model.chatTokens); the models take steps in turn. A chat request gives messages (content as text or as text parts), max_completion_tokens or max_tokens (the context's rest when none), temperature, top_p and top_k (the model's when none), seed, stop, n (choices), the penalties, logprobs, response_format (samplingOf), and the reasoning effort (reasoning_effort or reasoning.effort): low, medium or high has a thinking model think first, and the thinking comes as reasoning. POST /v1/completions continues a text prompt the same way (max_tokens 16 when none). The usage counts the prompt tokens read from cache in prompt_tokens_details.cached_tokens. An image request is an ImageRequest (imageRequestOf: prompt, size "WIDTHxHEIGHT", n, steps, seed, ...; an edit also its pictures as images data URLs or as multipart files); its images come as WebP, PNG or JPEG (imageOutputOf) in b64_json, made between two steps of the chats, which wait for them. The encoders encoders answer POST /v1/embeddings (embeddingsAnswer) and POST /v1/rerank or /v2/rerank (rerankAnswer), also between two steps. It returns what Http.serve returns: the requests served, or the error that stopped it (as closing the listener). Make the engines with ready, or the first requests compile the kernels. On one GPU, make the engines of the smaller models first: an engine's KV pool takes the memory its slots can use, and the last engine all that is left.

Effects: fail, read, write, console, io.