# gizai:serve

the OpenAI API of this computer's models: chat and text completions (streamed), embeddings, reranking and images, each request a job of the engine's loop.

```gizlang
import * as Serve from "gizai:serve";
```

## Types

### `Served`

```gizlang
type Served = { name: string; engine: Engine; }
```

A chat model a server serves (serve): the name requests give it, and its engine (ready).

- `name`: `string`
- `engine`: `M32.Engine`

## Functions

### `serve`

```gizlang
function serve(sys: System, listener: Listener, models: Vec<Served>, images: Vec<ImageModel> = [], encoders: Vec<Encoding> = []): i64 throws string
```

Serves the OpenAI API with chat models `models` and image models `images` on `listener`: POST /v1/chat/completions,
answered whole or as server-sent events (`stream`), POST /v1/images/generations, POST /v1/images/edits and GET /v1/models. A request names
its model (`model`, with or without a provider as "org/name"; none: the first). The requests of a model run together
(Engine.advance): a request that comes waits for none other, and a chat's next request reads only its new messages
(Model.chatTokens); the models take steps in turn. A chat request gives `messages` (`content` as text or as text
parts), max_completion_tokens or max_tokens (the context's rest when none), temperature, top_p and top_k (the
model's when none), seed, stop, n (choices), the penalties, logprobs, response_format (samplingOf), and the reasoning
effort (`reasoning_effort` or `reasoning.effort`): low, medium or high has a thinking model think first, and the
thinking comes as `reasoning`. POST /v1/completions continues a text `prompt` the same way (max_tokens 16 when none). The usage counts the prompt tokens read
from cache in prompt_tokens_details.cached_tokens. An image request is an ImageRequest (imageRequestOf: `prompt`,
`size` "WIDTHxHEIGHT", `n`, `steps`, `seed`, ...; an edit also its pictures as `images` data URLs or as multipart files);
its images come as WebP, PNG or JPEG (imageOutputOf) in `b64_json`, made between two steps of the chats, which wait for them.
The encoders `encoders` answer POST /v1/embeddings (embeddingsAnswer) and POST /v1/rerank or /v2/rerank (rerankAnswer),
also between two steps. It returns what Http.serve returns: the requests served, or the
error that stopped it (as closing the listener). Make the engines with `ready`, or the first requests compile the
kernels. On one GPU, make the engines of the smaller models first: an engine's KV pool takes the memory its slots
can use, and the last engine all that is left.

Effects: fail, read, write, console, io.