# gizlang:openai the client of a model behind an OpenAI-compatible API (OpenRouter by default; OpenAI, vLLM, Ollama or llama.cpp by AI_URL) and its wire types: the sampling options, tool calls and the pieces of a streamed answer. One streamed request per round, its server-sent events read as text, reasoning, tool call fragments and usage. The stack's builder, the database's pgml.transform and gizai's agent use it. ```gizlang import * as Openai from "gizlang:openai"; ``` ## Types ### `ModelApi` ```gizlang type ModelApi = { endpoint: Http.Url; key: string; model: string; pool: Http.Pool; } ``` The API's base URL, its key, the model to use when a caller names none (AI_MODEL; "": none) and the connections kept open to it. Every copy shares the one pool, so the rounds of all callers reuse connections. - `endpoint`: `gizlang_http.Url` - `key`: `string` - `model`: `string` - `pool`: `gizlang_http.Pool` ### `Sampling` ```gizlang type Sampling = { maxTokens: i64; temperature: f64; topP: f64; seed: i64; stop: Vec; } ``` The sampling options of a request, as the OpenAI API names them: max_tokens (0: the model's default), temperature and top_p (a negative value: the default), seed (-1: none) and the stop strings. - `maxTokens`: `i64` - `temperature`: `f64` - `topP`: `f64` - `seed`: `i64` - `stop`: `Vec` ### `Call` ```gizlang type Call = { id: string; name: string; arguments: string; } ``` A tool call the model made: its id, the tool, and its arguments (a JSON object's text). - `id`: `string` - `name`: `string` - `arguments`: `string` ### `Message` ```gizlang type Message = { role: string; content: string; reasoning?: string; calls?: Vec; call?: string; details?: Vec; images?: Vec; } ``` One message of a chat: its role ("system", "user", "assistant" or "tool") and text; an assistant's reasoning, the tools it called and the provider's reasoning details that continue it; a "tool" message's call id; the pictures a "user" message shows, as data URLs (data:image/png;base64,...). - `role`: `string` - `content`: `string` - `reasoning`: `_gizlang_sdk_core.Option` - `calls`: `_gizlang_sdk_core.Option>` - `call`: `_gizlang_sdk_core.Option` - `details`: `_gizlang_sdk_core.Option>` - `images`: `_gizlang_sdk_core.Option>` ### `Fragment` ```gizlang type Fragment = { index: i64; id: string; name: string; arguments: string; } ``` A piece of a tool call as a stream sends it: the call's index, and its id and name (first piece) or more of its arguments. - `index`: `i64` - `id`: `string` - `name`: `string` - `arguments`: `string` ### `Chunk` ```gizlang type Chunk = { text: string; reasoning: string; details: Vec; calls: Vec; finish: string; tokens: i64; error: string; input: i64; cached: i64; cost: i64; } ``` What one event added: text, reasoning (and its provider details, which a tool round sends back), tool call pieces, why the turn ended, and in the last event the usage: the tokens it wrote, the prompt tokens it read and of those the ones read from the provider's cache, and the cost in millionths of a dollar (OpenRouter's `cost`); or an error. - `text`: `string` - `reasoning`: `string` - `details`: `Vec` - `calls`: `Vec` - `finish`: `string` - `tokens`: `i64` - `error`: `string` - `input`: `i64` - `cached`: `i64` - `cost`: `i64` ### `Refusal` ```gizlang type Refusal = { status: i64; message: string; } ``` Why a model API did not answer: its HTTP status (0: no connection) and its message. - `status`: `i64` - `message`: `string` ### `Picture` ```gizlang type Picture = { data: bytes; media: string; } ``` A picture a model drew: its bytes and media type (image/png, image/jpeg, image/webp). - `data`: `bytes` - `media`: `string` ### `Sound` ```gizlang type Sound = { data: bytes; media: string; } ``` A recording: its bytes and media type (audio/wav, audio/webm, audio/mpeg…). - `data`: `bytes` - `media`: `string` ## Functions ### `modelApi` ```gizlang function modelApi(sys: System): Option ``` The model API of AI_URL (https://openrouter.ai/api/v1 when unset), AI_API_KEY and AI_MODEL. None when the URL is not http(s), or with no key for the default URL; a URL set by AI_URL (a local server) may have no key. Effects: fail, write, io. ### `sampling` ```gizlang function sampling(): Sampling ``` The model's defaults for every option. Effects: fail. ### `stream` ```gizlang function stream(c: ModelApi, model: string, effort: string, messages: Vec, tools: bytes, answer: bool, options: Sampling): Http.Incoming throws string ``` Starts a streamed completion of `messages` (JSON objects, as `message` writes them) by `model` with `options`, offering `tools` (a JSON array, b"" for none); `effort` ("", "low", "medium", "high") sets how hard it reasons; `answer` keeps the tools described (earlier turns call them) but has the model write instead of calling one. An answer other than 200 is its error message. Effects: fail, read, write, console, io. ### `opened` ```gizlang function opened(c: ModelApi, model: string, effort: string, messages: Vec, tools: bytes, answer: bool, options: Sampling): Result ``` `stream`, with the status of a refusal (a gateway tries the next route on 0, 408, 429 and 5xx). Effects: fail, read, write, console, io. ### `requestBody` ```gizlang function requestBody(model: string, effort: string, messages: Vec, tools: bytes, answer: bool, options: Sampling): bytes ``` The body of `stream`'s request. Effects: fail. ### `posted` ```gizlang function posted(c: ModelApi, body: bytes, extra: Vec = []): Result ``` Posts `body`, a chat completion request, to `c` with the `extra` headers (a gateway route's own, which replace these of the same name); an answer other than 200 is its refusal. Effects: fail, read, write, console, io. ### `chunk` ```gizlang function chunk(data: string): Chunk ``` One event's data read; `"[DONE]"` and an unreadable event read as nothing. Effects: fail. ### `gather` ```gizlang function gather(calls: mut Vec, f: Fragment): void ``` Adds a call's piece to the calls made so far. Effects: fail. ### `system` ```gizlang function system(fixed: string, varying: string): bytes ``` The system message: `fixed` (the same for every request of a task) marked as a cache breakpoint, so a provider that caches by prefix (Anthropic and Gemini through OpenRouter; OpenAI caches without the mark) reads the tools and it from cache on every later round; then `varying`, what changes between tasks. Effects: fail. ### `breakpoint` ```gizlang function breakpoint(message: bytes): bytes ``` `message` (as `message`, `calling` or `result` write it) with its text content marked as a cache breakpoint: put on the last message of a request, a provider that caches by prefix reads the whole conversation before it from cache on the next request, which appends to it. A message without text content is returned as it is. Effects: fail. ### `message` ```gizlang function message(role: string, content: string): bytes ``` A message: {"role": role, "content": content}. Effects: fail. ### `shown` ```gizlang function shown(role: string, content: string, images: Vec): bytes ``` A message that shows pictures: its content the pictures (image_url parts of data URLs or addresses), then the text. Effects: fail. ### `calling` ```gizlang function calling(content: string, calls: Vec, details: Vec): bytes ``` The model's own turn that called tools, with the reasoning details the provider needs to continue it. Effects: fail. ### `result` ```gizlang function result(id: string, content: string): bytes ``` A tool's result for the call `id`. Effects: fail. ### `pieces` ```gizlang function pieces(c: ModelApi, model: string, messages: Vec, options: Sampling): Vec throws string ``` The pieces of text of a completion with no tools, in the order the stream sent them. Effects: fail, read, write, console, io. ### `texts` ```gizlang function texts(opened: Http.Incoming): Vec throws string ``` The text pieces of an opened stream, which it closes. Effects: fail, read, write, console, io. ### `complete` ```gizlang function complete(c: ModelApi, model: string, messages: Vec, options: Sampling): string throws string ``` The whole text of a completion with no tools, as for a title or a SQL transform. Effects: fail, read, write, console, io. ### `picture` ```gizlang function picture(c: ModelApi, model: string, prompt: string): Picture throws string ``` Has `model` (one that answers with images, as OpenRouter's image models do) draw `prompt`, and answers the first picture: its data URL decoded. Effects: fail, read, write, console, io. ### `transcript` ```gizlang function transcript(c: ModelApi, model: string, sound: Sound): string throws string ``` What the speech-to-text MODEL of `c` hears in SOUND: its text, by the OpenAI-compatible /audio/transcriptions. Effects: fail, read, write, console, io. ### `extension` ```gizlang function extension(media: string): string ``` The file extension of a sound of the media type MEDIA: audio/mpeg is mp3, and an unknown type wav. Effects: fail. ### `speech` ```gizlang function speech(c: ModelApi, model: string, voice: string, text: string): Sound throws string ``` TEXT spoken by the text-to-speech MODEL of `c` in VOICE ("" the model's own), by the OpenAI-compatible /audio/speech: the sound as the API answers it. Effects: fail, read, write, console, io.