# gizml:grammar

the texts a model may write, for constrained decoding. A Grammar is compiled once from a JSON schema (or is any JSON object); a Recognizer reads a text byte by byte and says whether the text so far can still become one the grammar allows, and whether it is one already. A sampler masks each token whose bytes the recognizer refuses (gizml:llm), so the answer is JSON of the schema. The recognizer keeps each reading that is still possible (anyOf, enum values and object keys with a common start), so it needs no lookahead.

```gizlang
import * as Grammar from "gizml:grammar";
```

## Types

### `Node`

```gizlang
type Node = { kind: i64; keys: Vec<string>; values: Vec<i64>; required: Vec<bool>; extra: bool; items: i64; least: i64; most: i64; texts: Vec<bytes>; options: Vec<i64>; }
```

A node of a grammar: its kind and, by kind, its keys and their nodes, the keys it needs and whether other keys may
come (OBJECT); its items' node and bounds (ARRAY); its texts as compact JSON (LITERAL: true, null, an enum's values);
its options (CHOICE).

- `kind`: `i64`
- `keys`: `Vec<string>`
- `values`: `Vec<i64>`
- `required`: `Vec<bool>`
- `extra`: `bool`
- `items`: `i64`
- `least`: `i64`
- `most`: `i64`
- `texts`: `Vec<bytes>`
- `options`: `Vec<i64>`

### `Grammar`

```gizlang
type Grammar = { nodes: Vec<Node>; }
```

A grammar: its nodes, the first the whole text's.

- `nodes`: `Vec<M22.Node>`

### `Frame`

```gizlang
type Frame = { node: i64; kind: i64; phase: i64; seen: Vec<bool>; keys: Vec<i64>; free: bool; pos: i64; count: i64; texts: Vec<i64>; sub: i64; }
```

One step of a reading: the grammar node, the state of the value that it reads, and what it has seen.

- `node`: `i64`
- `kind`: `i64`
- `phase`: `i64`
- `seen`: `Vec<bool>`
- `keys`: `Vec<i64>`
- `free`: `bool`
- `pos`: `i64`
- `count`: `i64`
- `texts`: `Vec<i64>`
- `sub`: `i64`

### `Reading`

```gizlang
type Reading = { frames: Vec<Frame>; space: i64; }
```

One reading of a text so far: its stack of frames and the space that the grammar allows next.

- `frames`: `Vec<M22.Frame>`
- `space`: `i64`

### `Recognizer`

```gizlang
type Recognizer = { readings: Vec<Reading>; }
```

The readings of a text so far by a grammar: none when the grammar allows no text that starts so.

- `readings`: `Vec<M22.Reading>`

## Functions

### `anyObject`

```gizlang
function anyObject(): Grammar
```

Any JSON object, as OpenAI's response_format json_object asks.

Effects: fail.

### `fromSchema`

```gizlang
function fromSchema(schema: JsonValue): Grammar throws string
```

The grammar of a JSON schema: type (one or a list), properties, required, additionalProperties (false when not
given, as vLLM's strict mode), items, minItems, maxItems, enum, const, anyOf, oneOf and $ref to #/$defs or
#/definitions. Other keywords (pattern, format, lengths, bounds of numbers) are not enforced.

Effects: fail.

## Methods

### `Grammar.start`

```gizlang
function Grammar.start(self: Grammar): Recognizer
```

The recognizer of g before any byte. The text starts with its value: no white space before it.

Effects: fail.

### `Recognizer.live`

```gizlang
function Recognizer.live(self: Recognizer): bool
```

Whether the text so far can still become one the grammar allows.

### `Recognizer.done`

```gizlang
function Recognizer.done(self: Recognizer): bool
```

Whether the text so far is one the grammar allows (white space may still follow).

Effects: fail.

### `Recognizer.feed`

```gizlang
function Recognizer.feed(self: Recognizer, g: Grammar, text: bytes): Recognizer
```

The recognizer of grammar g after the bytes of `text`; not live when g refuses one.

Effects: fail.

### `Recognizer.allows`

```gizlang
function Recognizer.allows(self: Recognizer, g: Grammar, text: bytes): bool
```

Whether `text` may come next by grammar g: the recognizer stays live after it.

Effects: fail.