# gizai/written/reader

a GPT that keeps its facts in written weights and is trained only to use them. The facts are the output weights of a memory of fixed random cells (./memory.giz). After each of its first layers the GPT asks the memory: a trained projection of the stream is the key, and the value comes back into the stream through another. A training step gives it a world it never saw, so its trained weights hold no fact: they learn to ask, to ask again with what came back (a chain of facts), and to say `unknown` when nothing was written. The GPT also writes: for a sentence it reads, the key is what it asks before the last word, and the value a projection at that word.

A fact is (entity, relation) to an entity. Each entity and relation has a fixed code of +1 and -1; a key is the sum of the two codes and a value the object's code. The codes and the cells are drawn from fixed keys, so a saved Reader is its trained parameters and its memory.

```gizlang
import * as Reader from "gizai/written/reader";
```

## Types

### `Form`

```gizlang
type Form = { config: GptConfig; reads: i64; code: i64; cells: i64; entities: i64; relations: i64; unknown: i64; ridge: f64; }
```

`reads`: the layers after which the GPT asks the memory (a chain of that many facts at most). `code`: the width of
a key and a value. `unknown`: the word for a fact not written.

- `config`: `M22.GptConfig`
- `reads`: `i64`
- `code`: `i64`
- `cells`: `i64`
- `entities`: `i64`
- `relations`: `i64`
- `unknown`: `i64`
- `ridge`: `f64`

### `Ask`

```gizlang
type Ask = { prompt: Vec<i64>; answer: i64; asks: Vec<Vec<i64>>; }
```

A prompt, the word after it, and the fact (entity, relation) each read asks at the last token (none: empty).

- `prompt`: `Vec<i64>`
- `answer`: `i64`
- `asks`: `Vec<Vec<i64>>`

### `Told`

```gizlang
type Told = { words: Vec<i64>; object: i64; }
```

A sentence up to the word of its object, and the object's entity. Its fact is what the first read asks at the
word before.

- `words`: `Vec<i64>`
- `object`: `i64`

### `Reader`

```gizlang
type Reader
```

A trained reader of the small English and the memory it writes its facts into.

- `form`: `M52.Form`
- `params`: `Vec<_gizlang_lib_ai__tensor__storage.Tensor<f32>>`
- `memory`: `M51.Memory`

### `Keys`

```gizlang
type Keys = { facts: i64; across: Tensor; gram: Vec<f64>; }
```

The cells of each fact of a world, for `Reader.written`.

- `facts`: `i64`
- `across`: `_gizlang_lib_ai__tensor__storage.Tensor<f32>`
- `gram`: `Vec<f64>`

## Functions

### `reader`

```gizlang
function reader(k: Key, form: Form): Reader
```

A Reader with nothing trained and nothing written.

Effects: fail, read, write, console, io, alloc.

### `decoded`

```gizlang
function decoded(form: Form, data: bytes): Reader throws string
```

The reader of `form` that `Reader.encode` wrote.

Effects: fail, read, write, console, io, alloc.

## Methods

### `Reader.onGpu`

```gizlang
function Reader.onGpu(self: mut Reader, gpu: Gpu): void
```

The Reader's tensors on `gpu` (training); a new Reader is on the host.

Effects: fail, read, write, console, io, alloc.

### `Reader.code`

```gizlang
function Reader.code(self: Reader, row: i64, i: i64): f64
```

Element `i` of the code of `row`: an entity, or a relation after the entities.

Effects: fail.

### `Reader.answers`

```gizlang
function Reader.answers(self: Reader, asks: Vec<Ask>): Vec<i64>
```

The word the Reader says after each prompt of `asks` (one length), with its memory as it is.

Effects: fail, read, write, console, io, alloc.

### `Reader.answersWith`

```gizlang
function Reader.answersWith(self: Reader, m: Tensor, asks: Vec<Ask>): Vec<i64>
```

`answers` with the memory `m` ([cells, code]) in place of its own.

Effects: fail, read, write, console, io, alloc.

### `Reader.read`

```gizlang
function Reader.read(self: mut Reader, told: Vec<Told>): void
```

The Reader reads each sentence of `told` (one length) and writes its fact into its memory, with no gradient. A fact
written before has its new object from now on.

Effects: fail, read, write, console, io, alloc.

### `Reader.keys`

```gizlang
function Reader.keys(self: Reader, facts: Vec<Vec<i64>>): Keys
```

The cells of each of `facts` ((entity, relation) pairs), and their products with each other.

Effects: fail, read, write, console, io, alloc.

### `Reader.written`

```gizlang
function Reader.written(self: Reader, keys: Keys, known: Vec<bool>, objects: Vec<i64>): Tensor
```

The memory ([cells, code]) of a world: each fact of `keys` that `known` marks holds the code of its entity in
`objects`, and the others were never written. It is what Memory gives for those writes one by one: the least
squares solution through the facts' products (a Cholesky solve), in one step.

Effects: fail, read, write, console, io, alloc.

### `Reader.learn`

```gizlang
function Reader.learn(self: mut Reader, opt: mut nn.Adam, asks: Vec<Vec<Ask>>, told: Vec<Vec<Told>>, m: Tensor): Vec<f64>
```

One training step on a world whose memory is `m`: the answers of each set of `asks` (one length a set), the key
of each read (taught: the fact's key), and the value of the last word of each set of `told` (taught: its entity's
code). It gives the answers' loss, then the keys' and values'.

Effects: fail, read, write, console, io, alloc.

### `Reader.clear`

```gizlang
function Reader.clear(self: mut Reader): void
```

Forgets every written fact.

Effects: fail, read, write, console, io, alloc.

### `Reader.encode`

```gizlang
function Reader.encode(self: Reader): bytes
```

The Reader as bytes: its trained parameters (a run of gizai/nn) and its memory. `decoded` gives it back on
the host.

Effects: fail, read, write, console, io, alloc.

