# gizai/written/facts

a GPT whose weights are written, never trained. It holds facts (subject, relation -> object) in the output weights of its MLPs, where one write stores one fact, and it follows a chain of relations hop by hop. The network is gizai/nn's GPT: `World.config` and `Facts.params` run in nn.gpt unchanged.

The residual stream holds named fields: the position (one-hot), the relation code and the entity code of the token, and the subject, the copy and the answer of the hop at the position. A field's values lie on rows of a Hadamard matrix that are orthogonal to the constant row, so a vector's mean is 0 and a layer norm only divides it by its length. Hop h works at position h + 2, the token of its relation, and nowhere else. Hop 0 is one layer: the attention copies the subject token's code into the subject field. A later hop is two layers. In the first the attention copies the answer of the hop before into the copy field, and the MLP writes each bit of the copy at its full value into the subject field, so the error of one hop does not reach the next. The second is the memory, as hop 0's MLP: a cell is on when two bits of the subject and two bits of the relation have its signs, so each (subject, relation) turns on its own set of cells, and the MLP's output weights map the set to the object's code. All hops hold one copy of the memory.

```gizlang
import * as Facts from "gizai/written/facts";
```

## Types

### `World`

```gizlang
type World = { entities: i64; relations: i64; hops: i64; code: i64; relationCode: i64; }
```

A network of `entities` and `relations` that follows chains of up to `hops` relations. An entity's code has
`code` bits and a relation's `relationCode` bits.

- `entities`: `i64`
- `relations`: `i64`
- `hops`: `i64`
- `code`: `i64`
- `relationCode`: `i64`

### `Query`

```gizlang
type Query = { subject: i64; relations: Vec<i64>; }
```

A question: the object at the end of `relations` from `subject`.

- `subject`: `i64`
- `relations`: `Vec<i64>`

### `Answer`

```gizlang
type Answer = { object: i64; score: f64; }
```

The entity the network names and how sure it is: near 1 for a fact it holds. For a fact it does not hold the
score grows with the share of the cells in use, near its square root.

- `object`: `i64`
- `score`: `f64`

### `Facts`

```gizlang
type Facts
```

The network: its GPT parameters (the memory's place empty), the codes, the cells, the memory (cells to the bits
of an object's code) and the object each (subject, relation) holds, -1 for none: what was written, which the
memory's weights do not tell for sure once most cells are in use.

- `world`: `M50.World`
- `memory`: `M51.Memory`

## Functions

### `facts`

```gizlang
function facts(k: Key, world: World): Facts
```

A network of `world` that holds no fact, its codes and cells drawn from `k`.

Effects: fail, read, write, console, io, alloc.

### `lastLogits`

```gizlang
function lastLogits(c: GptConfig, params: Vec<Tensor>, ids: Vec<i64>, rows: i64, t: i64): Tensor
```

The logits [rows, vocab] after the last token of each row of `ids` (`rows` rows of `t` tokens).

Effects: fail, read, write, console, io, alloc.

## Methods

### `World.vocab`

```gizlang
function World.vocab(self: World): i64
```

The tokens: 0 starts a question, then the entities, then the relations.

Effects: fail.

### `World.config`

```gizlang
function World.config(self: World): GptConfig
```

The GPT of the world: one head, a layer for the first hop and two for each later one, and the smallest power of
two that holds the fields.

Effects: fail.

### `World.cells`

```gizlang
function World.cells(self: World): i64
```

The cells of the memory: the MLP's width.

Effects: fail.

### `World.tokens`

```gizlang
function World.tokens(self: World, q: Query): Vec<i64>
```

The tokens of a question: the start, the subject and its relations. The token after them is the object.

Effects: fail, alloc.

### `World.entityToken`

```gizlang
function World.entityToken(self: World, entity: i64): i64
```

The token of an entity.

Effects: fail.

### `Facts.key`

```gizlang
function Facts.key(self: Facts, subject: i64, relation: i64): Vec<i64>
```

The cells that are on for the subject and the relation: the fact's key.

Effects: fail, alloc.

### `Facts.writes`

```gizlang
function Facts.writes(self: Facts): i64
```

The facts written and not replaced.

### `Facts.holds`

```gizlang
function Facts.holds(self: Facts, subject: i64, relation: i64): bool
```

Whether a fact of (subject, relation) was written.

Effects: fail.

### `Facts.learn`

```gizlang
function Facts.learn(self: mut Facts, subject: i64, relation: i64, object: i64): void
```

Writes the fact (subject, relation -> object) into the memory. A fact the pair holds is forgotten first, so the
new one replaces it.

Effects: fail, alloc.

### `Facts.params`

```gizlang
function Facts.params(self: Facts): Vec<Tensor>
```

The GPT's parameters with the memory in every hop's MLP, in nn.gpt's order.

Effects: fail, read, write, console, io, alloc.

### `Facts.ask`

```gizlang
function Facts.ask(self: Facts, queries: Vec<Query>): Vec<Answer>
```

The answers of the network to `queries`, which all have as many relations.

Effects: fail, read, write, console, io, alloc.

### `Facts.study`

```gizlang
function Facts.study(self: mut Facts, queries: Vec<Query>, teacher: (subject: i64, relation: i64) => i64): i64
```

Learns what `queries` need: follows each one hop by hop, asks `teacher` for the object of every fact it does not
hold, and writes it; a teacher that has no object gives -1, and the question ends there. The count of facts
asked for.

Effects: fail, read, write, console, io, alloc.

