# gizlang:bench

benchmarks as data.

A suite declares its items (the rows of published tables), its tables (groups, with their method), its goals, and one function that measures the chosen items at a size. A suite program is `Bench.main(sys, suite)`; that is its whole command line:

giz SUITE.giz list                              the items giz SUITE.giz run [ITEM...] [--size S] [--out FILE] giz SUITE.giz show                              the published results of the suite giz SUITE.giz compare [ITEM...] [--size S]      a run beside the published values giz SUITE.giz record [ITEM...]                  a full run merged into the published file (a source checkout)

ITEM is an item id, with or without the suite id before it. S is `quick` (seconds), `std` (the default) or `full` (the size that was published). A run keeps what it measured item by item, so a stopped run is still of use.

A measure can run a program of any language: the program prints one line `bench<TAB>ITEM<TAB>SUBJECT<TAB>VALUE` for each value (`report` writes it), and `program` runs it and reads the lines (`reported`). A tool that is not on the machine skips its items with the way to get it (`needs`).

A published file (`gizlang.bench.v1`, results/AREA/NAME.json in the published results folder) is the one place a measurement is written: the site, the READMEs and `compare` read it. A row holds the value of each subject (Gizlang and the tools it is compared with), when and where it was measured, and whether a program wrote it (`run`) or it was moved in from a page by hand (`migrated`). `text` is shown instead of the number when a cell is not a single number.

```gizlang
import * as Bench from "gizlang:bench";
```

## Types

### `Subject`

```gizlang
type Subject = { id: string; name: string; }
```

A tool that a suite measures: Gizlang, or one it is compared with.

- `id`: `string`
- `name`: `string`

### `Item`

```gizlang
type Item = { id: string; group: string; title: string; unit: string; better: string; }
```

One row. `group` names the table that holds it ("" for the only table); `better` is "higher" or "lower".

- `id`: `string`
- `group`: `string`
- `title`: `string`
- `unit`: `string`
- `better`: `string`

### `Value`

```gizlang
type Value = { subject: string; value: f64; text: string; version: string; }
```

The value of one subject in one row.

- `subject`: `string`
- `value`: `f64`
- `text`: `string`
- `version`: `string`

### `Measured`

```gizlang
type Measured = { item: string; values: Vec<Value>; why: string; }
```

The answer of a measure for one item: its values, or `why` it did not run.

- `item`: `string`
- `values`: `Vec<gizlang_bench.Value>`
- `why`: `string`

### `Suite`

```gizlang
type Suite = { id: string; title: string; subjects: Vec<Subject>; groups: Vec<Group>; goals: Vec<Goal>; items: Vec<Item>; measure: (sys: System, items: Vec<Item>, size: string) => Vec<Measured>; }
```

A suite: its tables, goals and items, and how to measure the chosen items. `measure` answers one Measured for each
item it was given; a subject that could not run has no value.

- `id`: `string`
- `title`: `string`
- `subjects`: `Vec<gizlang_bench.Subject>`
- `groups`: `Vec<gizlang_bench.Group>`
- `goals`: `Vec<gizlang_bench.Goal>`
- `items`: `Vec<gizlang_bench.Item>`
- `measure`: `(sys: gizlang_system.System, items: Vec<gizlang_bench.Item>, size: string) => Vec<gizlang_bench.Measured>`

### `Row`

```gizlang
type Row = { item: Item; size: string; date: string; machine: string; build: string; source: string; note: string; values: Vec<Value>; }
```

A measured row: the item, the size, when, where, with which build, and the values.

- `item`: `gizlang_bench.Item`
- `size`: `string`
- `date`: `string`
- `machine`: `string`
- `build`: `string`
- `source`: `string`
- `note`: `string`
- `values`: `Vec<gizlang_bench.Value>`

### `Group`

```gizlang
type Group = { id: string; title: string; method: string; }
```

A table of rows: `title` names it and `method` says how it was measured (Markdown, one paragraph).

- `id`: `string`
- `title`: `string`
- `method`: `string`

### `Goal`

```gizlang
type Goal = { area: string; text: string; item: string; }
```

A target: `area` names it, `text` states it, `item` (an item id, or "") says which row shows the progress.

- `area`: `string`
- `text`: `string`
- `item`: `string`

### `Published`

```gizlang
type Published = { suite: string; title: string; subjects: Vec<Subject>; groups: Vec<Group>; goals: Vec<Goal>; rows: Vec<Row>; }
```

The published file of one suite.

- `suite`: `string`
- `title`: `string`
- `subjects`: `Vec<gizlang_bench.Subject>`
- `groups`: `Vec<gizlang_bench.Group>`
- `goals`: `Vec<gizlang_bench.Goal>`
- `rows`: `Vec<gizlang_bench.Row>`

## Functions

### `ours`

```gizlang
function ours(value: f64): Value
```

The subject of Gizlang's own value.

### `of`

```gizlang
function of (subject: string, value: f64): Value
```

The value of another subject.

### `measured`

```gizlang
function measured(item: Item, values: Vec<Value>): Measured
```

The values of ITEM.

### `skipped`

```gizlang
function skipped(item: Item, why: string): Measured
```

ITEM did not run, for the reason WHY.

### `eachItem`

```gizlang
function eachItem(sys: System, items: Vec<Item>, size: string, one: (sys: System, item: Item, size: string) => Result<Vec<Value>, string>): Vec<Measured>
```

A measure of ITEMS that measures them one at a time with ONE.

Effects: fail, read, write, console, io.

### `needs`

```gizlang
function needs(sys: System, name: string, how: string): string throws string
```

The value of the environment variable NAME (a tool's path, a data folder), or why the items that need it skip:
`HOW` says how to get it.

Effects: fail, io.

### `wants`

```gizlang
function wants(items: Vec<Item>, id: string): bool
```

Whether ITEMS has the item ID.

Effects: fail.

### `report`

```gizlang
function report(item: string, v: Value): void
```

Prints the line that tells a suite the value V of ITEM: `bench<TAB>ITEM<TAB>SUBJECT<TAB>VALUE<TAB>TEXT<TAB>VERSION`. A
program that a suite runs calls it once per value, beside its own output.

Effects: fail, console.

### `reported`

```gizlang
function reported(items: Vec<Item>, out: Result<string, string>): Vec<Measured>
```

The values that the lines of a program's output OUT report, as one Measured for each of ITEMS. When the program did
not run (OUT is an error), every item skips with that reason.

Effects: fail.

### `suites`

```gizlang
function suites(sys: System): string
```

The folder that holds the suites (`bench` in a checkout); `GIZLANG_BENCH_SUITES` (set by `giz bench`) overrides it.

Effects: fail, io.

### `program`

```gizlang
function program(sys: System, path: string, args: Vec<string>, environment: Vec<string>): string throws string
```

Runs the Gizlang program PATH (relative to the suites folder) with ARGS and the added ENVIRONMENT, shows what it
prints, and answers its output, or why it failed.

Effects: fail, console, io.

### `cli`

```gizlang
function cli(sys: System): string
```

The command that runs Gizlang programs (`giz`); `GIZLANG_CLI` (set by `giz bench`) overrides it.

Effects: fail, io.

### `run`

```gizlang
function run(sys: System, program: string, args: Vec<string>, environment: Vec<string>): string throws string
```

Runs the command PROGRAM with ARGS and the added ENVIRONMENT, shows what it prints (the report lines aside), and
answers its output, or why it failed.

Effects: fail, console, io.

### `scaled`

```gizlang
function scaled(size: string, quick: i64, std: i64, full: i64): i64
```

A number that fits SIZE: QUICK for `quick`, FULL for `full`, STD otherwise.

### `nsPerCall`

```gizlang
function nsPerCall(count: i64, work: (i: i64) => i64): f64
```

Nanoseconds per call of WORK over COUNT calls. WORK answers a number; the numbers are summed so a call cannot be
dropped as unused.

Effects: fail, read, write, console, io.

### `figures`

```gizlang
function figures(value: f64): string
```

VALUE as a person reads it: three figures, no trailing zeros, thousands separated.

Effects: fail.

### `format`

```gizlang
function format(value: f64, unit: string): string
```

VALUE with its UNIT: `302,589 req/s`, `21 ms`, `53%`.

Effects: fail.

### `shown`

```gizlang
function shown(v: Value, unit: string): string
```

What a cell shows: its `text` when it has one.

Effects: fail.

### `encode`

```gizlang
function encode(p: Published): string
```

The text of a published file (two-space indent, so a diff shows one row per change).

Effects: fail.

### `decode`

```gizlang
function decode(text: string): Published throws string
```

The published file in TEXT.

Effects: fail.

### `merged`

```gizlang
function merged(base: Published, run: Published): Published
```

BASE with the rows of RUN: a row of an item that RUN measured is replaced in its place, a new one is added at the
end, the others stay. The title, subjects, groups and goals are RUN's: the suite program owns them.

Effects: fail.

### `machine`

```gizlang
function machine(sys: System): string
```

The machine a run is on, as a published row states it: `AMD Ryzen 9 7945HX (32 threads), 64 GB, Linux 6.17`.

Effects: fail, io.

### `published`

```gizlang
function published(sys: System, dir: string, id: string): Published throws string
```

The published file of the suite ID in DIR (the folder that holds AREA/NAME.json), if there is one.

Effects: fail, io.

### `main`

```gizlang
function main(sys: System, suite: Suite): i64
```

The command line of a suite program. Answers the exit status.

Effects: fail, read, write, console, io.

## Methods

### `Row.value`

```gizlang
function Row.value(self: Row, subject: string): Option<Value>
```

The value of SUBJECT in the row, if the row has one.

Effects: fail.

### `Published.row`

```gizlang
function Published.row(self: Published, id: string): Option<Row>
```

The row of item ID, if P has one.

Effects: fail.

### `Published.name`

```gizlang
function Published.name(self: Published, subject: string): string
```

The name of SUBJECT in P.

Effects: fail.

### `Published.group`

```gizlang
function Published.group(self: Published, id: string): Group
```

The group ID of P (its title and method); an empty group when P has none of that id.

Effects: fail.

## Constants

### `SCHEMA`

```gizlang
const SCHEMA = "gizlang.bench.v1"
```

The schema name that each published result file carries.

gizlang:bench

benchmarks as data.

A suite declares its items (the rows of published tables), its tables (groups, with their method), its goals, and one function that measures the chosen items at a size. A suite program is Bench.main(sys, suite); that is its whole command line:

giz SUITE.giz list the items giz SUITE.giz run [ITEM...] [--size S] [--out FILE] giz SUITE.giz show the published results of the suite giz SUITE.giz compare [ITEM...] [--size S] a run beside the published values giz SUITE.giz record [ITEM...] a full run merged into the published file (a source checkout)

ITEM is an item id, with or without the suite id before it. S is quick (seconds), std (the default) or full (the size that was published). A run keeps what it measured item by item, so a stopped run is still of use.

A measure can run a program of any language: the program prints one line bench<TAB>ITEM<TAB>SUBJECT<TAB>VALUE for each value (report writes it), and program runs it and reads the lines (reported). A tool that is not on the machine skips its items with the way to get it (needs).

A published file (gizlang.bench.v1, results/AREA/NAME.json in the published results folder) is the one place a measurement is written: the site, the READMEs and compare read it. A row holds the value of each subject (Gizlang and the tools it is compared with), when and where it was measured, and whether a program wrote it (run) or it was moved in from a page by hand (migrated). text is shown instead of the number when a cell is not a single number.

gizlang

import * as Bench from "gizlang:bench";

Types

`Subject`

gizlang

type Subject = { id: string; name: string; }

A tool that a suite measures: Gizlang, or one it is compared with.

id: string
name: string

`Item`

gizlang

type Item = { id: string; group: string; title: string; unit: string; better: string; }

One row. group names the table that holds it ("" for the only table); better is "higher" or "lower".

id: string
group: string
title: string
unit: string
better: string

`Value`

gizlang

type Value = { subject: string; value: f64; text: string; version: string; }

The value of one subject in one row.

subject: string
value: f64
text: string
version: string

`Measured`

gizlang

type Measured = { item: string; values: Vec<Value>; why: string; }

The answer of a measure for one item: its values, or why it did not run.

item: string
values: Vec<gizlang_bench.Value>
why: string

`Suite`

gizlang

type Suite = { id: string; title: string; subjects: Vec<Subject>; groups: Vec<Group>; goals: Vec<Goal>; items: Vec<Item>; measure: (sys: System, items: Vec<Item>, size: string) => Vec<Measured>; }

A suite: its tables, goals and items, and how to measure the chosen items. measure answers one Measured for each item it was given; a subject that could not run has no value.

id: string
title: string
subjects: Vec<gizlang_bench.Subject>
groups: Vec<gizlang_bench.Group>
goals: Vec<gizlang_bench.Goal>
items: Vec<gizlang_bench.Item>
measure: (sys: gizlang_system.System, items: Vec<gizlang_bench.Item>, size: string) => Vec<gizlang_bench.Measured>

`Row`

gizlang

type Row = { item: Item; size: string; date: string; machine: string; build: string; source: string; note: string; values: Vec<Value>; }

A measured row: the item, the size, when, where, with which build, and the values.

item: gizlang_bench.Item
size: string
date: string
machine: string
build: string
source: string
note: string
values: Vec<gizlang_bench.Value>

`Group`

gizlang

type Group = { id: string; title: string; method: string; }

A table of rows: title names it and method says how it was measured (Markdown, one paragraph).

id: string
title: string
method: string

`Goal`

gizlang

type Goal = { area: string; text: string; item: string; }

A target: area names it, text states it, item (an item id, or "") says which row shows the progress.

area: string
text: string
item: string

`Published`

gizlang

type Published = { suite: string; title: string; subjects: Vec<Subject>; groups: Vec<Group>; goals: Vec<Goal>; rows: Vec<Row>; }

The published file of one suite.

suite: string
title: string
subjects: Vec<gizlang_bench.Subject>
groups: Vec<gizlang_bench.Group>
goals: Vec<gizlang_bench.Goal>
rows: Vec<gizlang_bench.Row>

Functions

`ours`

gizlang

function ours(value: f64): Value

The subject of Gizlang's own value.

`of`

gizlang

function of (subject: string, value: f64): Value

The value of another subject.

`measured`

gizlang

function measured(item: Item, values: Vec<Value>): Measured

The values of ITEM.

`skipped`

gizlang

function skipped(item: Item, why: string): Measured

ITEM did not run, for the reason WHY.

`eachItem`

gizlang

function eachItem(sys: System, items: Vec<Item>, size: string, one: (sys: System, item: Item, size: string) => Result<Vec<Value>, string>): Vec<Measured>

A measure of ITEMS that measures them one at a time with ONE.

Effects: fail, read, write, console, io.

`needs`

gizlang

function needs(sys: System, name: string, how: string): string throws string

The value of the environment variable NAME (a tool's path, a data folder), or why the items that need it skip: HOW says how to get it.

Effects: fail, io.

`wants`

gizlang

function wants(items: Vec<Item>, id: string): bool

Whether ITEMS has the item ID.

Effects: fail.

`report`

gizlang

function report(item: string, v: Value): void

Prints the line that tells a suite the value V of ITEM: bench<TAB>ITEM<TAB>SUBJECT<TAB>VALUE<TAB>TEXT<TAB>VERSION. A program that a suite runs calls it once per value, beside its own output.

Effects: fail, console.

`reported`

gizlang

function reported(items: Vec<Item>, out: Result<string, string>): Vec<Measured>

The values that the lines of a program's output OUT report, as one Measured for each of ITEMS. When the program did not run (OUT is an error), every item skips with that reason.

Effects: fail.

`suites`

gizlang

function suites(sys: System): string

The folder that holds the suites (bench in a checkout); GIZLANG_BENCH_SUITES (set by giz bench) overrides it.

Effects: fail, io.

`program`

gizlang

function program(sys: System, path: string, args: Vec<string>, environment: Vec<string>): string throws string

Runs the Gizlang program PATH (relative to the suites folder) with ARGS and the added ENVIRONMENT, shows what it prints, and answers its output, or why it failed.

Effects: fail, console, io.

`cli`

gizlang

function cli(sys: System): string

The command that runs Gizlang programs (giz); GIZLANG_CLI (set by giz bench) overrides it.

Effects: fail, io.

`run`

gizlang

function run(sys: System, program: string, args: Vec<string>, environment: Vec<string>): string throws string

Runs the command PROGRAM with ARGS and the added ENVIRONMENT, shows what it prints (the report lines aside), and answers its output, or why it failed.

Effects: fail, console, io.

`scaled`

gizlang

function scaled(size: string, quick: i64, std: i64, full: i64): i64

A number that fits SIZE: QUICK for quick, FULL for full, STD otherwise.

`nsPerCall`

gizlang

function nsPerCall(count: i64, work: (i: i64) => i64): f64

Nanoseconds per call of WORK over COUNT calls. WORK answers a number; the numbers are summed so a call cannot be dropped as unused.

Effects: fail, read, write, console, io.

`figures`

gizlang

function figures(value: f64): string

VALUE as a person reads it: three figures, no trailing zeros, thousands separated.

Effects: fail.

`format`

gizlang

function format(value: f64, unit: string): string

VALUE with its UNIT: 302,589 req/s, 21 ms, 53%.

Effects: fail.

`shown`

gizlang

function shown(v: Value, unit: string): string

What a cell shows: its text when it has one.

Effects: fail.

`encode`

gizlang

function encode(p: Published): string

The text of a published file (two-space indent, so a diff shows one row per change).

Effects: fail.

`decode`

gizlang

function decode(text: string): Published throws string

The published file in TEXT.

Effects: fail.

`merged`

gizlang

function merged(base: Published, run: Published): Published

BASE with the rows of RUN: a row of an item that RUN measured is replaced in its place, a new one is added at the end, the others stay. The title, subjects, groups and goals are RUN's: the suite program owns them.

Effects: fail.

`machine`

gizlang

function machine(sys: System): string

The machine a run is on, as a published row states it: AMD Ryzen 9 7945HX (32 threads), 64 GB, Linux 6.17.

Effects: fail, io.

`published`

gizlang

function published(sys: System, dir: string, id: string): Published throws string

The published file of the suite ID in DIR (the folder that holds AREA/NAME.json), if there is one.

Effects: fail, io.

`main`

gizlang

function main(sys: System, suite: Suite): i64

The command line of a suite program. Answers the exit status.

Effects: fail, read, write, console, io.

Methods

`Row.value`

gizlang

function Row.value(self: Row, subject: string): Option<Value>

The value of SUBJECT in the row, if the row has one.

Effects: fail.

`Published.row`

gizlang

function Published.row(self: Published, id: string): Option<Row>

The row of item ID, if P has one.

Effects: fail.

`Published.name`

gizlang

function Published.name(self: Published, subject: string): string

The name of SUBJECT in P.

Effects: fail.

`Published.group`

gizlang

function Published.group(self: Published, id: string): Group

The group ID of P (its title and method); an empty group when P has none of that id.

Effects: fail.

Constants

`SCHEMA`

gizlang

const SCHEMA = "gizlang.bench.v1"

The schema name that each published result file carries.