GizStack
View .mdOpen in ChatGPTOpen in Claude

Gizlang language reference

Gizlang source files (.giz) use the spelling of TypeScript. The meaning is the meaning of Gizlang. A familiar spelling does not give JavaScript semantics. Unsupported TypeScript does not compile. It never gets a different meaning.

The tests in tests/ are the executable form of this reference. If you know TypeScript, read TYPESCRIPT first. It shows on one page what you write differently.

Contents

  1. Declarations
  2. Comments, docs, annotations and aliases
  3. Statements and expressions
  4. Markup
  5. Errors
  6. Types and values
  7. SIMD
  8. Standard library
  9. Tests
  10. Effects and contracts
  11. Not in the language
  12. Diagnostics

Declarations

gizlang
const ANSWER = 40;                      // module constant
export const SCALE: f64 = 1;            // annotated constant

export type Point = { x: f64; y: f64; };
type Shape =
    | { kind: "circle"; radius: f64; }
    | { kind: "square"; side: f64; };

interface Area { area(self: Self): f64 effects {}; }

function Point.length(self: Point): f64 { return sqrt(self.x * self.x + self.y * self.y); }

export function main(): i64 { return ANSWER + 2; }

Module constants

  • A module constant names one literal. The literal is a number (i64, u64 or f64, with an optional minus sign), a bool, a string or bytes. A constant that is an arrow function (const inc = (x: i64) => x + 1;) is a function declaration instead.
  • Each use of the constant is that literal. A use of a string or bytes constant is a new value.
  • A constant without an annotation stays untyped, as the literal is. An annotation gives it a type.
  • Any other initializer is the error E_CONSTANT.

Records

  • A record is an immutable value with a fixed layout. .giz refuses TypeScript's readonly (it would say nothing); a .ts file keeps it.
  • A record cannot contain itself by value.
  • Make a record with a literal, { x: 1, y: 2 }, where the expected type is known, or with its type as a constructor, Point(1, 2), which takes every field in declaration order. Both give the same value. Use the literal when a reader needs the field names (more than three fields, or fields of one type); the constructor is for short records and annotations.
  • A field name can be a reserved word, as in TypeScript: { type: string; if: bool; }, ref.type and { type: "doc" }. A shorthand field or pattern binding ({ type }) reads or binds a variable, so a reserved word is refused there (E_NAME).

Unions

  • A string field kind discriminates the cases of a union.
  • u.kind === "name" (or !==) tests one case. The compiler checks the name against the union.
  • Make a value with Shape.circle(1.0), or with an object literal that has an expected type.
  • A generic union gets its type arguments from the expected type or from the matched value: return Result.ok(5);, Result.err(e) => ....
  • The cases share storage, as Rust enums do. A union takes the size of its tag and its largest case.
  • A pattern binding of a place that no arm writes borrows its field.

Private fields

A field with the name _name belongs to the module that declares its type. Other modules cannot read the field, make a value with it, or bind it in a match. Thus a module can give out values (capabilities, handles, validated data) that only its own functions can make.

Interfaces

  • An interface is a structural set of methods. Each method has an explicit self: Self.
  • Use an interface as a generic bound (<T extends Area>). Records, scalars, strings and bytes can implement an interface.
  • An interface name can also be a type: const shapes: Vec<Area> = [circle, square]. The value holds any implementation. Call its methods as s.area(). This is possible only when an immutable self is the only Self in the interface.
  • External mut parameters borrow the values of the caller. mut Self requires a generic bound, or receiver state in a Ref.
  • Generics stay specialized. There is no inheritance; a class is a record in a Ref with functions (Classes).

Classes

The parser lowers a class onto the forms above (compiler/parser.giz parseClass):

  • The fields, and the constructor's parameters that start with public, private, protected or readonly, make the record of the class's name. A field without an annotation takes its literal initializer's type.
  • The class's name written as a type is Ref<Point>: an instance is an object that every name, argument and element of it shares, as in TypeScript. p.x reads _ref_load(p).x. A statement that changes a place in it takes the record, changes it and stores it back (_ref_take, _ref_store), after its values are computed. a === b is _ref_same(a, b), the same object.
  • The constructor is the function constructor of a namespace of the class's name beside the record; new Point(...) calls it. Its body runs in order; the statement this.f = e; at its top level, for each field with no initializer, gives the field e, and this (_ref_create of the record, made there of those values in their order, then the field initializers) exists from the last of them. It returns this. A field that gets no value is E_TYPE.
  • A method is function Point.m(this: Ref<Point>, ...). A getter get area() is a method with no parameter, which a read of p.area calls; a setter set area(v) is the method set area, which p.area = v calls.
  • A static method is a function of the namespace (Point.origin()); a static field is a constant of it, so its value is a literal.
  • A generic class is generic in its record, constructor and methods. extends and abstract are E_PARSE. implements names interfaces; the record implements them by its methods.
  • A record (type) stays a value. A change of a copy that TypeScript would share is refused (E_BORROW, naming a class as the fix): a const copy (const b = a; b.x = 1;), or a let assigned another local (m = a;) and changed while that local is read after.

Enums, literal unions and tuples

  • enum Color { Red, Green = 5, Blue } is the alias type Color = i64 and a namespace Color of constants: a member without a value is one more than the member before (0 first, at most 255). A string enum (Up = "UP") is a string. Members are literals, as module constants are; const enum is the same. Enums of one name in two modules are two namespaces; a use takes the one of its module, or of the module its import names.
  • A union of literals ("a" | "b", 1 | 2, true | false), which may start with |, is the type of its literals. Literals of two types in one union are E_TYPE.
  • [A, B] to [A, B, C, D, E, F] (items may be labeled) are gizlang:core's Tuple2 to Tuple6, records of first, second, ... sixth. [a, b] where a tuple is expected makes one; t[i] with a literal i reads item i; const [a, , c] = e; (or let) binds the items of a tuple or the cells of a Vec, a hole skipping one.

Module state

A module-level let x: T = e; is state of its module (parser parseModuleLet). Without an annotation, e is a literal and gives the type. A read of x in a function of the module is gizlang:core's moduleGet, and x = v (so x += 1 and x++) is moduleSet: the state is bound to the name module:x, and e runs at its first use. It changes only by assignment; a changing method call on it is refused, as on any value that is not a place. Another module reaches it only through a function of its module (E_MODULE). The state lives as long as the World; under giz live it keeps its value across generations.

Async functions

async function f(): Promise<T> and async (x) => ... run to their end when they are called: Promise<T> is T and await e is e. Concurrent work uses gizlang:task.

Functions

  • Declare a function as function f(x: T): R.
  • A receiver function has the form function Point.m(self: Point). Receiver functions are also possible on scalars, string and bytes (function i64.double(self: i64)), and on generic types (function Box.count<T>(self: Box<T>)) and every Vec (function Vec.m<T>(self: Vec<T>)). Call them as x.m(). In a generic function, x.m() on a value of a type parameter calls the receiver function of the type it is given.
  • A function declared : void has no result. return; ends it early. The end of its body also returns. A call of a void function is a statement.

mut parameters

A mut parameter (function add(v: mut Vec<i64>, x: i64), self: mut Bag) is a let local of the caller, or a field or cell of one (add(o.items, 1)). The function changes it in place. For example, add(v, 1) makes v longer without a copy.

The compiler applies these alias rules:

  • No other argument of the call can read the part that the mut argument changes.
  • The call reads a scalar argument before the call starts, so add(v, v.length) is correct.
  • Fields with different names are separate, so add(p.left, p.right.n) is correct.
  • f(v, v), f(p.left, p) and two cells of one Vec are refused with E_BORROW.
  • Two mut arguments cannot be aliases. This rule includes scalar places.
  • When the mut argument is a whole local, another argument can be a call that reads or changes that local. That call runs to its end before the outer call starts, and its result holds no view. Thus g.add(g.matmul(x, w), b) means exactly const t = g.matmul(x, w); g.add(t, b);.
  • Function values keep these rules. mut is part of their parameter type ((v: mut Vec<i64>) => void). Adapters forward the places of the caller without a copy.

Generics

  • The compiler specializes generics statically. Instantiations nest to a maximum depth of 128.
  • A method on any record can have its own type parameters: function Pool.with<F, E>(self: Pool, work: F, failure: (error: PgError) => E): ReturnType<F>.
  • Type arguments come from the arguments. An object literal or an empty [] gets its type from the other arguments, as in TypeScript: m.set("k", { n: 1 }), Test.equal(names(), []).
  • When the arguments fix none of a call's type arguments, they come from the type its value is expected to have: const m: Map<string, f64> = newMap();, { items: newMap() } for a Map field, return newMap();.
  • A type argument can be a non-negative integer literal: Boxed<0x78>. The value and its value type (120 is i64, 120u is u64) are part of the type identity, so Boxed<0x78> and Boxed<120> are one type, and Boxed<7> is another. In the generic body, the parameter is that integer as a constant: function Boxed.tag<MASK>(self: Boxed<MASK>): u64 { return self.word & u64(MASK); }. An integer type argument is not a value type: Holder<3> for { value: T } is E_TYPE.

Modules and packages

  • Imports can be named, default or namespace imports, with aliases. Re-exports can be named, star or export * as.
  • A namespace is a static binding. It is not a value.
  • A call x.m() finds a receiver function only in the modules that the file imports, directly or through them. import {} from "./more.giz"; imports a module only for its receiver functions, such as the methods that it adds to a type of another module.
  • import ... from "gizlang:memory" selects an SDK module.
  • A package may pin the sha256 of each own source in the gizlang section of package.json. A pinned source must match its pin (E_PACKAGE_INTEGRITY). An unpinned one is read as it is: its folder's owner (git, an editor) already owns its bytes. A dependency's sources are always pinned.
  • The dependencies of a package are package folders inside it ({"geo": {"path": "vendor/geo", "sha256": ...}}). The sha256 of each dependency's package.json pins it, and that file pins its sources. Thus one hash fixes the whole closure, and two versions never meet as one name.
  • "geo:shapes" imports the module shapes of geo. Inside geo, its own modules and dependencies have names as from its root. The compiler does not load the entry of a dependency.

Registry requirements

  • "requires": {"geo": "1.2.0"} names the lowest versions that a package needs from a registry.
  • The registry is $GIZLANG_REGISTRY. It is a folder of NAME/VERSION packages, or that folder served as is over HTTP. A version is immutable after giz publish.
  • giz get gets the package.json of a version, then each source that it pins. It checks each source against its sha256.
  • giz get DIR [geo@1.3.0] raises requirements. For each name it selects the highest version that the requirement graph asks for (minimal version selection: the build changes only when a requirement changes). It puts each selected version in the vendor/ folder of each package that requires it, and refreshes the pins.
  • A new major version is a new name.
  • npm resolves npm:NAME@SPEC items. giz get runs npm install in DIR. npm records them in the dependencies and lock file of that package.json, and installs them in DIR/node_modules, where Vm.javascript(sys).load("NAME") finds them (SYSTEM).

Comments, docs, annotations and aliases

gizlang
/** Stores the record in a table. */
export type Table = { name: string; };
export type Key = {};                   // a marker: only annotations use it
type Ids = Vec<i64>;                    // an alias: the same type, another name

/**
 * A person, kept in `people`.
 * @see Table
 */
@Table("people")
export type Person = {
    /** Its key. */
    @Key
    id: i64;
    name: string;
    tags: Ids;
};

/**
 * The person's name, trimmed.
 * @param self - the person
 * @returns the name without surrounding spaces
 * @throws nothing: the Result's err is never made
 * @example
 * ```
 * @Table("x") // a fence keeps @ lines inside the example
 * person.name()
 * ```
 * @deprecated use Person.label
 */
export function Person.name(self: Person): string { return self.name.trim(); }

Comments

The comment syntax and style are those of TypeScript: // and /* */ are comments, and /** */ is a doc comment in TSDoc style. A comment can be wherever TypeScript allows one, and it has no meaning. A /** */ that is not immediately before a declaration (before an import, in a body, at the end of a file) is a plain comment.

Doc comments

  • A doc comment has one form, the TypeScript /** ... */. It is immediately before a declaration, a field, a union case or an interface method. Only white space can be between them.
  • /*** and /**/ are plain comments.
  • The leading * of each line is not part of the text.
  • The summary continues until the first line that starts with a tag.
TagMeaning
@param NAME TEXTA parameter. An optional - can follow the name. Can repeat.
@returns TEXTThe result.
@throws TEXTThe meaning of the err case of the Result.
@exampleAn example. A ``` fence keeps @ lines inside the example. Can repeat.
@deprecated REASONA use of the declaration gives the warning W_DEPRECATED from giz check and the LSP. It never makes a build fail.
@see NAMEA related declaration. Can repeat.
@was NAMEOn a record field: an earlier name. Live state and stored tables (follow in db/typed.giz) follow the rename. The field rename of giz edit writes it.
@was MODULE.NAMEOn a function (or NAME from the entry module): its location before a move. Under giz live, the calls of a running program reach it. The move of giz edit writes it.

These are the error E_DOC, so docs cannot drift from the code:

  • An unknown tag.
  • A repeated tag, other than @param, @example and @see.
  • An @param that names no parameter of the documented function.

A program reads the fields of a record type, with their earlier names, as fieldsOf<T>() (gizlang:memory). A program reads the whole declaration of a non-generic record or union type, with its docs, annotations, fields or cases and their types, as Memory.describe<T>() (gizlang:memory): the JSON that giz outline --json gives for that type, fixed at compile time, except that a type names no module path and an annotation argument that is not a string is a string of its source token (an integer keeps every digit). The description ends with "types": the same description of every record and union that the type reaches through its fields, cases, vectors and references (each once, a generic type through its arguments), so a program reads a nested type without a second call. A program turns a value of a record or union type into words and back as Memory.flatten(value) and Memory.unflatten<T>(words): each node is its layout kind and its contents (runtime/value_layout.c), so a program reads any record without generated code.

Annotations

  • An annotation @Type(literal, ...) (or @Type for a record with no fields) applies to the same targets as a doc comment. It comes after the doc comment.
  • The compiler checks each annotation as a call of the constructor of that record, with scalar literals (E_TYPE, E_CONSTANT).
  • An annotation does not change what a program does. Tools read it.
  • A record with no fields exists only for annotations. To make or use its value is E_TYPE.

Tool output

  • giz outline gives the doc of each declaration ({summary, params:[{name,text}], returns, throws, deprecated, examples, see}, with keys only for the tags that are present) and its annotations ([{type, arguments}]). It also gives those of fields and union cases.
  • giz project prints docs and annotations back. Thus the projection stays a fixed point.

Aliases

type Ids = Vec<i64>; names a type. An alias cannot be generic. The two names are the same type.

Statements and expressions

Bindings and assignment

  • const bindings are immutable. let bindings are mutable.
  • x op= y means x = x op y, for + - * / % & | ^ << >>. The compiler evaluates an index only one time when it is not a plain name or literal. Thus counts[i64(b)] += 1 calls i64 one time.
  • Assignment and the statement .push(x); work at any path of fields and cells in a let local: o.name = n, o.rows[i][j] = x, ps[0].x = 1, o.items.push(x). They change only the value of that local, in place. Copies made before keep their values. On a parameter that is not mut, or a const that stays a const (TYPESCRIPT), both are refused: v.push(x) with E_IMMUTABLE, and p.items.push(x) with E_BORROW. The longer Vec would be lost.

Control flow

  • if (...), else if and while (...).
  • for (const i of range(lo, hi)), for (const x of v) over any Vec expression, and for (init; condition; update).
  • return, break and continue. break and continue apply to the innermost loop of the same function. Owned locals bound after the start of the loop are released on the way out.
  • The exhaustive statement match (value) { ... }:
gizlang
type Shape = | { kind: "circle"; radius: f64; } | { kind: "square"; side: f64; };

export function area(shape: Shape): f64 {
    match (shape) {
        Shape.circle(radius) => { return 3.0 * radius * radius; }
        Shape.square(side) => { return side * side; }
    }
}

Each arm covers one case, one time, and binds its whole payload. Name a union from a namespace import through the namespace: Text.IntegerParse.ok(n).

Closures and function values

  • Arrow functions (x: i64): i64 => { ... } capture immutable values by value. A let that one names is a shared cell (a Ref): the function and its lambdas see each other's changes. They cannot capture views or mut parameters.
  • Named functions are values too.
  • A function type (x: i64) => i64 (=> void for no result) is a value type. A field, a Vec element, a parameter or a result of that type holds any closure or named function with that signature: const fs: Vec<(x: i64) => i64> = [double, (x) => x + 1]. s.area() calls a field.
  • An untyped lambda gets its parameter types from the function type.
  • A call through a function value can have any effect, unless the type declares a row: (x: i64) => i64 effects {fail}. Then calls through it have only those effects, and a function that becomes such a value must stay in that row (E_EFFECT).
  • A closure that holds a Ref keeps the Ref live while the function value lives. An interface value over a record with a Ref is such a closure.
  • Generic parameters (f: F) stay specialized for each closure.
  • A plain function type fn (x: i64) => i64 holds only code: a named function or a lambda that captures nothing. It has no count, no root and no trace, so a hot call through it costs one indirect call. A lambda that captures a value is E_TYPE there. A plain type and the function type of the same signature are different types.

Other expressions

  • value satisfies Type checks an object literal against a type.
  • An object literal that no type expects (const p = { x: 1, y: 2 };) has the record type of its field names and value types. Two literals with the same fields in the same order have the same type.
  • In the object literal of a record, ...r gives every field that is not written from r. r is a name, field or element of that record type: { ...item, stock: item.stock + 1 }, { ...order.lines[i], qty: 2 }. Without an annotation, the literal has the type of r.
  • An operator on a record is its receiver function of that name: a + b is a.add(b), - * / are sub mul div, < <= > >= are lt le gt ge, unary - is neg. === stays content equality. A number next to a record that is a name lifts through its k: x * 2.0 is x.mul(x.k(2.0)), 1.0 - x is x.k(1.0).sub(x). Next to any other expression it is E_TYPE, since lifting would evaluate that expression twice: name it first. Tiles, tensors and your own vector or decimal types read as math this way (tests/language/functions/record-operators.giz).
  • a ** b (and a **= b) is a.pow(b) on every type, right-associative and tighter than *: 2 ** 3 ** 2 is 512. f64.pow is libm pow. i64.pow multiplies exactly: b must be at least 0, and an overflow fails as * does.

Markup

Markup (TSX) is an expression of type Html from gizlang:html. Write it as in React:

gizlang
import {Html} from "gizlang:html";

type Todo = { id: string; title: string; done: bool; };

export function item(t: Todo): Html {
    return <li class="todo" data-k={t.id}><input type="checkbox" checked={t.done}/>{t.title}</li>;
}

Holes and attributes

  • A child hole {x} takes text (escaped), i64, f64 or Html (Content).
  • An attribute hole takes text or a number ( name="..."), or a bool (AttributeValue). A bool gives the bare name when true and nothing when false.
  • Any other type is E_PROTOCOL at the hole.
  • Quoted attributes and {"..."} children are fixed text. The compiler escapes them at compile time. In a quoted attribute a backslash is a character, as in HTML (pattern="[a-z\-]"); only a backslash before the quote is an escape. A {"..."} child is a string literal, so its escapes apply.
  • Element text is HTML as written: entities stay. The whitespace rule of JSX applies: a line break and the spaces around it go.
  • <br/> and the other void elements have no closing tag. <>...</> is a fragment.

Components

  • A tag that starts with an upper-case letter or has a dot (<Card title="x">...</Card>, <Ui.Card/>) calls that function. The attributes are one record argument. The children, if any, are one Html argument.
  • A bare attribute on a component is E_PARSE.

Helpers

There is no spread, && or ? : in markup. Use these helpers:

HelperUse
when(c, () => <b/>)Content only when c is true.
each(items, (x: T) => <li>{x}</li>)Content for each item.
raw(trusted)Trusted HTML, not escaped.
empty()No content.

Cost

  • The compiler joins the fixed parts of each element at compile time. It writes the holes into one buffer in order. Thus an element costs the same as to write its bytes by hand.
  • Markup must be in a function body. A module constant cannot hold it (E_PARSE).
  • out += <li>{x}</li>; on a bytes local or mut parameter writes the element directly into out, with no Html value between. A writer function (function row(out: mut bytes, t: Todo): void) costs the same as to write its bytes by hand. A component that returns Html costs one more buffer.
  • giz project prints markup as written.

Errors

A function that can fail says so in its signature: function half(n: i64): i64 throws string. It returns a Result<i64, string>, and its callers take the value (i64) of a call: a failure returns from the caller as the same failure, so a caller that throws the same type needs no code for it.

gizlang
function half(n: i64): i64 throws string {
    if (n % 2 !== 0) { throw "odd"; }
    return n / 2;
}
function quarter(n: i64): i64 throws string { return half(half(n)); }
function describe(n: i64): string {
    try { return `${quarter(n)}`; } catch (e) { return `failed: ${e}`; }
}
  • throw e; is return Result.err(e); in a function that throws. In a try block it leaves the block.
  • try { ... } catch (e) { ... } runs the handler when a call that throws, or a throw, fails in the block. e is the failure; its type is that of the first failure of the block, or the one written (catch (e: string)), and every other failure of the block has it (E_TYPE). break and continue in the block leave the loop around it.
  • try { ... } finally { ... } (also after catch) runs the finally block when the statement ends: after the block or the handler, on return, break and continue that leave the statement, and before a failure goes on to the caller. A return v; holds v before the block runs. Where nothing can take a failure that the block lets pass (a function that does not throw, with no try block around), it is a fault (R004).
  • return v; in a function that throws returns Result.ok(v). : void throws E returns no value.
  • A call that throws is a fault (R004, as x!) when its caller does not throw and no try block catches it. A thrown string is said on stderr after the fault's trace (gizlang: thrown odd 7).
  • A call that throws is the Result itself where a Result is expected:
    • a typed const (const r: Result<i64, string> = half(n)), an argument (a generic Result<T, E> one too), or a return of a function that returns a Result;
    • a match subject whose arms are the cases of Result, the operand of try, and f().kind === "err";
    • an untyped const r = half(n) that the rest of its block reads as a Result (match (r), r.isOk(), r.kind, or return r; in a function that returns a Result), or as try r, r! or r.unwrap() where the value is not an Option;
    • the value of a lambda whose result type its returns give: a lambda cannot throw (Task.spawn(() => read())). A method that T lacks and Result has reads it too: f().valueOr(x), f().mapErr(g), f()!. A function value and an interface method keep their Result.
  • A generic function that throws throws in each instance: half(n).mapErr(g) is the value, as half(n) is.
  • A Fault (gizlang:task: Task.join and Thread.join of a task or thread that faulted) is a fault again where the function does not throw Fault, and try then applies to the joined value: in a function that throws E, try Task.join(t) of a task whose value is a Result<T, E> is T, and the task's failure goes on to the caller. So is try r of const r = Task.join(t); where the code reads r as a Result (match (r), r.isOk()), it is the join's, whose err is the Fault. A member of the join itself is the join's too: Task.join(t).isOk() tells whether the task ended without a fault, and Task.join(t).unwrap() is its value.
  • void f(); runs the call and drops its value, as in TypeScript. A failure of a call that throws is dropped too: void sys.remove(path); ignores a file that is not there.

A union whose first case holds the value, Result<T, E> or Option<T> from gizlang:core, is a value as any other. try e unwraps that first case. Any other case of e returns from the function. It becomes the case with the same name in the declared result type. That case must have the same payload types.

gizlang
import {Result} from "gizlang:core";

function divide(a: i64, b: i64): Result<i64, string> {
    if (b === 0) { return Result.err("division by zero"); }
    return Result.ok(a / b);
}
export function ratio(a: i64, b: i64): Result<i64, string> {
    const q = try divide(a, b);           // returns Result.err(...) when divide fails
    return Result.ok(q * 100);
}
  • Error unions widen by case name. try on | {kind: "ok"; ...} | {kind: "empty"} works in a function that returns | {kind: "ok"; ...} | {kind: "empty"} | {kind: "range"; ...}.
  • The early return runs ensures, as every return does. Contracts cannot contain try.
  • Faults (overflow, bounds, assert) are not values. They stop the call. Refer to Diagnostics.

Types and values

TypeMeaning
i64, u64Checked integers. Overflow, division by zero and out-of-range shifts fail. wrapping_add, wrapping_sub and wrapping_mul are modular.
f64 (number)Strict IEEE double. There is no fast-math.
i8 to i32, u8 to u32, f32Storage widths for record fields, union fields and Vec elements. A Ref can hold records of them. Ref<u8> alone is an error. A read widens to i64, u64 or f64. A write that does not fit fails (R003, as for a byte of bytes). An f32 write rounds. Locals, parameters and results are 64-bit. The exception is a native function, which uses its C types (SYSTEM).
bool (boolean)True or false.
stringValid UTF-8. len and indexes count bytes. An indexed byte is a u64. Literal escapes are \n \r \t \b \f \" \' \\ \/, \uHHHH (a surrogate pair gives one scalar) and \xHH (U+00HH, as in TypeScript). \' is only in a '...' literal. Any other escape is E_LITERAL.
bytesImmutable bytes. Write literals as b"..." in ASCII. Write any other byte as an escape: b"\xff" and b"\u00FF" are byte 255, and each escape is one byte. Thus a raw b"ÿ" and b"\u0100" are E_LITERAL.
Vec<T> (T[])An immutable vector value. push returns the longer vector. map allocates one time. v.map<T>(callback) selects the element storage explicitly: callback values widen as usual, then each write checks the integer range or rounds to f32. Without a type argument, the value type of the callback sets the elements.
View<T>, MutableView<T>Numeric borrows, for parameters only. They never escape. Mutable views cannot be aliases.
Ref<T>Managed identity on the precise heap (gizlang:memory). The heap collects when allocation has doubled since the last collection. collect() collects immediately.

Value semantics and memory

  • Strings, bytes and vectors are values. A change to one binding never changes another binding.
  • They are reference-counted and copied only when shared. Thus v = push(v, x) and s = s + t on a let local append in place, in amortized constant time.
  • Memory is released when its last owner goes away. There is no garbage collector.

Built-in functions

Built-inMeaning
len(x), x.lengthThe bytes of a string or of bytes, or the elements of a Vec or view.
s.byteLength, s.byteAt(i), s.byteSlice(a, b), s.byteIndexOf(x, from), s.byteLastIndexOf(x)A string's UTF-8 bytes: their count, byte i (a u64), the text of bytes a to b (R001 for a split character), the byte offset of x.
slice(x, start, end), x.slice(start, end)A copy of the bytes or elements start..end. R001 for a bad range or a split UTF-8 character.
push(v, x)v with x added at the end. For bytes, x is a u64 byte (R003 above 255).
filled(n, x)A new Vec of n copies of a scalar x (R001 for n < 0), set as one block fill. Where bytes are expected, it gives n copies of the byte x: let out: bytes = filled(n, 0u), then write by index, as C writes a buffer. A zero x costs only the pages that the program uses.
with_capacity(n, x)An empty Vec of the type of the scalar x with room for n cells (R001 for n < 0): n pushes, or appends of n cells in all, never grow it. Where a narrower Vec is expected, it gives that Vec: let codes: Vec<u32> = with_capacity(n, 0u).
a + b, a += b on VecsA Vec of the cells of a then of b, for a Vec whose cells hold no buffer (numbers, bools, records of them). One memcpy of b; a += b on a local or a field appends in place when no one shares a. A list literal on the right takes the type of a.
mul_high(a, b)The high 64 bits of the 128-bit product of two u64 values. With wrapping_mul, this gives the whole product.
f64_bits(x), f64_from_bits(u)The IEEE bits of an f64 as a u64, and back.
f32_bits(x), f32_from_bits(u)x rounded to f32 (as an f32 field stores it), as its 32 bits in a u64, and back. R003 if u has more than 32 bits.
trailing_zeros(u), leading_zeros(u), count_ones(u)The bit counts of a u64, as i64. 0 has 64 zeros.
===, <, ... on strings and bytesBytewise comparison. For strings, this is code point order.
string(x)Valid UTF-8 bytes as a string (R003 otherwise), or a number or bool as text. An f64 gives the text of JavaScript String(x): the shortest digits that read back exactly (0.1, 1e+21, 1e-7, NaN, Infinity).
print(text)Writes text and a newline to the console. The effect is console.
printError(text)As print, on standard error. console.error and console.warn call it.

Conversions and equality

  • There are no implicit conversions. Use i64(x), u64(x) or f64(x).
  • i64 and u64 of an f64 truncate toward zero. NaN or a value out of range is fault 3.
  • Equality is strict and typed.
  • === compares the content of records, unions and Vecs: fields, the active case and its fields, and items. This also applies to a recursive type.
  • These types have no ===: a record with hidden fields (_name, for example Map, whose layout can differ for equal contents), views and functions.
  • === on Ref values (class instances) compares identity: the same object.

SIMD

A SIMD type holds lanes of one width in one value. Its name has the form <i|u|f><bits>x<lanes>, for example u8x32, i32x8 or f64x4.

  • The lane width is 8, 16, 32 or 64 bits. Float lanes are 16, 32 or 64 bits.
  • The lane count is a power of two from 2 to 64.
  • The total size is a maximum of 2048 bits.
  • Simd<T> is 512 bits of T. For example, Simd<f32> is f32x16.
  • SIMD values are locals, parameters and results. A record field, a Vec item or a Ref cannot hold one.

The compiler does not know the CPU. The same code runs as AVX-512 on a CPU that has it, and as AVX2, NEON, SSE or scalar code on other CPUs.

Every build is for any x86-64 or ARM64 Linux with glibc 2.35 or later. No build is for the CPU of the machine that builds it, so a program built on one machine runs on all others. On x86-64, each function that uses SIMD types also has copies for x86-64-v3 (AVX2, FMA) and v4 (AVX-512), and its first call takes the copy for the highest level that the CPU has. All other code is compiled once, for the baseline. GIZLANG_CPU=x86-64|v3|v4 selects one level. A -O0 program (--fast, tests) has only baseline code of its own, and the SDK kernels that it links have their copies. On ARM64, the baseline (ARMv8-A) has NEON.

FormMeaning
T(x), T([a, b, ...])x in every lane, or the given lanes. Give as many lanes as T has.
T(v)Another SIMD value, lane by lane, when the lane counts are equal. Integers wrap. Floats to integers saturate, and NaN becomes 0. When only the sizes are equal, its bits (u8x16(u64x2(...))).
T(data, i)The lanes stored from i. data is bytes or a string (little-endian bytes from byte i), a Vec of the lane width, or a view of 64-bit lanes (from element i). A range past the end is fault R001.
data[i] = vStores the lanes of v from i into a let bytes or Vec, or into a mutable view.
v[i], v[i] = xOne lane, widened as a storage width is. The second form sets a lane of a let local. R001 out of range.
+ - *, / (float lanes), & | ^ ~ << >> (integer lanes), unary -Lane by lane, with wrap. A scalar of the lane value type on either side applies to every lane. A shift uses its count modulo the lane width. >> is arithmetic on signed lanes.
=== !== < <= > >=A mask: the unsigned type of the same shape, with all bits set where the comparison is true.
select(mask, a, b), any(m), all(m), bitmask(m)The lane of a where the mask lane is not zero, else the lane of b. Whether some or all lanes are not zero. The top bit of each lane as a u64, with lane 0 lowest.
shuffle(a, [k, ...]), shuffle(a, b, [k, ...])The lanes at constant indexes of a, then of b. The result has as many lanes as indexes.
lookup(table, i)For a table of any lanes of any type, its lane at each lane of i modulo its lane count, with as many lanes as i. One instruction: a u8x16 table with any index pshufb or NEON tbl; a table that fills one register and is as wide as the index vpermb (u8x64, AVX-512 VBMI, else lane by lane, so prefer 32 lanes there), vpermw (u16x32), vpermps (f32x16, a 4-bit codebook) or vpermd. Two u16x32 tables of f16 bits and a select make a 64-entry codebook.
reduce_add, reduce_min, reduce_maxOne lane value. A float sum can add in any order.
min, max, abs, fma, saturating_add, saturating_sub, floor, ceil, roundLane by lane. round rounds to even. Float min and max ignore a NaN. fma, floor, ceil and round also take an f64.
len, sqrt, wrapping_add/sub/mul, mul_high, trailing_zeros, leading_zeros, count_onesThe scalar built-ins, lane by lane. len is the lane count.
prefetch(data, i)Starts to load cell i of a Vec, bytes or view into the cache, for a walk that reads it soon (llvm.prefetch). It never faults, also past the end. True.
gizlang
function digits(text: bytes): i64 {   // ASCII digits, 32 bytes a step
    let count = 0;
    let i = 0;
    while (i + 32 <= text.length) {
        const v = u8x32(text, i);
        count += count_ones(bitmask((v >= 48u) & (v <= 57u)));
        i += 32;
    }
    while (i < text.length) { if (text[i] >= 48u && text[i] <= 57u) { count += 1; } i += 1; }
    return count;
}

Standard library

Import an SDK module by name: import {join} from "gizlang:text";. The SDK modules are written in Gizlang, unless this section says otherwise. Tasks, files, the network, native libraries and live state are in SYSTEM.

The module gizlang:NAME is the file sdk/LAYER/NAME.giz, as sdk/package.json gives its path. The layers are packages of PACKAGES, and a module imports only its own layer and those below it:

LayerHolds
sdk/core/Language values, text, collections, system, tasks, time, JSON, crypto, compression, regex, locale
sdk/format/CSV, HTML, PDF, images, protobuf, journal, erasure codes
sdk/net/HTTP 1-3, TLS, gRPC, mesh, gateway, ACME, metrics, the model hub client, workers
sdk/data/PostgreSQL, SQLite and ODBC clients, ublk

A package outside the SDK names the layers it imports in its depends.

Hardware and models are toolchain libraries, imported by their own names as GizDB is (COMPILER "Library modules"). Their sections below use those names.

LibraryFolderHolds
gizhw:NAMEhw/The machine, devices and their readings, GPUs and GPU kernels, NPUs, cameras, audio, video codecs
gizml:NAMEml/Tensors and network operations, autograd, networks, classical ML, tokenizers, LLM, diffusion, video, ONNX, the AI client

gizml imports gizhw and the SDK; gizhw imports only the SDK. An import prefix says where a module resolves: gizlang: is the SDK, and a library name is a toolchain library, which no dependency can take.

gizlang:core

  • Result, Option, valueOr and unwrap. r.valueOr(y) gives the value of the Result r, or y. unwrap gives the value, or fault R004. x ?? y gives the value of the Option x, or y, which runs only when x is none.
  • assert(false) ends a path, as return does.

gizlang:text

String methods:

  • indexOf, includes, startsWith, endsWith, split, replaceAll, repeat.
  • trim, trimStart, trimEnd, padStart, padEnd, lastIndexOf.
  • toUpperCase and toLowerCase. These change ASCII letters only.

Functions:

FunctionMeaning
join(parts, sep)Joins strings.
parseI64A union: IntegerParse.ok(n), invalid or overflow.
parseF64An Option.
decimalPrefix(data, start, end)The number at a byte range, read one time with eight digits for each step. Gives its value, its end, and whether it is valid number text or strict JSON. parseF64, csv and json share it.
readU16LE/BE, readU32…, readU64…, readF32…, readF64…Binary fields at an offset, unsigned.
writeU16LE(out, value) … writeF64BEAppend binary fields.
packI64s(out, v), packF64s, packU32s, packF32s; unpackI64s(data, at, n) … unpackF32sAppend or read a whole little-endian array, 64 bytes a SIMD step: the same bytes as one write…LE for each value.
signed(u, bits), unsigned(i, bits)Two's complement conversion.
writeF64(out, x)Appends the shortest digits of string(x) without a temporary string (Schubfach).
fixed(x, digits)The %.*f of printf: exact, with ties to even.
radix(n, base)n in base base.
base64Encode, base64DecodeBase64. The runtime uses AVX2 when the CPU has it.
base64UrlEncode, base64UrlDecodeUnpadded URL-safe Base64. Decoding refuses noncanonical text.
hexEncode(bytes)Lowercase hexadecimal.
hexDecode(text)Option<bytes>. It accepts upper and lower case and whole bytes, also empty input. It rejects odd widths, white space, prefixes and non-hex characters before it allocates.
uvarintSize, writeUVarint, readUVarint(data, offset, bits)Unsigned LEB128.
varintSize, writeVarint, readVarintZigzag signed LEB128.

The varint readers return Result<{value, next}, string>. They reject truncation, invalid offsets and values larger than the declared bit width (1 to 64). They accept existing encodings that are not minimal but are in range. Offsets and next are byte positions. The readers do not allocate a slice and do not silently narrow a value.

gizlang:path

Pure POSIX string operations: join(parts: Vec<string>), normalize, dirname, basename, extension and isAbsolute.

  • normalize collapses slashes and . segments, and resolves ... It keeps leading relative .. segments, and stops at / for absolute paths. It removes trailing slashes. An empty result becomes ..
  • join ignores empty parts and normalizes their concatenation. A later absolute part does not reset the path.
  • dirname and basename ignore trailing slashes. They do not resolve dot segments. The dirname of an empty or bare name is .. The basename of / is empty.
  • extension includes the last dot. It is empty for a leading dot and for the names . and ...
  • Backslashes are ordinary characters.
  • These functions do not access files. They do not relax the path restrictions of a Dir.

gizlang:unicode

Unicode 17.0.0. The tables are generated from the UCD (tools/unicode).

  • codePoints(text): Vec<i64> and fromCodePoints(points): string.
  • appendCodePoint(out, scalar): void. This is the shared UTF-8 encoder. Scalars exclude surrogates.
  • Full case mapping with toUpper and toLower (Straße → STRASSE, final sigma), and fold.
  • normalize(text, "NFC" | "NFD" | "NFKC" | "NFKD").
  • graphemes(text): Vec<string>: extended grapheme clusters (one emoji family, one flag).
  • slug(text): string: a fragment id or address. It lowercases the NFC form and keeps letters, marks and digits (General_Category L, M, N). Each run of other scalars becomes one -. There is no - at either end. A text without letters or digits gives "". Example: slug("한글 API_2!") is "한글-api-2".
  • isEmoji(grapheme: string): bool: true for exactly one RGI emoji. It accepts basic emoji and the listed keycap, flag, tag, modifier and ZWJ sequences.

The case methods of gizlang:text stay ASCII, for protocol text.

gizlang:regex

JavaScript regular expressions, in time linear in the text, as in RE2. There are no backreferences and no lookaround.

  • regex(pattern, flags) gives a Result<Regex, RegexError>.
  • Then use test, exec(text, from), matchAll, count, replaceAll ($1, $&, $$) and split. Offsets are byte offsets. m.group(i) gives a group.
  • Flags: i (ASCII), s. g and u have no effect.
  • Literal prefixes and required substrings skip starts that cannot match. Small literal alternatives use selective byte scans. Lazy loops with literal delimiters record captures directly. Other patterns use a bounded lazy DFA and capture replay, without an allocation for each character.

gizlang:crypto

SHA-256 uses the CPU's SHA instructions when it has them, chosen at run time.

FunctionMeaning
sha256, sha512SHA-256 and SHA-512.
newSha256, newSha512Incremental hashes with update and a digest that does not consume the state.
hmacSha256, hmacSha512, hkdfSha256HMAC and HKDF.
equalConstant-time byte comparison.
chacha20Poly1305Seal/Open, aes256GcmSeal/OpenAuthenticated encryption (ciphertext plus tag). Open gives an Option. AES-GCM uses libcrypto, for the AES instructions of the CPU.
x25519, x25519BaseKey agreement. Callers must reject an all-zero shared secret.
ed25519PublicKey, ed25519Sign, ed25519VerifySignatures.
p256Key(sys), p256PublicKey(key32)Generate a P-256 private scalar from OS random bytes. Export its public point as 64-byte x || y.
es256Sign(key32, message), es256Verify(publicKey64, message, signature64)ECDSA P-256 with SHA-256. Signing uses RFC 6979. The signature is 32-byte r || s, as ES256 requires.
p256Jwk(publicKey64), p256Thumbprint(publicKey64)Public JWK and RFC 7638 SHA-256 thumbprint.
der(tag, contents), es256Der(signature64)Definite-length DER and a DER ECDSA signature.
p256Csr(key32, domains), p256PrivatePem(key32)PKCS#10 DER request with a CN and DNS SANs; PKCS#8 PEM private key. Invalid keys or DNS names give none.
pemEncode(label, data), certificateValidity(chain)PEM encoding and the first X.509 certificate's validity interval in Unix milliseconds. Validity extraction does not verify trust.
  • Keys and seeds are 32 bytes. ChaCha nonces are 12 bytes and must be unique for each key. Signatures are 64 bytes.
  • Get secure key bytes from System.randomBytes(n).

Named algorithms go through libcrypto (the Node crypto binding is a thin adapter of these):

  • newDigest(name) (update, copy, digest(length) for XOFs), digest(name, data), newHmac(name, key), hmac, pbkdf2, scrypt, hkdf, randomBytes(n), randomPrime(bits, safe), isPrime.
  • newCipher(name, encrypt, key, iv, authTagLength) gives a Cipher with update, final, setPadding, setAad, tag, setTag; cipherInfo(name), cipherAccepts(name, keyLength, ivLength).
  • Key: parsePrivateKey/parsePublicKey(data, format, keyType, passphrase), write, raw, rawKey, generateKey(kind, bits, curve, exponent, qBits, generator), sign, verify, rsaCipher, derive, textParam/intParam/numberParam/octetParam. keyFromParams(kind, selection, params) builds a key from textParam/numberParam/octetParam values (SELECT_PARAMETERS|PUBLIC|KEYPAIR); ecKeyFromPrivate(curve, priv) derives the public point; generateFromParams(key) generates from a parameters-only key (DH).
  • digests(), ciphers(), curves() list what this libcrypto has. lastError() is its last error text.

gizlang:acme

ACME v2 uses HTTPS, ES256 JWS, and HTTP-01 (RFC 8555). It uses the existing HTTP client, TLS transport, and JSON codec. OpenSSL commands are test oracles only.

FunctionMeaning
certificate(sys, dir, directoryUrl, domains, contact)Returns Result<Certificate, AcmeError>. Uses the stored certificate while more than one third of its lifetime remains. Otherwise, orders and stores a new certificate.
challenge(dir, request)HTTP-01 handler. Mount it at /.well-known/acme-challenge/ on public HTTP port 80 before requesting a certificate. Unknown tokens give 404.
renew(sys, dir, directoryUrl, domains, contact, credentials)Starts a task. It renews at two thirds of the certificate lifetime and replaces the shared TLS credentials. Join or monitor the task to receive errors.
jws(key, kid, nonce, url, payload)Flattened ES256 JWS. An empty kid uses jwk. An empty payload is POST-as-GET.
retryAfter(headers, now)Reads delta seconds or an HTTP date. Refuses a wait longer than 30 seconds.
  • Give one private Dir for each directory URL and DNS name set. Names are sorted and made lowercase. Use 1 to 100 distinct DNS names. Wildcards are not HTTP-01 names. The first sorted name is the CN and has a maximum of 64 bytes.
  • The module stores account.key, domain.scalar, domain.key (PKCS#8 PEM), chain.pem, and identity.json. Keys have mode 0600. A file lock serializes requests.
  • Certificate has chain, key, from, and until. The times are Unix milliseconds. AcmeError has stage, HTTP status, ACME problem, and detail.
  • Use an empty contact or a mailto: URL. Calling certificate agrees to the CA's terms.
  • Each request has a 10-second limit. An operation has a 120-second limit. A response has a 65,536-byte limit. Polling stops after 30 responses. A bad nonce has at most three attempts.

gizlang:math and gizlang:random

  • gizlang:math has the libm functions on f64 (floor, pow, exp, log, sin, atan2, hypot, …), with PI, E, round, min, max and sigmoid (no overflow for a large |z|).
  • scalbn(x, exponent: i32) scales by a power of two through libm. It includes binary64 subnormal rounding and signed zeros. It does not make an intermediate pow(2, exponent).
  • The binary64 layout constants (F64_EPSILON, F64_FRACTION_BITS, ...) and addCompensated, a compensated sum that refuses a term that is not finite.
  • gizlang:stats has the sample statistics and the streaming ones that the SQL aggregates merge: Welford and Chan variance, moment matrices, nearest-rank histograms and least squares by Givens QR (stepOlsFactor, fitOls, olsInference).
  • gizlang:random is a seeded generator: newRandom(seed), then r.next(), r.below(n) and r.float().

gizlang:time

  • utc, utcMs, iso and weekday change milliseconds since 1970 into UTC dates and back, as the JavaScript Date does.
  • zone(dir, "Asia/Seoul") reads a TZif v1 to v3 file relative to a gizlang:system Dir at the root of the zoneinfo directory. It gives Result<Zone, TimeError>. An I/O error keeps its code.
  • zone.offset(ms) is the offset in seconds east of UTC. zone.local(ms) is a DateTime.
  • zone.instant(dateTime) gives a Result with the "compatible" resolution of Temporal: the earlier instant in an overlap, and forward by the gap for a skipped time.
  • POSIX footer rules continue after the transition table. An empty footer keeps the last offset.
  • Leap-second files are refused, because instants are POSIX milliseconds.
  • addDays, addMonths and addYears operate on a DateTime. They keep its clock and clamp the ends of months. daysBetween(from, to) is the signed difference of civil dates. It ignores the clock. Apply zone.instant after calendar arithmetic to resolve DST.
  • zone.iso(ms) includes an offset, with seconds when necessary.
  • parseIso(text) gives Result<i64, TimeError> for extended ISO calendar date-times with Z or ±HH:MM[:SS], signed six-digit years, and a maximum of three fraction digits.

gizlang:system summary

A System value gives time() (the wall clock) and entropy() (an unpredictable seed). No other code reads either of them. It also gives:

  • stdin(), stdout() and stderr(): the standard streams.
  • spawn(program, args, options): starts a child process.
  • run(program, args): captures the status and output of a child process.
  • runWith(program, args, options, input): run with a working directory, environment additions and standard input.
  • temporaryDir(): a private directory. It is removed on close, or at task or World teardown. Its temporaryPath() lets child programs use it.
  • exit(code): ends the process without a print of the result of main.
  • A POSIX layer with Result<_, IoError> (its code is the negated errno): descriptors (openFd, readFd, writeFd with a position or -1 for the cursor, seekFd, syncFd, truncateFd, closeFd), metadata (stat(path, follow), statFd, statFs, access, changeMode, changeOwner, setTimes), directories (makeDir, makeTempDir, readDir, removeDir), links (linkPath, symlinkPath, readSymlink, resolvePath), renamePath, unlink, copyFd; terminals (isTerminal, terminalSize, terminalState, terminalRaw, terminalRestore); OS facts (hostname, osInfo, uptime, loadAverage, memoryInfo, cpuInfo, availableParallelism, interfaces, userInfo, homeDirectory, priority, credentials). The Node fs, os and tty bindings are thin adapters of it.

A Dir adds rename, atomic replaceFile (staging in the same directory, fsync, rename), info (FileInfo.size, modified, kind, mode), setMode, sub, and a deterministic walk that does not follow links.

A void main prints nothing of its own (a script). The compiler ignores a first line that starts with #!. A script imports its own files by relative path ("./util.giz", "../lib/x.giz"). gizlang FILE builds them as one package. For more, refer to SYSTEM.

gizlang:json

  • Json holds null, boolean, number, string, array and object (with ordered keys).
  • parse(text) gives a Result with a JsonError offset. stringify(value) gives text. They agree with JavaScript JSON.parse and JSON.stringify, but a lone surrogate escape becomes U+FFFD.
  • The parser copies each unescaped string one time and decodes escaped runs in slices. It indexes wide objects with cached key hashes. Numbers stay f64, also integers above 2^53.

reader(text) keeps the input without a copy and gives a JsonReader:

  • Enter a container with object() or array(). Loop with key() or item(). They return false at the matching end.
  • Compare keys with keyIs(text) without an allocation. This also works for escaped keys.
  • Read a value with number(): f64, integer(): Option<i64>, string(): string, boolean(): bool or isNull(): bool. isNull consumes a null. Otherwise it leaves the value unread.
  • integer accepts only integer tokens in the range of i64, exactly, also above 2^53. It consumes a valid fraction, an exponent or an integer out of range, and gives none.
  • skip() validates a whole value without a build of it. key and item also skip any unread previous value.
  • Finish each nested loop before you advance its parent.
  • The reader decodes only the strings that you request. The reusable stack costs O(depth). The container limit is 512, as for parse.
  • error(): Option<JsonError> keeps the first error and its byte offset. Later reads stop. fail(message) stops them with an error of a decoder.
  • When you close the root or consume a root scalar, the reader checks the trailing text. If you stop early, the remaining text is not checked.
  • giz codec writes the encoder and decoder of a record type on this reader (TOOLS).

writeString(out, text), writeNumber(out, n) and writeInteger(out, n) append one value to bytes. Thus you can serialize a typed record field by field. writeExactInteger(out, n) writes n as a number when |n| <= MAX_SAFE_INTEGER (2^53 - 1), else as a string of its digits. Use it where a browser reads the JSON: a JavaScript number is a double.

gizlang:metrics

Counters, gauges and histograms that every task of a process changes without a lock, and their text in the Prometheus exposition format 0.0.4.

  • Declare a metric with its label names and every series (its label values): counter(name, help, labels, values), gauge(name, help, labels, values, scale) and histogram(name, help, labels, values, bounds, scale). The result is Result<Metric, string>.
  • metric.at(values) is the series (an Option). Keep it: the lookup is linear.
  • series.add(n), series.set(v), series.observe(v) and series.value() use i64 values in the base unit. scale (a power of ten) is the number of base units in one exposed unit.
  • text(metrics) is the exposition text. handler(metrics) is the GET /metrics response.
  • Two declarations of the same series share one value (Task.atomicGlobal).

gizlang:csv

An RFC 4180 reader(data: bytes):

  • Loop with next(). Then count() and field(i): Option<Field> give absolute start and end byte ranges without copies. The ranges exclude the outer quotes. escaped marks doubled quotes.
  • text(i): Result<string, CsvError> copies and unescapes when you ask. It rejects invalid UTF-8.
  • number(i): Option<f64> and integer(i): Option<i64> parse whole fields in place (Text.decimalPrefix). Empty, escaped, missing or invalid numbers give none.
  • LF and CRLF end records. Quoted newlines stay.
  • Empty input has no records. An empty line has one empty field. A final delimiter adds an empty field.
  • The reader uses the range table again for each record.
  • error(): Option<CsvError> keeps the first syntax error and its byte offset. Later reads stop.

writer() appends bytes. field(data) adds a field and quotes it only when necessary. end() writes CRLF. data() returns the bytes.

Before you read or write, format(delimiter: u64, quote: u64): bool selects different bytes that are not CR or LF. The defaults are comma and double quote. An invalid or late configuration stops the reader or writer. error() reports it.

gizlang:erasure

Erasure codes over GF(2^8) (polynomial 0x11D). A Code has data data pieces and a parity piece for each coefficient row of parity.

  • reedSolomon(data, parities): any data of the pieces give back the others.
  • lrc(data, groups, globals): a locally repairable code. Each group of data pieces has a local parity (their XOR), then globals global parities follow (Cauchy rows). lrc(12, 2, 2) stores 16 pieces for 12 (1.33x), rebuilds one lost piece from 6 others and survives any 3 losses.
  • encode(data) gives the parity pieces. All pieces have one length.
  • sources(want, available) names the pieces to read to rebuild piece want: the rest of its group when they are available, else data independent pieces. None when too many are lost.
  • rebuild(want, pieces) rebuilds piece want from the pieces given (none where absent).
  • The products run 32 bytes a step with nibble table lookups: about 5.5 GiB/s of data for lrc(12, 2, 2) on one AVX2 or AVX-512 core (bench/fs/erasure.giz). Baseline x86-64 has no byte shuffle and is about 20 times slower.

gizml:tensor

A Tensor holds an n-dimensional array. The data is row-major f32 data on the host, or a GPU buffer with empty data.

  • Make a tensor with tensor(shape, values), fromData(shape, f32s), full, zeros, ones, scalar or arange.
  • Examine it with reshape (one -1 is inferred), transpose (2-d), get, size and rank.
  • Element-wise add, sub, mul, div, maximum, minimum and greater broadcast as NumPy does.
  • neg, abs, sqrt, square, exp, log, tanh, sigmoid, relu, gelu and geluErf take one tensor. exp, log and tanh use Cephes polynomials. gelu uses the tanh approximation, and geluErf the erf form (torch's default) with Eigen's f32 erf.
  • lazy(t) records such a chain. eval() runs it fused: one pass over the output in blocks, on every core when it is large, with no intermediate tensors.
  • sum(policy), mean and max reduce every element to an f64. Reduction.strict() adds in f64 lanes (exact for integer data below 2^53). deterministic() and fast() add in f32 lanes. Each policy splits at fixed offsets, so a sum never depends on the core count.
  • sumTo(shape) and maxTo(shape) reduce the axes that shape keeps as 1 (or leaves out in front, as broadcasting does): x.sumTo([b, 1]) sums each row of a (b, n) tensor, x.maxTo([1, n]) takes each column's largest element. A row sum adds as Reduction.strict(); other sums add in index order in f32. maxTo ignores NaN: it gives -Infinity where every element is NaN or the axis is empty.
  • softmax is over the last axis.
  • attention(q, k, v, scale, causal, rate, key) is scaled dot-product attention over (b, t, d) tensors. It keeps no (tq, tk) score matrix: it takes the keys in blocks of 16 with an online softmax, in f32x16 lanes on every core or in one GPU kernel, with the same bits in both places. causal hides key j from query i when j > i + tk - tq (a KV cache has tq < tk). A rate above 0 drops weights with the mask that dropout draws for them. causalMask(tq, tk) is that mask as an additive tensor.
  • matmul multiplies (..., m, k) by (k, n) with packed 6x32 FMA tiles. From 2^21 multiply-adds, it runs in row blocks on every core.
  • matmulT(a, transposeA, b, transposeB) reads either side transposed in place, with no copy.
  • Rows and layout: take and addRows, slice, concat and permute, batchMatmul.
  • gizml:netops has the network operations on these tensors, in the same placement and order rules: attention and causalMask, layerNorm and its gradient, label crossEntropyLoss and crossEntropyGrad, keyed dropout, im2col and col2im, convolution and pooling, groupNorm, argmax, and the packed, 8-bit and INT8-row forms that gizml:diffusion uses.

GPU placement:

  • The same tensor runs on a GPU (gizhw:gpu). t.on(gpu) moves it. host() brings the f32 values back. gpu() gives its placement and dtype() its storage type.
  • cast(F16) and cast(BF16) round GPU storage to nearest even. Host storage stays F32.
  • lazy(t).eval() dispatches to the GPU of the tensor and emits one kernel.
  • matmul, sum(policy), sumTo, maxTo, softmax() and attention use the same API in either place. They do the CPU operations in the CPU order, so the results are the same bits as on the CPU. Zeros from max and min can have a different sign, as on the CPU between -O0 and -O2. F16 and BF16 matmul of one type runs on the matrix cores (AMD RDNA 3 and 4 WMMA, NVIDIA sm_80+ mma.sync): the products are exact in f32, but the sums are the hardware's, so each element is within the f32 error bound of its k-term sum and not the CPU's bits.
  • Placement is explicit, as in PyTorch: t.on(gpu) moves a tensor to the GPU of gizhw:gpu's open, t.host() moves it back, and each operation runs where its operands are.
  • Recovery tests pass on the CPU and on the local AMD gfx1151 GPU. A CPU fallback in the GPU test does not prove execution on the hardware.

Random tensors come from keys, not from a hidden state:

  • key(seed) makes a Key. k.split(n) makes independent keys.
  • uniform(k, shape, low, high) and normal(k, shape) draw with Philox4x32-10. Word i comes from the key and i only, so a draw is the same on any core count.

gizhw:gpu

gizhw:gpu opens the GPU through its driver library at run time. Nothing is linked at build time.

  • open(sys): Option<Gpu> tries AMD first, then NVIDIA.
    • AMD: ROCm libamdhip64. Kernels are LLVM IR that libamd_comgr compiles in the process.
    • NVIDIA: libcuda. Kernels are PTX that the driver compiles.
  • GIZLANG_DEVICE=hip, cuda or cpu selects the device. With gpu, hip or cuda, open faults when no GPU opens.
  • GIZLANG_GPU_SOURCES=FOLDER writes each kernel's source as it is compiled to FOLDER/NAME-HASH.ptx (.ll on AMD, .metal on Apple), its key on the first line, to read with ptxas -v or nvdisasm.
  • A Gpu has kind, arch (gfx1151, sm_120) and name.
  • buffer(count), upload(data) and download(buffer) move f32 values. When the last Buffer is dropped, its device memory goes into a pool, by size.
  • bytes(size), uploadBytes and downloadBytes transfer typed storage.
  • kernel(key, name, source) compiles source() the first time that it sees key.
  • launch(kernel, blocks, threads, arguments) queues a launch. sync() waits. A driver failure is fault R004.
  • compile(key, make) compiles a gizhw:kernel program. run passes buffers, then scalars.
  • gizhw:kernel has the matrix multiply-accumulate of a wave of 32 threads: mmaLoadA reads a 16x16 F16 or BF16 tile of A by rows and mmaLoadB one of B by columns from workgroup memory (offsets and strides multiples of 8), mma(a, b, c) adds a b to an F32 accumulator (mmaZero), and mmaGet(c, r) is the lane's element r (0 to 7) at row mmaRow(r) and column mmaCol(r). llvm(arch) emits AMD WMMA in the layout of the generation (gfx11 or gfx12) and ptx(arch) emits mma.sync.m16n8k16; hasMma(kind, arch) tells where.
  • load4(buffer, index, ok) loads 16 aligned bytes from global memory (zeros where ok is false), sload4 the same from workgroup memory, sstore4 stores them unchanged and lane(v, j) is word j. The words are F32 for F32 memory, else raw I32. An I32 stored into F16 or BF16 elements keeps its low 16 bits unchanged, so a copy from a U16 load converts nothing.
  • tune(key, choices, work) measures equivalent variants after warm-up. It caches the fastest one on the opened GPU. Tensor matmul uses it for K tile depths 16, 32 and 64 above 2^20 multiply-adds, keyed by shape, layout and dtype. All variants keep the same FMA order.
  • Cache reuse and numerical equality between CPU and GPU pass on AMD gfx1151. A performance improvement has not been measured.

gizml:tile

GPU kernels as ordinary Gizlang over tiles (blocks of elements). The program says what one block of the output is; the module chooses threads, layouts, workgroup memory, barriers, bounds masks and matrix cores. Host tensors run on the CPU's cores, a GPU's tensors on that GPU. COMPUTE has the design.

gizlang
import * as T from "gizml:tensor";
import * as Tile from "gizml:tile";

/** relu(a b) into c: a is [m, k], b is [k, n] and c is [m, n]. */
export function productRelu(a: T.Tensor, b: T.Tensor, c: T.Tensor, m: i64, n: i64): T.Tensor {
    return Tile.run([Tile.cdiv(m, 128), Tile.cdiv(n, 128)], (p: Tile.Program): void => {
        const i = p.id(0); const j = p.id(1);
        const A = p.read(a, [128, 32]); const B = p.read(b, [32, 128]);
        const acc = p.fold(p.int(0), A.blocks(1), p.zeros([128, 128]),
            (k: Tile.Index, acc: Tile.Tile): Tile.Tile => acc.add(A.at([i, k]).dot(B.at([k, j]))));
        p.write(c, [128, 128]).put(acc.relu(), [i, j]);
    })[0];
}
  • run(grid, body): body once per grid point (1 to 3 sizes, axis 0 fastest). It returns the tensors written, in the order of write. All tensors of a launch are on the host or on one GPU.
  • Program: id(axis), count(axis), int(n), k(x), zeros(shape), full(shape, x), iota(shape, axis), read(t, block), write(t, block), loop(from, to, inits, body), fold(from, to, init, body), warps(n).
  • Tile: a 1-d or 2-d block whose sizes are powers of two (at most 65536 elements); a 1-d tile is the row [1, C]. It computes in F32 and keeps the storage type of its values (dtype). add sub mul div maximum minimum broadcast as numpy does; lt le gt ge eq ne give masks for select, and, or, not; neg abs exp log sqrt rsqrt sigmoid tanh gelu geluErf relu floor; sum(axis), max(axis), min(axis) keep the axis as size 1; dot; transpose; to(dtype) rounds to F32, F16, BF16 or FP8. k(x) is a constant in the tile's program.
  • In.at(coords) (from read) loads the block at block coordinates, window(offsets) at element offsets; Out.put and putWindow store. Out-of-range elements read 0 and are not written. blocks(axis) and dim(axis) are Indexes.
  • Index: add sub mul div rem minimum maximum and k(n); value() is the index as a [1, 1] tile.
  • dot of F16 or BF16 operands of one type runs on matrix cores where hasMma; acc.add(a.dot(b)) accumulates there. Other operands use an exact F32 FMA product.
  • tune(configs, grid, body) keeps the fastest configuration per GPU and tensor shapes. source(grid, storage, kind, arch, warps, body) returns the PTX or AMDGPU LLVM a launch would run, with no GPU.
  • Elementwise results equal the CPU's bit for bit. Reductions and dot add in an order that depends on the device.

gizml:grad and gizml:nn

gizml:grad calculates derivatives. A Graph records each operation on its Var values as it runs:

  • input, and constant for data that does not need a gradient.
  • The element-wise operations, matmul, transpose, reshape, sum, mean, sumTo and maxTo (ties share the gradient evenly, as torch.amax). logSoftmax and softmaxCrossEntropy (soft targets) are made of these, so they run on the GPU and have both modes.
  • Network operations: embedding, layer norm, GELU (both forms), attention, label cross-entropy, layout, batch matmul, keyed dropout, convolution and pools. The network rules use custom and support reverse mode.
  • g.autocast(F16) or g.autocast(BF16) runs the matrix products (matmul, batch matmul, convolution, in both modes) on GPU operands in that type, on the matrix cores, as torch.autocast. Every other operation and every gradient stays F32. On the CPU it changes nothing.
  • g.backward(loss) gives every gradient in one reverse walk (grads.of(v)). Broadcast gradients are added back.
  • g.tangent(inputs, directions, output) gives the derivative along a direction in one forward walk.
  • g.custom(inputs, value, backward) adds an operation with its own gradient.
  • g.checkpoint(inputs, f) keeps only the result of f, and runs f again for its gradient.

gizml:nn trains with gizml:grad:

  • linearInit (the starting values of PyTorch) and linear, mse, oneHot and accuracy. Label cross-entropy is Graph.crossEntropy.
  • inputs and gradients move between parameter lists and a Graph.
  • sgd(rate, momentum), adam(rate) and adamW(rate, decay). Their step(params, grads) updates in fused passes.
  • save(tensors) and load(data) for safetensors files. save writes F32. load reads F32, F16, BF16 and F64, and the integer types (I64, I32, I16, I8, U8, BOOL) as their values.
  • BERT: bertWeights names the tensors of a HuggingFace checkpoint, and bert(config, weights, ids, types) gives the hidden state of each token (post-norm attention, erf GELU).
  • encoderOf(files) makes a sentence-transformers Encoder from the files of its repository (config.json, tokenizer.json, model.safetensors, modules.json, the pooling config). files is a ModelFiles reader: a path in the repository gives its bytes, or none. Encoder.embed(text, normalize) pools the token states as the model is configured.
  • Decoder-only GPT: gptInit, gpt, kvCache, gptStep and greedy generate. The model uses pre-norm causal multi-head attention, learned positions and a 4x GELU MLP. gpt, gptStep and bert use the one attention operation.
  • A random key passed to gpt enables the configured dropout. Inference uses no key. generate returns only the new tokens.
  • CPU tests verify a decrease of finite training loss, a learned counting sequence, and KV-cache logits that agree with the full pass within 1e-4. GPU GPT training is not verified.

gizml:tokenizer

  • parse(json) reads a HuggingFace tokenizer.json with a WordPiece model, the BERT normalizer and pre-tokenizer, a template, BERT or RoBERTa post-processor, added tokens and the truncation length. Another model type is an error.
  • Tokenizer.encode(text, limit) gives the ids and the type ids. A limit of 0 uses the truncation of the file. The special tokens of the template stay when the text is cut.
  • splitter(kind) gives the langchain text splitter character, recursive_character, python, markdown or latex (langchain defaults: size 4000, overlap 200). Splitter.split(text) gives the chunks.

gizlang:hub

  • hub(sys, path, download) is a folder of Hugging Face Hub repositories at REPO/FILE. folder(sys) is the shared folder $GIZLANG_CACHE/huggingface.
  • Hub.file(repo, file) reads a file. When the folder does not have it and download is true, it is downloaded once from huggingface.co (with $HF_TOKEN).

gizml:onnx

  • encode(model, version) is the ONNX file (IR 8, opset 17, ai.onnx.ml 3) of a gizml:ml model: linear models, summed or averaged tree ensembles, PCA and k-means. Other estimators are an error.
  • decode(data) is the model of an ONNX file with one ai.onnx.ml LinearRegressor, LinearClassifier, TreeEnsembleRegressor or TreeEnsembleClassifier. Its tree tests give the branch that ONNX Runtime gives after it rounds the input to f32.
  • portsOf(model) gives the graph's inputs and outputs (Port: name, element type, dimensions; -1 is the row count).

gizlang:compress

  • DEFLATE, gzip(data), IEEE crc32(data) and Castagnoli crc32c(data). The checksums use the CPU's CRC or carry-less multiply instructions when it has them, chosen at run time.
  • newCrc32c() supports update(bytes) and digest(), without assembly of input chunks.
  • CRC-32C and the CRC-32 of gzip are different checksums.
  • newDeflater(framing, level) is the streaming encoder: write(input), flush(kind) with NO_FLUSH, SYNC_FLUSH (ends on the empty stored block 00 00 ff ff), FULL_FLUSH or FINISH, then read() takes the output made so far. setLevel applies from the next block. It pairs with inflater(framing).
  • newBrotliEncoder() and newBrotliDecoder() stream Brotli (RFC 7932) through libbrotli: the encoder's write(input, op) takes BROTLI_PROCESS, BROTLI_FLUSH or BROTLI_FINISH and read() gives the output; the decoder's write(input) is an error with libbrotli's name (errorCode() its code) and done() says the stream ended. setParameter(param, value) takes the BROTLI_PARAM_* / BROTLI_DECODER_PARAM_* numbers.
  • DEFLATE uses a 32 KB hash-chain window. Inputs below 512 bytes use greedy matching. Larger inputs use lazy matching. The chain limit is 4 below 512 bytes, 16 below 16384 bytes, and 32 otherwise. It writes one block in dynamic or fixed Huffman codes, the shorter of the two.
  • adler32(data) and xxh64(data, seed) give the checksums of zlib and zstd. newXxh64(seed) supports update(data, from, to) and digest().

Decoders. Two streaming decoders read DEFLATE (RFC 1951, 1950 and 1952) and zstd (RFC 8878).

  • newInflater(framing, limit) reads RAW, ZLIB or GZIP input. newZstdDecoder(limit, windowLimit) reads zstd frames. Give ZSTD_WINDOW_LIMIT (128 MiB) as the window limit to read all frames that the zstd tool writes, --long=27 included.
  • Push API: zstdDecoder() and inflater(framing) make decoders with no output limit. Give each piece of input to feed(bytes), and call finish() after the last piece. Each call returns Result<bytes, string>: the output that is ready, or the error message with its input offset. finish() returns an error if the stream or frame did not end.
  • Pull API: put input in with write(bytes). Call close() after the last input. read() gives the next output, at most 256 KiB (inflate) or 1 MiB (zstd). It gives 0 bytes when the decoder needs more input, or a DecodeError. Stop when done() is true. Use this API to keep the output in memory small: feed returns all the output of its input at once.
  • inflate(data, framing, limit), gunzip(data, limit), unzstd(data, limit) and unzstdWith(data, dictionary, limit) decode all of the data in one call.
  • lz4(data) gives one LZ4 block (the block format, no frame) and unlz4(block, limit) decodes one. The encoder is fast, not small (about 300 MB/s on text, more on data that does not compress); decode is about 700 MB/s.
  • limit is the maximum number of output bytes. More output is the error TOO_LARGE. A zstd frame with a window larger than windowLimit is also TOO_LARGE.
  • Errors have a code, the input offset and a message. The codes are CORRUPT, TRUNCATED, TOO_LARGE, CHECKSUM, UNSUPPORTED and NEEDS_DICTIONARY. Bad input gives an error, not a fault.
  • gzip reads all members of a file. Bytes after the last member are CORRUPT. Raw and zlib input stops at the end of the stream and does not read more bytes.
  • zstd reads raw, RLE and compressed blocks, Huffman and FSE tables, repeat offsets, content checksums, skippable frames and the frames that follow. useDictionary(dictionary) sets a raw-content or formatted dictionary (zstd --train) before the first write. The decoders refuse the inputs that zstd 1.5.7 refuses in tests/golden-decompression-errors.
  • Memory: inflate keeps a buffer of 288 KiB. zstd keeps the frame window plus as much again (at least 1 MiB, and not more than the content size of the frame). Streamed from a file, the 212 MB Silesia corpus decodes in 5 MiB (gzip -6), 22 MiB (zstd -19) and 208 MiB (zstd --ultra -22 --long=27, a 128 MiB window).
  • tests/sdk/compress/decode/ decodes the zstd golden files and reference streams whole and in pieces of 1, 7 and 4096 bytes, decodes 400 MiB in flat memory, and decodes mutated streams (a fuzz loop). All 12 Silesia files and the 4 zstd golden-compression files agree byte for byte after gzip -1, -6 and -9, and zstd --fast=5, -1, -3, -9, -19, -3 --no-check and --ultra -22 --long=27 (160 streams).

Decode speed of the 212 MB Silesia corpus (bench/sdk/compress/run.giz 7 FILE..., 2026-10-13, a shared 16-CPU host). The reference is zstd 1.5.7, and zlib 1.3.2 in zstd --format=gzip. Values are MB/s of output, the median of 7 alternating rounds, with the range.

Inputgizlang:compressReferenceRatio
zstd -3585 (570–609)1442 (1367–1536)41%
zstd -19496 (421–533)1121 (726–1254)44%
zstd --ultra -22 --long=27353 (274–399)851 (578–946)41%
gzip -6327 (296–344)410 (322–433)80%

Zip archives:

  • newZipWriter() writes an archive in order: file(name, modified, data, deflated) for a whole entry, or entry and data for one that streams (stored); finish() writes the central directory. ZIP64 comes in where it is needed.
  • zipEntries(size, read, most) lists the entries of an archive of size bytes, ZIP64 included. read(at, count) gives bytes of the archive, so only the end records and the central directory are read. unzipEntry(entry, read, limit) gives one entry's bytes and checks its CRC. Encrypted entries and methods other than stored and deflated are UNSUPPORTED.

gizlang:image

  • Image is width, height, channels (3 is RGB, 4 is RGBA) and pixels, rows top to bottom.
  • format(data) is "jpeg", "png" or "".
  • decode(data, least) reads a JPEG (baseline or progressive, gray or YCbCr, 8 bits) or a PNG (every color type, depth and Adam7). It decodes at the smallest scale (1/2^k) whose longer side is at least least, and turns a JPEG upright by its EXIF orientation. A JPEG runs the inverse DCT on the low n × n coefficients of each block only. At 1/8, a progressive JPEG reads only its DC scans. A PNG streams through inflate into power-of-two boxes, so a full-size picture is never in memory. A file that is cut short decodes as far as it goes.
  • fit(image, side) scales down by area averaging, with the colors weighted by alpha.
  • jpeg(image, quality) writes a baseline JPEG (Annex K tables, color at full resolution). png(image) writes an 8-bit PNG with Paeth-filtered rows.
  • A 12-megapixel JPEG becomes a 480-pixel thumbnail in about 130 ms (-O0). A progressive one takes about 33 ms.

gizlang:pdf

  • text(data, most) is the text of a PDF file, at most most bytes: its pages in order, an empty line between two pages. A line ends where the text moves to another line; a space comes where the text jumps right (by the font's glyph widths).
  • It finds objects by scanning the file, so a broken cross-reference table does not matter. It reads object streams, FlateDecode, ASCIIHexDecode and ASCII85Decode streams, form XObjects, ToUnicode CMaps (one- and two-byte codes) and WinAnsi with Differences.
  • An encrypted file is an error. A scanned page (a picture of text) has no text.
  • A 210 KB design paper reads in about 20 ms (-O0). Its words are 99% of those of pdftotext (poppler).

gizlang:http

HTTP/1.1 and HTTP/2 over any Stream. HTTPS is the same code over a stream that gizlang:tls secures.

  • serve(listener, handler) gives the handler a complete Request. The handler returns a Response or a value with plan(): ReplyPlan. A plan selects a buffered response or a progressive response through a borrowed Exchange. Connections wait in a Ready set. Handlers run in worker tasks: WORKERS at the start, plus one more when none is idle, to a maximum of MOST_WORKERS.
  • It supports keep-alive, pipelining (answered in one write) and chunked bodies. It limits heads to MOST_HEAD and buffered bodies to MOST_BODY. It refuses both framings at the same time, and heads that are not UTF-8.
  • A handler fault ends only its connection. A connection that is idle for IDLE_MS is closed.
  • fetch(system, host, port, request) and exchange(stream, host, request) send one plain-HTTP request on a new connection, with connection: close. Tls.fetch(system, origin, request) sends it over TLS when the origin is https, else through this fetch.
  • url(text) reads an absolute http or https URL into a Url (secure, host, port, target). Url.resolve(location) reads a redirect against it; Url.text() writes it out.
  • A Pool keeps HTTP/1.1 client connections alive and reuses them for each origin (scheme, host and port). pool(system, options) dials plain HTTP. Tls.pool(system, options) and Tls.poolTrusting(system, options, authority) also dial HTTPS. A reused HTTPS connection keeps its TLS session, thus it does no new handshake.
  • pool.open(origin, request) and pool.fetch(origin, request) send a request on an idle connection of the origin, or on a new one. When the body of the response is read to its end, the connection goes back to the pool. pool.close() closes the idle connections and refuses new requests. Url.origin() gives the origin of a URL. origin.authority() is its Host field: the host (an IPv6 literal in brackets) and :port unless the port is the default of the scheme. Every client request sends it.
  • PoolOptions(perOrigin, total, idleMs) sets the maximum number of idle connections for one origin and for all origins, and the time that a connection is kept idle. poolDefaults() is 64, 64 and 30 s. The pool closes an expired connection when it is next used.
  • The pool closes a connection, and does not keep it, after an error, after close of a body that is not fully read, when the request or the response has connection: close, after an HTTP/1.0 response without keep-alive, and after a response body without a framing. A connection with more bytes after the body is also closed.
  • A server can close an idle connection (a stale connection). If a reused connection ends before any byte of the response, the pool sends an idempotent request (GET, HEAD, OPTIONS, TRACE, PUT, DELETE) one more time on a new connection. It does not send other requests again.
  • A pool belongs to the thread that made it. The tasks of that thread can share it.
  • serveReady(set, handler) serves the connections that another task puts in a set. Tls.serve(listener, credentials, handler) serves HTTPS in this way. The handshake of each connection runs in its first read.
  • To select streamed request bodies, use serveStreamed(listener, handler, onHead) (serveReadyStreamed, Tls.serveStreamed for the other two). The hook receives a Request with an empty body. Return Option.none() to use the complete-request handler. Return Option.some(render) to handle the request through a borrowed Exchange. The loop does not collect its body before it calls render.
  • In that exchange, next() reads one body piece. whole() reads at most MOST_BODY bytes. acceptBody() sends 100 Continue after admission. Use respond, write, and finish, or send(response), to finish the response. The connection loop retains at most MOST_HEAD plus one 16 KiB read piece for a streamed request.
  • After a streamed exchange, the connection stays open only if the handler reads the body to its end and finishes the response. An unread body closes the connection after the response. HTTP/1.0 and Connection: close also close it.
  • A connection that starts with the HTTP/2 preface is served as HTTP/2 (RFC 9113): h2 that TLS chooses by ALPN, or h2c with prior knowledge. The same handler, hook and Exchange serve each stream in a task of its own. Replies keep the peer's flow-control windows and frame size. HPACK reads Huffman strings and the dynamic table (4,096 bytes); replies use no Huffman coding and no dynamic table. A connection has at most H2_STREAMS (100) open streams. There is no server push. tests/sdk/http/h2/run.giz checks the RFC 7541 C.4 requests, a body, flow control and PING.
  • gizlang:http3 serves HTTP/3 (RFC 9114) over QUIC with the same handler and reply plans as HTTP/2; see HTTP/3. It needs OpenSSL 3.5 or later (giz openssl builds one); Http3.supported() says whether it is there.
  • webSocket(request, protocol, session) answers a WebSocket opening (RFC 6455) on HTTP/1.1 with 101 and runs session(socket); any other request gets 426. openWebSocket(sys, stream, host, path, headers) opens one as a client. socket.receive() gives the next Message (text or data, fragments joined, at most MOST_MESSAGE bytes) and answers ping; send, sendBytes, ping and close(code, reason) write. A frame that breaks the protocol closes with 1002, bad UTF-8 text with 1007, a message too large with 1009. GizWeb refuses a cross-site opening.
  • bytes.isUtf8(start, end) tells whether string() of a range will succeed.
  • httpDate writes an HTTP date. parseHttpDate(value, nowMs) reads it, including obsolete forms.

HTTP protocol policy

gizlang:http does what a web server does, as functions that a handler calls. Thus a site is one typed handler, not a configuration file.

FunctionMeaning
file(dir, request, path)Serves static files, as the nginx root does: index, directory redirect, media types, ETag and Last-Modified revalidation, one byte range.
proxy(pool, upstream, request)Forwards, as proxy_pass with upstream keepalive does: on a kept connection of the Pool to the Origin. It drops hop-by-hop headers and sends the authority of the upstream as Host. A failure is 502.
under(path, prefix)Routes by whole path segments.
compressed(request, response)Compresses a response with gzip, as the nginx gzip on does.
setCookie(name, value, maxAge, secure): Result<string, string>Makes a cookie with Path=/, HttpOnly and SameSite=Lax. maxAge is Option<i64> in seconds. none makes a session cookie. Zero clears the value and adds Max-Age=0. secure adds Secure. Rejects negative ages, invalid RFC 6265 token names and values outside unquoted cookie-octets.
decodeForm, queryDecode forms and query parameters. See Form decoding.
redirect, withHeader, pathOf, encode, decode, httpDate, mimeType, logLineOther server tasks.
Form decoding

decodeForm(encoded): Result<Params, HttpError> reads application/x-www-form-urlencoded text. It splits at & and at the first = in each field. A field without = has an empty value. It skips empty parts. It changes + to a space and decodes percent escapes one time. It keeps all names and values in input order, including duplicates and empty names. Params.get(name) returns the first value.

LimitValueCounts
MAX_FORM_FIELDS1,024Nonempty parts, including duplicate names.
MAX_FORM_BYTES1,048,576Input bytes before decoding, including separators and escapes.

The decoder returns HttpError with status 400 and code 0 for a malformed percent escape, invalid decoded UTF-8 in a name or value, or input above a limit. It returns no partial result.

query(request): Result<Params, HttpError> uses decodeForm on the text after ?. A request without ? gives empty Params. Decode the query one time for each request.

Routing:

  • Declare Route<T>(method, pattern, target) values. Compile them one time with router(declarations): Result<Router<T>, string>.
  • routes.resolve(request) returns a typed target and Params, or an HTTP error response. Params holds {name} segments and a final *rest, decoded one time (get, integer).
  • The specificity of literal, capture and rest segments does not depend on the order of declaration. Startup rejects ambiguous bindings and invalid patterns.
  • An explicit HEAD route comes before GET. Known paths return 405 with a complete Allow. Unknown paths return 404.
  • routes.href(declaration, params) makes escaped URLs from the same registered declaration. It rejects missing or extra captures.
  • A pattern literal does not give Params compile-time types. A typed route does: Web.path<P>(method, pattern) takes the record P of its parameters (the pattern names must be its fields), Web.typed(path, handler) gives the handler a P, and Web.link(path, p) makes the URL from a P, so a misspelled parameter is a compile error.

Middleware is a generic function of the request and the next handler, nested from the outermost. gzip(request, next) is one example. Thus routes and layers stay ordinary typed calls.

The design of the web stack is in the web internals.

gizlang:postgres

PostgreSQL protocol v3 over a System Stream.

  • connect(system, options) (host, port, user, password, database) authenticates with trust, cleartext, MD5 or SCRAM-SHA-256 (SASLprep, with verification of the server signature).
  • query(sql, params) uses the unnamed extended protocol with text Option<string> parameters (none is NULL). It returns Rows with columns, rows of text options, command and affected.
  • exec(sql) runs a script of many statements and returns each result.
  • Both return Result<..., PgError>. PgError has sqlstate, message, detail, an optional position and the transport code.
  • Notices are ignored. Server errors are drained through ReadyForQuery, so the connection can be used again.
  • connection.transaction(work, failure) commits a successful Result or rolls back an error. Use that connection only for that work. The work returns Result<T, E>. Its error reaches the caller unchanged. failure converts the PgError of BEGIN or COMMIT to E.
  • close() ends the connection.
  • pool(sys, options, size) opens size connections that tasks share. pool.with(work, failure) lends one connection to work that returns Result<T, E>. failure converts the PgError of a failed loan to E. It waits while all connections are lent. When the work leaves a connection closed, busy or in a transaction, the next loan opens it again. pool.close() waits for the connections and closes them.
  • Decode numbers with parseI64 and parseF64 of gizlang:text, and PostgreSQL t/f with boolean(value).
  • There is no ORM, no COPY and no TLS.

gizlang:sqlite

The same shape as gizlang:postgres, over the system SQLite library (-lsqlite3).

  • open(path) creates the database when it is missing. Use ":memory:" for a database in memory. It returns a Database.
  • query(sql, params) runs one statement with ?1-style text parameters. It returns Rows of text options with columns and affected.
  • exec(script), transaction(work, failure) and close(). failure converts the SqliteError of BEGIN or COMMIT to the error type of the work, as in gizlang:postgres.
  • A database keeps its first 64 statements prepared and uses them again by SQL text.
  • Errors are SqliteError, with the extended code and the message of SQLite.
  • Blob columns are refused. Calls block the thread of the task.

gizlang:collections

Every program has gizlang:core, gizlang:collections and gizlang:text without an import: their free functions are methods of their first argument (v.filter(f)), as TypeScript's built-ins are. A callback parameter is bounded by its function type (F extends (value: T) => bool), so an arrow function needs no annotation and the call is direct.

MethodMeaning
v.filter(f), v.some(f), v.every(f), v.count(f)Filter, tests and count.
v.find(f), v.findIndex(f)The first match as Option<T>, or its index (-1 for none).
v.includes(x), v.indexOf(x), v.at(i)Search by ===, and the item at i (negative from the end) as an Option.
v.reduce(f, initial), v.sumBy(f)A fold, and the checked i64 sum of a projection.
v.concat(w)v then w.
v.toSorted(before)A stable sort. before(a, b) puts a first: items.toSorted((a, b) => a.sku < b.sku).
v.sort(before)Sorts a let Vec in place. It is a natural merge sort: n log n, one buffer, and existing runs stay. Sorted or reversed input takes linear time.

Map<K, V> and Set<K> keep insertion order. Keys are scalars, strings, bytes, or records with hash (interface Key).

  • const m = new Map<string, i64>(); m.set("a", 1); (new X(...) calls newX(...); new ns.X(...) calls ns.newX(...)).
  • m.get(k) (an Option), m.has(k), m.delete(k), m.size, m.keys(), m.values().
  • m.merge(k, v, combine) sets v for a new key, else combine(old, v), in one lookup: counts.merge(word, 1, (a, b) => a + b).
  • const s = new Set<string>(); s.add(k);, with has, delete, size and values.

Page directories

PageDirectory<T> stores indexed immutable versions behind a persistent 32-way radix directory:

gizlang
import {newPageDirectory} from "gizlang:collections";

function main(): void {
    let pages = newPageDirectory<Vec<i64>>();
    pages.append([1])!;
    print(`${pages.count()}`);
}
  • Use count(), capacity(), get(index): Option<T>, set(index, value): bool, append(value): Result<i64, string> and pop(): Option<T>.
  • To append or remove the tail keeps the preceding indexes. Invalid indexes leave the value unchanged.
  • Writes replace bounded vectors of immutable payload carriers along one radix path, with ordinary copy-on-write of values. Unchanged entries share their buffers across captured versions. To read an entry returns its value with ordinary value and Ref semantics.
  • Ordinary payloads do not use the managed Ref heap. Structural reservation includes each payload carrier. Nested payloads and retained historical versions have separate owners.
  • Recursive generic values through Vec buffers are supported. Inline recursive records stay invalid.
  • pathWork() gives the logical directory nodes and prior slots that the last successful change touched. It does not give allocated bytes, RSS or a measured throughput guarantee.
  • Physical consumers own their page size and contents, independently of this shared directory.

newManagedPageDirectory<T>() returns a ManagedPageDirectory<T> with immutable managed node and entry identities, for a directory inside Ref state.

  • To load the outer state copies its root edge. A write publishes only the same bounded radix path and one payload.
  • It uses the same traversal and the same owner of structural credit as the value directory.
  • Its data uses the managed quota. Ordinary bulk data uses newPageDirectory<T>() and the ordinary allocation budget.
  • Both types use the same radix algorithm. Their carriers select effects at specialization. Owned lookups stay pure and can run in compute workers. Managed lookups require the read effect. A dynamic choice of storage must keep that authority explicit.

Tests

A named function testName(): void with no parameters is a test. A test function that returns bool (the earlier form) is E_TYPE, and the message names the new form. A test passes when it returns without a fault and with every buffer freed.

gizlang:test asserts: equal, differ, check, includes, equalLines (the first line that is different) and equalItems (the first index that is different). An assertion that is false faults inside itself. Thus the failure shows the values and the calling line, with a replay test:

text
fail testAdd R004
    gizlang: R004 in equal line 2 (actual = 4, expected = 5) <- testAdd (app.giz:12)
    gizlang: replay: function testReplayEqual(): void { equal(4, 5); }

Property tests

forAll(seed, runs, generate, smaller, holds, failing) tests a property on generated inputs. Each run has its own seed from seed, so a seed repeats its inputs. When holds rejects an input, forAll shrinks the first such input to a minimal counterexample (smallerI64, smallerVec). failing asserts on it.

Test execution

  • Tests run in parallel, one process for each CPU, each in its own World.
  • The prints of a test show only when it fails. The runner names tests that take more than one second.
  • giz test FILE runs the tests of a file. It records each pass under the meaning hash of the test. The meaning hash is the code without names, comments or layout. Each name that the code uses is replaced by the meaning hash of what it names. Thus the next run repeats only the tests that an edit can affect. A rename or a comment reruns no tests.

Live programs

giz live FILE runs main. When a .giz file or package.json in that folder changes, or a module that the program imports from a different location changes, it publishes the new code if the code checks and its tests pass. The tests include the X.test.giz file beside each source.

After a publication, each call that main makes to a named function runs the newest code with the same signature. This includes calls from a closure written in main, for example a task that it spawns. The call runs fully in that generation: its callees are of the same generation, so one call never mixes old and new code. The loop of main keeps the code that it started with.

Effects and contracts

gizlang
function increment(x: i64): i64
requires x < 100
ensures result > x
effects {fail}
{ return x + 1; }

Effects

EffectMeaning
readReads managed state.
writeWrites managed state.
failCan fault.
consolePrints.
ioTasks, time, files and the network.
  • An effect row can also contain the rows of callable parameters: effects {f, fail}.
  • A process main holds the console. The compiler and library calls do not. A print there is fault R006.
  • The compiler infers effects over the call graph.
  • A declared row is an upper limit that callers depend on. The same is true for the row of an interface method (no row allows any effect), for a generic bound and an interface value, and for the row of a function type. An implementation that goes past the row is E_PROTOCOL.
  • A call through an interface value whose method has no row costs the effects of the implementations of that method in the program, not every effect. Thus a Task.thread or parallel body can call such a method when its implementations are pure.
  • A Task.parallel body can only fail. What it calls through values must declare a maximum of {fail}.

Contracts

  • requires and ensures run at run time. They can only fail. They cannot read or write managed state, and they cannot print.
  • result names the return value. It is available only in ensures.

Not in the language

The compiler rejects these features with a compile error (E_PARSE). It never gives them a different meaning silently.

  • Inheritance (extends) and abstract classes (Classes).
  • Runtime decorators. Annotations are only metadata.
  • Type-level computation: conditional, mapped or template types.
  • using.

TYPESCRIPT owns the differences from TypeScript and the TypeScript forms that lower onto this core.

Diagnostics

Compile errors

giz check FILE prints each error as:

text
FILE:LINE:COLUMN: error CODE in DECLARATION: meaning

The output also gives the reason when it is known (expected and actual types, argument counts, a case that a match does not cover), the source line with a caret, and a hint for a probable name or a TypeScript habit.

With --json, the command prints the JSON of the compiler. Each error has a code, the byte span and a stable_id. The stable_id is the graph entity, or in source the name of the enclosing declaration.

  • Types, constants, function bodies and effect rows are checked one declaration at a time. When many fail, errors lists each (a maximum of 20) after the first in error.
  • A function that names a failed declaration is not checked, because its errors follow from it.
  • When the check passes, it runs the tests of the program and of the test files of its sources (X.test.giz beside X.giz), as for a live generation. It runs each test only when its meaning changed after its last pass. Thus an edit that breaks a fact that the tests hold fails the same check (fail NAME and the trace of the failed assertion). An unchanged test costs nothing.
  • --json checks only. It does not run tests.

The codes are the stable error vocabulary of the language. compiler/layout.giz defines them.

CodeMeaning
E_PARSE, E_LEXSyntax, or invalid UTF-8 or token.
E_TYPE, E_CAST, E_FIELD, E_INFERENCE, E_GENERICType mismatch, conversion, field, inference, generic arity or bounds.
E_NAME, E_UNBOUND, E_DUPLICATE, E_SHADOWName resolution.
E_BORROW, E_ESCAPE, E_IMMUTABLEViews, captures, assignment.
E_EFFECT, E_CONTRACT_EFFECTAn effect goes past its declared row, or a contract has effects.
E_RETURN, E_MATCHControl flow, exhaustiveness.
E_PROTOCOL, E_METHODInterface implementations, receiver functions.
E_MODULE, E_PACKAGE, E_PACKAGE_INTEGRITYImports, manifests, pins.
E_LITERAL, E_CONSTANTLiteral ranges, constant initializers.
E_LIMIT, E_ENTRY, E_GRAPH, E_MEMORY_UNSUPPORTEDCompiler limits, entry point, canonical graph input.
E_CONFLICTAn edit request whose base is not the source that it was applied to.
E_DOCA doc comment tag that is unknown, repeated, or names no parameter.
E_INTERNALA compiler fault with no code of its own. Report it with the source that causes it.
W_DEPRECATED, W_UNREACHABLE, W_UNUSED_RESULTWarnings. They never cause a failure. The first is a use of a @deprecated declaration. The second is statements after a return or a failing assert. These statements are skipped (not checked and not run), as TypeScript allows. The third is a statement that drops the updated copy a method makes of a variable or field (the method returns the receiver's type and has no mut parameter and no effect but failing): v.removeAt(i); leaves v as it was.

Run-time faults

A run-time fault prints runtime RNNN and exits with status 1 at once: tasks, threads and devices that are still open do not delay the exit. On stderr it names the call path of the fault by declaration name, innermost first. Each frame has the line in that declaration where it failed (line 1 is the line of its name). The innermost frame also shows a maximum of six argument values:

  • Numbers and bools.
  • The first 32 bytes of a string.
  • A Vec of i64, u64 or f64, by its first four items ([1, 2]).
  • The length of bytes and of other vectors.
  • Records, by their fields.
text
gizlang: R001 in inner line 3 (xs = [1, 2, 3], i = 5) <- outer line 4 <- main line 1

A closure shows as f closure with the lines of f. A receiver function shows as Type.name. The path has a maximum of 16 frames, then a count. In a program of more than one file, each frame also names its file: check line 2 of b.giz, and of gizlang:test for the SDK. giz test shows such a frame at that file's line.

When every argument of that frame was kept exactly, a second line gives the call as a test that repeats it. Exact arguments are numbers, bools, strings of a maximum of 32 printable bytes, Vecs of a maximum of four numbers, and records of those.

text
gizlang: replay: function testReplayInner(): void { inner([1, 2, 3], 5); }
FaultCause
R001Index out of bounds.
R002Division by zero.
R003Integer overflow or conversion.
R004requires or assert.
R005Shift.
R006Console.
R007ensures.
R009Stack exhausted: recursion into the last 256 KiB of the stack of a thread or task.
R010No variant matched: a match reached no arm, or a live call found no code for its declaration (a generation removed it).
R011Memory limit (GIZLANG_MEMORY_LIMIT bytes, default 80% of the machine's memory).
R012Task cancelled.
R013Deadlock: a task waits for something that nothing can cause.
R014A task or handle used after it ended or closed.

Imported find and sumBy on a Vec<T> also permit v.find(x => ...) and v.sumBy(x => ...): the lambda parameter has type T. Receiver spelling is an ordinary SDK call, not another collection representation.

Uniform call syntax: when x.f(args) finds no receiver function or function-valued field, a free function in scope or an ambient builtin named f is called with x as its first argument: values.toSorted(before) is toSorted(values, before), text.slice(1, 3) is slice(text, 1, 3). The function's own signature, effect row and mut rules decide whether the call is valid. Math.f(x) is the builtin f(x) unless a namespace named Math has the member.