# System interface This document describes the parts of Gizlang that a program uses to reach outside itself: tasks, files, the network, processes, native libraries and live state. The language itself is in [LANGUAGE](../reference/language.md). ## Contents 1. [Tasks](#tasks) 2. [System](#system) 3. [Files and directories](#files-and-directories) 4. [Streams, listeners and sockets](#streams-listeners-and-sockets) 5. [TLS](#tls) 6. [Many connections](#many-connections) 7. [Processes and standard streams](#processes-and-standard-streams) 8. [Native functions](#native-functions) 9. [Block devices](#block-devices) 10. [Live state](#live-state) ## Tasks ```gizlang import {System} from "gizlang:system"; import * as Task from "gizlang:task"; function fib(n: i64): i64 { if (n < 2) { return n; } return fib(n - 1) + fib(n - 2); } function countLines(sys: System, path: string): i64 { return string(sys.readFile(path)!).split("\n").length; } function main(sys: System): void { const a = Task.spawn(() => { return fib(20); }); const b = Task.spawn(() => { Task.sleep(5); return "done"; }); const total = Task.join(a); // Result const lines = Task.channel(16); Task.spawn(() => { Task.send(lines, "hello"); Task.close(lines); return 0; }); const first = Task.receive(lines); // Option const rows = Task.parallel(4, (band: i64): Vec => { return [f64(band)]; }); const counted = Task.thread((own: System): i64 => { return countLines(own, "big.log"); }); const many = counted.join(); // Result print(`${total.unwrap()} ${first ?? ""} ${rows.length} ${Task.join(b).unwrap()} ${many.unwrap()}`); } ``` ### Tasks on one thread `gizlang:task` runs closures as tasks on the thread of the program. | Function | Meaning | | --- | --- | | `spawn(() => {...})` | Starts a task. Returns `Task`. | | `zip(a, b)` | Starts two tasks, joins both, and returns a typed `Pair` or a `Fault`. | | `join(t)` | Waits for the task. Gives `Result`. The value can be taken one time. | | `sleep(ms)` | Suspends the calling task. | | `pause()` | Lets the other ready tasks run. | | `now()` | Reads a monotonic clock. | | `cancel(t)` | Gives fault 12 at the next suspension point of the task. The task unwinds through its ordinary exits. | | `retry(work, n)` | Runs `work` in new tasks until one succeeds. | | `within(ms, work)` | Cancels `work` at a deadline (fault 12). | - `sleep`, `join` and I/O suspend only the calling task. - A task never lives longer than the task that spawned it. When the work of a task returns, its children that are not joined are cancelled, and the task waits for them. The same is true for the process `main`. - A fault ends only its own task. The parent sees it as `Result.err`. - Spawn has the effects `{io, fail}`. The effects of the closure belong to the task that spawns it. ### Channels `channel(capacity)` makes a bounded first-in, first-out queue between tasks. - `send(c, v)` moves `v` in. It waits while the channel is full. It returns false after the channel is closed. - `receive(c)` waits for the oldest value and gives `Option`. It gives `none` when the channel is closed and empty. - `close(c)` ends sending. - A channel belongs to the task that made it. When that task ends, the channel closes and releases the values that it still holds. - A receive that nothing can satisfy is fault 13. - `trySend(c, v)` never waits. It drops `v` when the channel is full or closed. - `respond(c, v)` also drops a late response when the task of the recipient has ended, and returns false. It cannot send into a slot that is used again. Ordinary sends to stale handles still give fault 14. - A value costs approximately 13 ns through a plain channel between tasks. ### Crossing channels and cells `crossing(capacity)` is a channel between threads. - The tasks of any thread (`thread`) can send, receive and close it while the task that made it lives. - Each value is received one time, however many threads receive. - A sent value is marked shared. The sender can keep a copy and change it (a write copies). - Values cannot hold `Ref` values. The compiler refuses them. - A wait on a crossing channel is never fault 13, because a different thread can still send. - A plain channel belongs to its thread. A use from a different thread is fault 14. - A value costs approximately 170 to 200 ns through a crossing channel between threads. `cell(initial)` is one value that every thread reads. It has the role of `ArcSwap` in Rust. - `c.load()` gives the latest stored value. `c.store(v)` replaces it. - `c.close()` drops the value at once, not when the task that made the cell ends. A load or store after it is a fault. A long task that makes cells for data it later retires closes them to give the memory back. - Neither operation waits. A reader keeps what it loaded. - Its values are marked shared and cannot hold `Ref` values, as on a crossing channel. A cell is a crossing channel with one slot: a store replaces it, and a load copies it. - A load costs approximately 11 ns. `Atomic` is one `i64` that every task of every thread changes. The operations use C11 ``. They do not wait and do not allocate. Each one takes an optional memory order, sequentially consistent (`"seqCst"`) by default: a load `"relaxed"`, `"acquire"` or `"seqCst"`, a store `"relaxed"`, `"release"` or `"seqCst"`, and `fetchAdd` and `compareExchange` any of these or `"acqRel"` (`counter.fetchAdd(1, "relaxed")`). The types `LoadOrder`, `StoreOrder` and `Order` admit only the orders valid for each operation. - `atomic(initial)` makes a new atomic. It uses 8 bytes until the process ends. - `a.load()` gives the value. `a.store(v)` replaces it and gives `true`. - `a.fetchAdd(delta)` adds `delta` and gives the value from before the addition. An overflow wraps. - `a.compareExchange(expected, desired)` sets `desired` only if the value is `expected`. It gives `true` if it did. - `atomicGlobal(name)` gives the one atomic of `name` in the process. The first call makes it with the value 0, and every thread gets the same atomic. The call takes a lock, so keep the result and do not call it in a hot loop. - A copy of an `Atomic` refers to the same cell. ### `Ref` values in tasks Closures, results, channels and function values can hold `Ref` values. The heap keeps them live while a task, a channel or a function value owns them. A function value is a root while it lives. Thus a function value stored in the object that its own closure refers to keeps that object until the value is replaced. ### Parallel work `parallel(n, body)` runs `body(i)` for each `i` below `n` on the threads of the process (a pool of one thread for each CPU that it can use). It gives the results in index order as a Vec. - The body can only compute and fail (effects `{fail}`: no I/O, console, `Ref` values or shared state). Thus no order of its calls is observable. - What the body captures is read-only while it runs, and stays shared after. Its counts change atomically. A later write copies it, unless the writer holds its only reference. Then it belongs to the writer again and is written in place. - A fault in any call is the fault of `parallel`. The remaining indexes do not start. - A `parallel` inside a body shares the pool as any other: a sort inside a parallel group uses every CPU. Sections that run at the same time share the threads evenly, and the pool grows to what they want together, up to one thread for each CPU. Past 64 sections at once, a new one runs on the thread of its caller. - A `parallel` also runs on the thread of its caller when the last calls of its body took less time in total than to wake the threads (20 µs). A section that becomes longer shares the remainder. The number of threads is the number of CPUs that the process can use: its affinity, or fewer when the CPU quota of a container (cgroup v2 `cpu.max`) allows less. `GIZLANG_THREADS=N` sets a maximum of N threads, and also limits `Task.cores()` to N. A thread that has finished its part of a section waits 100 µs for the next section before it sleeps. It does this only when that section started within 100 µs of the end of the previous section (only the last gap counts). If not, it sleeps immediately. `fill(total, chunk, body)` makes one Vec in the same way. - `body(start)` gives the `chunk` items from `start`. The last chunk is the remainder. A different length is fault R001. - Each thread copies its chunk into place, so no serial copy joins the parts. - `gizml:tensor` fuses and multiplies through it: `relu(a*b+c)` over 4M floats went from 3.1 ms to 0.45 ms. - Its items own nothing: numbers, bools and records of them. Memory budget: - The threads take slices of the memory budget of the caller as they allocate. Thus one busy body can use what the others do not use. - A value shared between threads (captured, sent on a crossing channel, or stored in a cell) leaves the budget of its World. It counts against one total of shared bytes, with a maximum of the memory limit. - The thread that frees it last returns it there. Thus a writer keeps its budget when a reader frees its old snapshots. ### Threads `thread((sys: System) => {...})` runs work on its own OS thread, in its own World: its own event loop, tasks, handles and memory budget. `sys` is its own root `System`, as a process `main` gets. - It gives `Thread`. `t.join()` waits for `Result`, and suspends only the calling task. - What the closure captures is marked shared, as for `parallel`. The thread can allocate and use `Ref` values inside its own World. - A closure or result that holds references of the spawner is refused. Captured erased functions that can access hidden mutable reference state are also refused. - A handle of the spawner captured into the thread is fault 14 there. - A thread that still runs when the task that started it ends is cancelled, as a child task that is not joined is (fault 12 at its next suspension point, for example a `Ready.wait`). The task then waits for it. - A join is cancelled with the task that joins. - A thread that waits on nothing else is still fault 13. - A `thread` runs on a waiting thread of the `parallel` pool, or on a new one that joins the pool when the work returns. Thus a round trip costs approximately 55 µs. Most of this time is a new World. ## System The authority of a program over the outside world comes as one value: `main(sys: System)` from `gizlang:system`. All other authority comes from it, and the program passes it on explicitly. These types have private fields. Thus library code cannot forge them, and cannot reach anything that it was not given. | Operation | Meaning | | --- | --- | | `sys.files()` | The working directory, as a `Dir`. | | `sys.directory(path)` | Any other directory, by an absolute or relative path (for example, a path from the command line). | | `sys.connect(host, port)`, `sys.listen(host, port)` | TCP. Names resolve through DNS. | | `sys.bind(host, port)` | UDP. | | `sys.listenUnix(path)`, `sys.connectUnix(path)` | Unix-domain sockets. Refer to [Streams](#streams-listeners-and-sockets). | | `sys.arguments()`, `sys.environment(name)` | The command line and the environment. | | `sys.time()` | Wall-clock milliseconds since 1970. | | `sys.entropy()` | An unpredictable 64-bit seed. | | `sys.randomBytes(n)` | `Result`: `n` secure bytes from the OS, through the synchronous CSPRNG primitive of libuv, for keys and nonces. Zero gives empty bytes. A negative length gives `IoError`. An OS failure gives no partial result. | | `sys.usage()` | Process measurements. Refer to [Usage](#usage). | | `sys.stdin()`, `sys.stdout()`, `sys.stderr()` | Standard streams. Refer to [Processes](#processes-and-standard-streams). | | `sys.printError(line)` | Writes `line` and a newline on standard error, for diagnostics. | | `sys.exit(code)` | Ends the process. Refer to [Temporary directories and exit](#temporary-directories-and-exit). | ```gizlang import {System, IoError} from "gizlang:system"; import {Result} from "gizlang:core"; function count(sys: System): Result { const text = try (try sys.files()).readFile("notes.txt"); return Result.ok(text.length); } ``` ### Usage `sys.usage(): Result` observes Linux process RSS, peak RSS, user and system CPU microseconds, and physical disk bytes, from `getrusage` and `/proc`. - The optional cgroup v2 observation includes current memory, CPU microseconds and disk bytes for the full cgroup. The cgroup can contain other processes. - When the process is not in a cgroup v2, the result is `Option.none`. - Memory and disk counters are `Option.none` separately when their controller is disabled at the actual cgroup of the process. The SDK does not use counters from an ancestor in their place. - Other counters that are not available or not valid fail. They do not become false zero values. - These are observations. They are not resource reservations or attribution to each request. ### Memory limit `gizlang:memory.limitBytes()` gives the configured World and shared memory limit of the allocator. It is not free RAM, and it is not the remaining allocation credit. The DB resource policy gets the limit from this owner. It does not parse the environment and defaults again. ## Files and directories ### Paths and process directories - `sys.currentDirectory()` returns the physical working directory of the process, independently of `PWD`. - `sys.absolute(path)` returns an absolute path unchanged. It reads a relative path against `currentDirectory()`. The path does not have to exist. - `sys.realPath(path)` resolves an existing path in the namespace of the process. - They return `Result`. They refuse invalid UTF-8 and paths that are too long. - The OS resolver returns its path and error together. It keeps missing paths, components that are not directories, and symbolic link loops as different errors. - `sys.makeDirectories(path)` creates parent directories. It does not collapse symbolic links and `..` lexically. - `sys.removeTree(path)` and `dir.removeTree(relativePath)` remove children through the directory capability. They never follow child links. A missing tree is already removed. File system roots and dot roots are refused. ### Copy, move and links - `dir.cloneFile(from, target, to, immutable)` creates an exclusive, durable file clone. It returns `FileClone.reflink()` or `FileClone.hardLink()`. It never copies file bytes. Mutable files require Linux `FICLONE` support. Other file systems return an error. Set `immutable` to true only when both namespaces keep the inode immutable for its entire lifetime. This permits `linkat` when reflink is unavailable. Replacement and unlink are safe; writes, truncation and inode metadata changes violate this contract. Hard-link fallback also requires WRITE on the source directory grant. A read grant alone cannot give a destination authority to change the source inode. - `sys.copyFile(from, to)` streams bounded chunks and publishes through `replaceWith`. It copies ordinary permission bits before publication. - `sys.copyTree(from, to)` copies regular files, directories and stored link text, without following links. It refuses to copy a directory into itself. It publishes the completed tree with one rename. - `sys.rename(from, to)` opens the two parent capabilities. `dir.renameTo(destination, from, to)` gives the same confined operation between existing grants. - `sys.move` uses a durable tree copy, then removes the source, when the rename gives `EXDEV`. - `dir.readLink` reads stored link text. `dir.makeLink` stores that text without resolving it. - File and tree publication share the same owner of rename and parent-directory sync. Failed private trees are removed without a change to the destination. - Traversal keeps one bounded cursor page for each directory depth. Regular file data uses bounded stream buffers. ### Touch and append - `sys.touch(path)` updates an existing writable file through its descriptor. It does not create a missing file. - `stream.setMode(mode)` and `stream.touch()` require write authority on the held file. - `APPEND` is an open modifier that requires `WRITE`. POSIX `O_APPEND` implements it. Writes append, also after a seek to zero. - `dir.appendFile(path, bytes)` uses this mode. It works with independent append writers without cooperative locks, and it does not replace an existing inode. ### `Dir` operations `dir.open(path, READ | WRITE | CREATE | TRUNCATE)` opens a file below `dir`. Paths are relative and never leave the directory: no `..`, no absolute paths and no symbolic links. | Operation | Meaning | | --- | --- | | `readFile`, `writeFile` | Whole files. | | `makeDir`, `remove`, `list` | Directories. `remove` removes a file or an empty directory. | | `readBounded(path, limit)` | Admits the size of an opened regular file before it allocates. Then it reads exactly that number of bytes from the same handle, and refuses more bytes. It closes the handle on success and on failure. A negative limit or a file that is too large gives `INVALID_ARGUMENT`. If the file becomes smaller or larger during the read, the result is `IO_FAILURE`. A concurrent change of the same size still requires a content digest of the caller. | | `rename(from, to)` | Renames entries atomically inside the capability. | | `replaceFile(path, bytes)` | Creates an exclusive 0600 staging file in the same directory, writes it, runs fsync, closes it, renames it over the destination, and syncs the parent directory. | | `replaceWith(path, writer)` | The same publication protocol, for a bounded streaming writer. | | `publishFile(path, bytes)`, `publishWith` | The same staging owner, with an atomic rename that does not replace. Existing files, directories and links stay unchanged. Concurrent publishers can admit only one final entry. If the OS has no such rename, the operation fails. It does not use check-then-rename. | | `info(path)` | Examines an entry without opening it or following its final link. `""` examines the `Dir` itself. | | `setMode(path, mode)` | Changes the bits of a file or directory. It refuses links and special files. | | `sub(path)` | Delegates an existing directory without following links. A missing directory is `NOT_FOUND`. | | `walk(path)` | Returns entry paths relative to the `Dir`, sorted depth-first. It excludes the start directory. It includes links, but does not traverse them. | | `entries(path)` | A directory cursor. Refer to [Directory cursors](#directory-cursors). | Replace and publish failures: - A failure before the rename removes the staging file and keeps the old entry. - A failure after the rename reports an uncertain publication. It does not remove the new entry. `FileInfo` has: - `size`. - `modified`, in UTC milliseconds. - `kind`: `"file"`, `"directory"`, `"link"` or `"other"`. - `mode`: the POSIX permission bits, including the set-id and sticky bits (0 to 4095). File streams return the same metadata. `sub` and `walk` accept `""` for the `Dir` itself. No operation normalizes malformed paths or permits `..` or absolute paths. ### Directory cursors `dir.entries(path): Result` opens a confined directory cursor. - `read(limit)` accepts 1 to 4096. It returns a maximum of that number of names. An empty result means the end. - Names exclude `.` and `..`, must be UTF-8, and keep the traversal order of the OS. - A delegated cursor keeps its directory descriptor after its parent capability closes. `close()` releases it. - A concurrent change of the directory does not give a stable snapshot. - `dir.list(path)` uses the same cursor and sorts the full listing in Gizlang. If you need bounded memory, use the cursor directly. ### Temporary directories and exit `sys.temporaryDir()` returns a new private 0700 directory under `/tmp`. - When it closes, or when its owner task ends, its contents are removed recursively, without following links. The same resource destructor as World teardown does this. - Close the temporary parent after its delegated files and directories. If you close it first, their names are removed, also when a delegated descriptor stays open. - `dir.temporaryPath()` returns its absolute path, to pass to a child process. Ordinary and delegated `Dir` capabilities return `DENIED`. - The path is valid only while the original temporary `Dir` stays open. `sys.exit(code)` accepts 0 to 255. It ends the process without a print of the result of `main`. It runs the cleanup of file and `Dir` resources, and kills the children that it has not waited for, as the end of their owner task does. An invalid code is an `IoError`. No cleanup is possible after an external SIGKILL. ## Streams, listeners and sockets ### Streams A `Stream` is a file or a connection. | Operation | Meaning | | --- | --- | | `read(limit)` | Reads. Empty at the end. | | `readAll()`, `write(data)`, `close()` | Read all, write, close. | | `info()` | Files only. The same `FileInfo`. | | `seek(offset)` | Files only. | | `readAt(offset, count)`, `writeAt(offset, data)` | Files only. pread and pwrite: they do not use or move the position of `read` and `write`, and several run at once on one file. `readAt` is short only at the end of the file. | | `readMany(offsets, size)` | Files only. `size` bytes from each offset, in order, in one value: the scattered reads of an index walk in one call. On Linux they go to the device together in one io_uring submission of the calling thread. A part past the end of the file is an error. | | `lock()` | Files only. An exclusive flock, without a wait. `true` when this process now holds it, `false` when another process does. It lasts while the file is open, so one server owns a data folder. | | `timeout(ms)` | Connections only. After the timeout, a read or write that still waits fails with `TIMED_OUT`. Each connection has one timer that is armed again lazily, as the timers of nginx are. | | `unread(bytes)` | Puts back the start of a message. Refer to [Many connections](#many-connections). | | `duplicate(READ \| WRITE)` | Refer to [Descriptors](#descriptors). | - A `Listener` has `accept()` and `port()`. - `sys.listenUnix(path)` and `sys.connectUnix(path)` do the same over a Unix-domain socket. `port()` is 0. When the listener closes, its file is removed. An isolated process can reach a socket in its folder, or in the folder of its database, and nothing outside. - A `Socket` (UDP) has `send(host, port, data)`, `receive(limit)` (a `Datagram` with the `host` and `port` of its sender) and `port()`. ### Errors and ownership - Failures that the program must handle return `Result.err(IoError)`. Its `message()` names the cause. - `NOT_FOUND`, `DENIED`, `BAD_PATH` and `TIMED_OUT` name the codes that a caller can tell apart. - Every handle belongs to the task that opened it, and closes when that task ends. A use of a closed handle is fault 14. - `stream.detach()` gives a TCP stream to no task, thus it stays open after its task ends. `stream.adopt()` gives it to the calling task. The client `Pool` of `gizlang:http` keeps its idle connections in this way. - I/O suspends only the calling task. When the task is cancelled, its pending operation is cancelled. - A handle has one pending operation at a time. A second operation from a different task fails immediately (busy). Thus one task accepts connections and gives them to other tasks. - For files and imported native descriptors, `close()` returns `false` while an operation is pending. Seek, read, write and sync refuse concurrent use with busy. Finish the pending operation before you close. - A positionless OS read that has already started can need input or the end of input to finish, also after the task is cancelled. ### Descriptors `sys.importDescriptor(fd, READ | WRITE)` duplicates a descriptor that an authorized native OS ABI gave, into a stream that a task owns. - It verifies the requested access against the actual descriptor. - It refuses directories and invalid modes. - The caller keeps the ownership of the original descriptor. When either duplicate closes, the other stays open. - Device integrations can use the same API. `stream.duplicate(READ | WRITE)` gives file streams and imported native-descriptor streams independent I/O ownership. It narrows the permissions when you request it. - Each duplicate has its own pending-operation slot and a copy of the logical file cursor. - The OS open-file description state is shared. This includes flock ownership and the behavior of positionless devices. - It refuses TCP, TLS and in-process streams. It cannot bypass TLS or broaden an existing capability. ## TLS `gizlang:tls` secures a TCP `Stream` in place. Only a program that imports the module links OpenSSL. ```gizlang import {System, Stream, IoError} from "gizlang:system"; import * as Tls from "gizlang:tls"; /** A TLS stream to example.com. */ export function secure(sys: System): Stream throws IoError { return try Tls.client(try sys.connect("example.com", 443), "example.com"); } ``` | Function | Meaning | | --- | --- | | `Tls.client(stream, host)` | Verifies the server against the trusted roots of the system, and checks that its certificate names `host`. | | `Tls.clientTrusting(stream, host, pem)` | Trusts only the given certificates. It refuses an empty explicit authority. | | `Tls.credentials(sys, chainPath, keyPath)` | Reads absolute PEM paths and validates the chain and private key. Returns shared `Credentials`. | | `Tls.credentialsOf(sys, chain, key)` | The same from PEM values, such as a certificate that `gizlang:acme` issued. `reload` has no files to read for them. | | `credentials.replace(chain, key)` | Validates PEM bytes, then swaps the TLS context atomically. A failed replacement keeps the current context. | | `credentials.reload()` | Reads the configured files and replaces the context. | | `Tls.server(stream, credentials)` | Secures any accepted byte stream with the current credentials. | | `Tls.serve(listener, credentials, handler)` | Serves HTTPS with the handler of [HTTP serve](../reference/language.md#gizlang-http); `Tls.serveStreamed(listener, credentials, handler, onHead)` adds its request-head hook. ALPN offers h2 and http/1.1, so a browser can choose HTTP/2. | | `Tls.authority(pem)` | An explicit trust authority. | | `Tls.openUrlTrusting(system, url, request, pem)`, `openAtTrusting` | HTTPS with explicit trust. `openAtTrusting` takes an admitted connection address. | - After the handshake, the stream reads and writes through TLS. `close()` sends close_notify first. - Handshake and certificate failures are `IoError` values that name the OpenSSL or X.509 error. - Keep the task that creates `Credentials` alive while servers use it. The runtime releases the context when that task ends, after its child tasks settle. - Replacement affects subsequent server connections. Existing connections keep their context. A listener needs no restart. Write both files before `reload`, or call `replace` with both PEM values. ### Explicit trust - The authority owner admits 1 to 65,536 UTF-8 bytes that contain only public PEM CA certificates and white space. - It refuses leaf certificates, private keys, and malformed or trailing content. - It uses the same OpenSSL context and parser as the handshake. - Explicit trust refuses empty bundles and HTTP before it connects. It checks the URL host. It never uses the system roots in place of the given authority. - `client` is the separate operation with system roots. - `tests/runtime/system/tls-authority/run.giz` verifies these limits and real TLS requests with the test credentials on Linux. ### macOS (not executed) The Darwin source path is not executed yet. - For ordinary `client` calls, it uses the `SecTrust` SSL policy of the operating system. It keeps the configured DNS name or IP reference. - It does not copy anchors. It does not read the OpenSSL root-path environment overrides, and it does not use the default OpenSSL files. Linux keeps its OpenSSL system-root path. - Explicit authority keeps its separate PEM policy. - Peer conversion admits a maximum of 32 certificates, 65,536 DER bytes for each certificate, and 262,144 DER bytes in total. Native intermediate fetch is disabled. - Evaluation uses the bounded pending quota of I/O services and the libuv worker pool. Only the calling task waits. Cancellation settles the worker before trust is freed or a TLS layer is admitted. - A `SecTrustEvaluateWithError` that is already running cannot be interrupted. Thus its settlement is not an absolute guarantee of wall time. - Legacy renegotiation is disabled for this system-trust context. ## Many connections A server with many connections keeps them in a `Ready` set, not in a task for each connection, as nginx keeps them in epoll. One task, or a few, can serve thousands of connections in this way (`serve` of `gizlang:http`). 1. `ready(idleMs)` makes a set. It grants nothing. 2. `set.accept(listener)` accepts into the set. 3. `set.wait()` returns the next connection that has bytes or has ended. The data is read ahead, so its `read` returns immediately. 4. The connection now belongs to the task that took it (a handler fault closes it). 5. `set.watch(stream)` gives it back, to be reported one more time. `set.channel(c)` puts a crossing channel in the set, with no socket. Watch the stream that it returns. `set.wait()` returns that stream, one time for each watch, when a different thread sent a value or closed the channel. Drain the channel with `Task.tryReceive`, then watch the stream again. One set watches a channel. A hand-off from a different thread and its reply cost approximately 4 µs, with no system call for each value. `stream.unread(bytes)` puts back the start of a message that a parser cannot finish yet. Those bytes are read first, but they do not report the connection again. If the connection ends, or stays silent longer than `idleMs`, the reads report that. ### A server of your own protocol `serveConnections(set, turn)` is the one scheduler of every server here (`Http.serve` is a `turn` on it). Fill a `Ready` set with `set.accept(listener)`; `turn(stream)` reads what the connection sent, answers, gives the connection back with `set.watch(stream)` (or closes it) and returns the number of requests it served. It starts `WORKERS` tasks and adds one when a connection is taken while none waits, up to `MOST_WORKERS`. A fault in a turn ends only its connection. When the listener closes, the turns in flight finish and the call returns the sum. `tests/sdk/system/protocol` is a complete line protocol. ### Behind a balancer (PROXY protocol) A TCP balancer (HAProxy, AWS NLB, nginx `proxy_protocol on`) writes a PROXY header (version 1 or 2) first on each connection it forwards. `listener.proxyProtocol(true)`, set before the first accept, makes `Http` read it: the client address in the header becomes `stream.peer()` and `Http.PEER_HEADER`, once for the connection. A connection without a header is closed, and a header later in the connection is not HTTP. A LOCAL or UNKNOWN header (the balancer's health check) keeps the socket's peer. `Web.Serving.proxyProtocol` does this for `Web.listen`. Turn it on only for a port that nothing but a balancer can reach, since a client could write the header itself. `proxyHeader(bytes)` parses a header for a protocol of your own; the header must come whole in the first read (it is at most 536 bytes). ### One reactor for each core `serveThreads(sys, count, work)` runs `work(own, i)` for `i` in `0..count`: index 0 on the calling thread, the others on threads of their own (`Task.thread`), each with its own `System`, `Ready` set and memory. Each call listens on the same port with `listenShared`, and the kernel spreads the connections among them. This is the way for a server of any protocol, not only HTTP: ```gizlang import {System, IoError, serveThreads} from "gizlang:system"; import * as Task from "gizlang:task"; import * as Http from "gizlang:http"; function handlerOf(own: System): (request: Http.Request) => Http.Response { return (request: Http.Request): Http.Response => { return Http.text(200, "hello"); }; } function main(sys: System): void { const port = 8080; const ended = serveThreads(sys, Task.cores(), (own: System, i: i64): Result => { const listener = own.listenShared("0.0.0.0", port)!; return Http.serve(listener, handlerOf(own)); // or your own loop over ready(), accept and wait }); print(`${ended.unwrap()} requests`); } ``` - It returns the sum of the counts, or the first error. A thread that faults is `IO_FAILURE`. - Make the handler (and any connection to a database) inside `work`, for each thread. A `Ref` or a handle of one thread is not reachable from another one. State that every thread sees lives in a database. - A listener handed to the process (`sys.handed`) is index 0's, so it must be made with `listenShared`: the other threads join its port. - `Web.listen` does this with `Serving.threads` (default `Task.cores()`). ### Which servers are alive (`gizlang:mesh`) `Mesh.join(sys, id, host, port, meta, seeds, intervalMs, holdMs)` makes this process a node of a service. Each interval it exchanges its view with every node it knows (and the seeds) in one short TCP connection. A node not heard of for `holdMs` is dead. - `mesh.alive()` lists the live members, this node first. - `mesh.owner(key)` is the live member with the highest hash of (member, key) (rendezvous hashing). A join or a leave moves only the keys of that member, and all nodes with the same view agree. - `mesh.leave(sys)` stops answering and probing, and tells each member, which drops the node at once. Close the listener of the service first: the requests it answers still finish. - `Http.Balancer.follow(mesh.alive())` makes the live members the upstreams of a balancer, at the port in their `meta`. `tests/sdk/http/cluster` is a service of three servers where one leaves and no request is lost. - `meta` is one word that the node tells the others, such as the port of its service. A mesh belongs to the thread that made it. ### Worker processes `GIZLANG_WORKERS=N` (or `auto`, one for each CPU) runs `main` in N processes, as nginx runs workers. - Each worker has its own memory and event loop. - Listening sockets share their port, and the kernel distributes connections among the workers. - The first process forwards SIGINT and SIGTERM to the workers, and exits with the first failure. - Workers share nothing else. State between them goes through a store or messages. ## Processes and standard streams ### Standard streams `sys.stdin()`, `sys.stdout()` and `sys.stderr()` give `Result`. - Reads, also from piped stdin, suspend only the calling task. - When you close one, its capability closes. The standard descriptor of the process stays available to inherited children. ### Spawn `sys.spawn(program, args, options)` finds `program` in `PATH` and gives `Result`. Arguments are passed literally, without a shell. `spawnOptions()` starts with inherited streams, the current working directory and the current environment. A `SpawnOptions` has: | Field | Meaning | | --- | --- | | `stdin`, `stdout`, `stderr` | `Stdio.inherit()`, `Stdio.pipe()`, `Stdio.ignore()` or `Stdio.stream(stream)`, independently for each. | | `cwd: Option` | The working directory. | | `environment: Vec` | Additions written as `NAME=value`. The last addition replaces that name. | | `isolated: bool` | Refer to [Isolation](#isolation). | | `handed` | Refer to [Handed resources](#handed-resources). | | `limits` | Refer to [Resource limits](#resource-limits). | `Stdio.stream(stream)` gives a native file or an imported descriptor to the child as the selected standard descriptor. It requires the corresponding read or write access, and the full physical access rights of the OS descriptor, as a stream grant does. Native child code can use either direction, whatever the standard name of the descriptor is. Thus a narrowed capability cannot export broader access. ```gizlang import {System, IoError, SpawnOptions, Stdio, spawnOptions} from "gizlang:system"; export function runBoth(sys: System): i64 throws IoError { const output = try sys.run("printf", ["hello %s", "world"]); const defaults = spawnOptions(); const options: SpawnOptions = { ...defaults, stdin: Stdio.pipe(), stdout: Stdio.pipe() }; const child = try sys.spawn("cat", [], options); print(string(output.stdout)); return (try child.wait()).exitCode(); } ``` ### Child processes A `Child` has: - `stdin()`, `stdout()` and `stderr()` as `Option`. Only piped streams are present. - `pid()`. - `wait(): Result`. A wait suspends only its caller, and you can repeat it. `ExitStatus.code(n)` is a normal exit. `ExitStatus.signal(n)` is a signal that ended the child. - `kill(signal): Result`. - `close()`: kills and reaps a child that still runs. The child and its pipes belong to the task that spawned them. When that task ends, it kills and reaps a child that still runs. `sys.run(program, args)` is written in Gizlang over spawn and tasks. It ignores stdin, drains both output pipes concurrently, closes them, and returns `Result` with `status`, `stdout` bytes and `stderr` bytes. ### Signals A program reads process signals as a stream in a task. It does not install handlers: ```gizlang import {System, TERM, INT, HUP} from "gizlang:system"; function main(sys: System): void { const signals = sys.signals([TERM, INT, HUP])!; // ... start the server tasks ... const first = signals.next()!; // the task waits here // Write a checkpoint, close the listeners, then return. } ``` - `sys.signals(ids)` gives `Result`. The ids are the Linux signal numbers (`HUP`, `INT`, `QUIT`, `USR1`, `USR2`, `TERM`, `WINCH` are named; the runtime maps them on other platforms): every signal a handler can catch except `CHLD` (child processes use it) and the faults `ILL`, `BUS`, `FPE` and `SEGV`. Another id gives `EINVAL`. On Windows only `INT` and `TERM` come. - `next(): Result` gives the next signal that the program has not read yet. It suspends only its caller. A signal that comes when no task reads is kept for the next read. If the same signal comes two times before a read, the program can see it only one time, because the kernel keeps one pending signal of each kind. - After a program subscribes to a signal, that signal does not stop the program. The program stops when `main` returns. - `close()` stops the subscription. A task that waits in `next()` then gets an error. The subscription also closes when the task that opened it ends. The runtime installs one handler for each signal. The handler only counts the signal and wakes the event loop of each reader, so no Gizlang code runs in a signal handler. The runtime does not use `signalfd`: `signalfd` needs the signal blocked in all threads from the start of the process, and threads that the runtime does not start (the libuv pool, a host program) do not block it. ### Isolation `isolated: true` starts a Gizlang program on Linux that is confined to `cwd`. `cwd` is required. Before any of its code runs, the program: 1. Moves into its own user, mount, network, IPC, UTS, cgroup and PID namespaces. 2. Makes `cwd` its whole file system, and mounts `cwd` noexec. 3. Drops every capability. 4. Installs a seccomp filter. The filter refuses mounts, namespaces, credentials, kernel modules, tracing, bpf, perf, userfaultfd, keyrings, io_uring and new machine code (executable memory that is not from a file, `mprotect` to executable, memfd). The implementation is `runtime/isolate.c`. - The network of the program has no interface up. Thus only its pipes and the listeners that it was given can reach anything. - No process outside is visible or can receive a signal. This is important because every app runs as the same user. - The process that its parent started waits outside. It relays SIGTERM, SIGINT, SIGHUP, SIGQUIT, SIGUSR1 and SIGUSR2. It ends with the status of the program, or 128 + the signal when a signal ended the program. - Where the kernel cannot do this (no unprivileged user namespaces), the child exits with 71 without running. It never runs without confinement (`tests/tools/isolate`). > **Warning:** Isolation is a first step to a process for each tenant. It has had no > independent review. ### Handed resources `handed: [ProcessGrant.listener(listener), ProcessGrant.stream(stream), ...]` explicitly grants a maximum of 16 resources to any child, isolated or not. - Both kinds use descriptor slots 3, 4, ... in list order. - A Gizlang child takes a listener one time with `sys.handed(index)`. Native consumers use the descriptor ABI. - Stream grants must keep the full access rights of the actual descriptor. A narrowed capability cannot export broader OS access. - The parent keeps its own resource. - Listeners stay bound to the network of the parent. Thus an isolated child can serve connections that it cannot open itself. - Until the parent calls `accept`, connections go to its children (`start_listening` in `runtime/io.c`). ### Isolated live host The live host (`_live-host`, which `giz live` and the web stack run) keeps its builder outside when it is spawned isolated. - The builder watches the sources and runs the compilers. - Only the forked host, which loads generations and runs the program, is confined. - Its work folder must be inside `cwd`. It is the only folder from which the host can execute files, and it is read-only there. Thus the host runs only code that its builder wrote (`runtime/live.c`). ### Resource limits `limits: Limits(memory, cpu, processes)` runs a Gizlang child in its own cgroup (v2) below the cgroup of the spawner. The units are bytes, thousandths of a core, and processes and threads. 0 leaves a limit off. - The cgroup of the spawner must be delegated (systemd `Delegate=yes`). The first time, the processes of the spawner move to a leaf `self`. - The child moves itself in before any of its code runs. Thus nothing that it forks starts outside. If it cannot move in, it exits with 71. - When it uses more than its memory, the whole cgroup is killed. - When the child ends, what it left running is killed and the cgroup is removed. - A live host enters with its builder, so the compiles of an app also count. - Where the cgroup of the spawner is not delegated, the spawn fails (`tests/tools/isolate`). ## Native functions `declare function name(params): T effects {...} from "library";` binds a C function. It is linked with `-llibrary`. The `from` clause is optional for the C library. ```gizlang declare function cbrt(x: f64): f64 effects {} from "m"; declare function compress2(dest: mut bytes, destLen: mut u64, source: bytes, sourceLen: u64, level: i32): i32 effects {} from "z"; type Tm = { sec: i32; min: i32; hour: i32; mday: i32; mon: i32; year: i32; wday: i32; yday: i32; isdst: i32; gmtoff: i64; zone: u64; }; declare function gmtime_r(time: mut i64, out: mut Tm): u64 effects {}; declare function qsort(base: mut bytes, count: u64, size: u64, compare: (a: u64, b: u64) => i32): void effects {}; ``` The declared types are the C types: | Gizlang | C | | --- | --- | | `i8` to `i64`, `u8` to `u64`, `f32`, `f64`, `bool` | The integer, float or `bool` of that width. A value that does not fit the parameter fails (R003). | | `string` | A `const char *` to a copy that ends with NUL. A string that contains NUL fails (R003). As the result, the returned text is copied. NULL is `""`. Invalid UTF-8 fails. | | `bytes` | A pointer to the first byte, or NULL when empty. Pass the length as a separate argument. | | A record of the scalars above | A pointer to a C struct with those fields in order. | | `mut` scalar | A pointer to it: an out-parameter of C. It is read back into the local after the call. | | `mut bytes`, `mut` record | C writes into the buffer or struct of the local. | | `u64` | Any other pointer, for example a handle that C returns (`sqlite3 *`). | | `(a: T, ...) => R` | A C function pointer. The argument names a top-level function whose types are the value types of the signature (`i64` for `i32`). A maximum of 8 scalar parameters. `R` is a scalar or `void`. | `giz bind HEADER LIBRARY NAME...` writes these declarations from a C header ([TOOLS](tools.md#c-and-typescript-bindings)). ### Calls and callbacks - A `mut` argument is a `let` local. - A callback runs on the calling task. After a fault inside a callback, later calls return 0 to C without running. The fault fails the caller of the native call when C returns. - A callback result that does not fit its C type is R003. - Pointers that a callback receives are `u64`. To read through one, declare a C function such as `memcpy`. ### Authority The checker cannot see native code. Thus the effects of a native function are its declared row, which is `{io, fail}` by default. A native function has all the authority of the process. - Only the application can declare a native function: its entry module, or a package module whose manifest entry grants the library (`"native": ["m"]`, or `"c"` for functions without `from`). Any other declaration is `E_MODULE`. - The manifest of a dependency can grant only what its parent granted to it (the same `"native"` on its entry in `dependencies`). If not, the package is `E_PACKAGE`. - Symbols that start with `gizlang_` are runtime services with the Gizlang calling convention. Only the SDK declares them. ### JavaScript and Python A program calls JavaScript, TypeScript and Python in its own process through the library `gizvm` (`vm/main.giz`). The VM runs them ([VM](vm.md)). No other process starts. ```typescript import {System} from "gizlang:system"; import * as Vm from "gizvm"; import {Value} from "gizvm"; function base(sys: System, path: string): string throws string { return Vm.javascript(sys).load("node:path").call("basename", [Value.text(path)]).toText(); } ``` - `Vm.javascript(sys)` and `Vm.python(sys)` open the realm of the language. A process has one of each. The first open starts it. TypeScript runs in the JavaScript realm. - `Realm.load(SPEC)` gives a module, which loads once. JavaScript resolves SPEC as Node does from the working folder (`node:path`, a package in `node_modules`, `./lib.ts`). Python takes a module name (`json`) or a file (`./model.py`). `Realm.module(NAME, SOURCE)` makes a module from source text, for example a text module of the program. - A module, a function, a class and every other object come back as a handle (`Value.object`). `get`, `call`, `callNamed` (Python keyword arguments), `invoke` and `construct` use a handle. `release` frees it. - Plain data is copied at each call: `none`, `boolean`, `integer`, `float`, `text`, `binary` (`Uint8Array`, `bytes`), `list` (an array, a list or a tuple) and `map` (a plain object, or a dict with text keys). A value in a cycle, or nested deeper than 64, stays a handle. - A returned promise or coroutine is awaited. An exception in the realm is thrown as a `string`: its message and stack. - Tasks enter a realm one at a time. Each task's stack is scanned by the VM's collector. - A `declare function` from a runtime module (`from "node:path"`) is `E_MODULE`. - `tests/vm/embed/main.giz` covers these rules. ### Import limits The `"imports"` field of a manifest entry limits what a module can reach to the modules that it lists (`"gizlang:core"`, or a file by its path in the package, `"./stack.giz"`). - The limit applies to imports, re-exports and qualified names (`_gizlang_sdk_text__join`, the import that the projection writes inline). - Anything else is `E_MODULE` at the import, or an unknown name. - An entry with `"imports"` is code that the application does not trust, for example the test file of a hosted app that runs as the entry. Thus it too can declare C functions only with `"native"`. A script folder can grant its own files in this way. Put a `package.json` in the folder that lists them by `"path"`, with `"imports"` or `"native"` and no `"sha256"`. `gizlang` pins the files when it builds the script. Thus a host that writes a file and its grant again together never sees them out of step. This is for a host that confines each app that it runs (`tests/tools/scripts`). ## Rings `System.ring(entries)` makes an io_uring of 128-byte submissions for passthrough commands (`IORING_OP_URING_CMD`): the ublk control and queue commands, NVMe passthrough. It is Linux only; elsewhere it fails with errno 38 (`ENOSYS`). An isolated process cannot make one: its seccomp filter refuses io_uring. | Item | Meaning | | --- | --- | | `ring.command(fd, op, userData, w0, w1, w2, w3)` | Queues command `op` on descriptor `fd` with `userData` (0 to 2^31 - 1) and 32 bytes of command data in four little-endian words. False when the ring is full. | | `ring.enter(wait)` | Submits the queued commands and waits until `wait` completions are ready (0: no wait). Gives the number submitted. An interrupted wait fails with errno 4 (`EINTR`); call it again. | | `ring.next()` | The oldest completion, consumed: `{userData, result}` (`result` below 0 is -errno), or none. | | `ring.close()` | Closes the ring. | | `sys.mapShared(fd, offset, length, writable = false, alignment = 1)` | `length` bytes of `fd` from `offset`, mapped shared and read-only unless `writable`, at an address that is a multiple of `alignment` (a power of two; 1 for any). Gives the address, which `Native.readInto` reads. `sys.unmap(address, length)` unmaps it. | | `sys.pageSize()`, `sys.processId()` | The page size in bytes, and the id of this process. | A ring belongs to one thread: its submitter and its reaper. ## Block devices `fs/ublk.giz` makes a local block device, `/dev/ublkbN`, through the Linux ublk driver. Your Gizlang code serves its requests. It is a storage module, as `fs/fuse.giz` is, not an SDK module. Use it to attach a volume on the local host. For a volume on a different host, use NBD: it is the network wire only. ```gizlang import {System, READ, WRITE} from "gizlang:system"; import {Result} from "gizlang:core"; import * as Ublk from "gizfs:ublk"; /** Serves the file disk of FOLDER as a block device, then removes the device. */ export function serveDisk(sys: System, folder: string): void { const defaults = Ublk.options(16384); // 8 MiB in 512-byte sectors const options = {...defaults, queues: 2, depth: 32}; const device = Ublk.create(sys, options, (own: System, queue: i64): Ublk.Handler => { const file = own.directory(folder)!.open("disk", READ | WRITE)!; return (request: Ublk.Request, data: mut bytes): Result => { if (request.op === Ublk.READ) { data = file.readAt(request.offset, request.length)!; return Result.ok(data.length); } if (request.op === Ublk.WRITE) { return Result.ok(file.writeAt(request.offset, slice(data, 0, request.length))!); } return Result.ok(0); }; })!; // ... use device.path ... device.stop(sys)!; device.delete(sys)!; } ``` | Item | Meaning | | --- | --- | | `Options` | `sectors` (512-byte sectors, as the kernel counts), `logicalBlockSize` (512 to the page size), `queues` (1 to 4096), `depth` (requests in flight on each queue, 1 to 4096), `maxIo` (the largest request in bytes, a multiple of the page size, at most 32 MiB), `flags`. `options(sectors)` gives 4096-byte blocks, one queue, depth 64, 512 KiB and `VOLATILE_CACHE`. | | `flags` | `VOLATILE_CACHE`: the kernel sends `FLUSH`, and `FUA` writes. `DISCARDS`: the kernel sends `DISCARD` and `WRITE_ZEROES`. `READ_ONLY`. | | `create(sys, options, serve)` | Adds, configures and starts the device. It returns when the kernel has made `/dev/ublkbN`. Gives `Result`. | | `Request` | `op` (`READ`, `WRITE`, `FLUSH`, `DISCARD`, `WRITE_ZEROES`), `flags` (`FUA`), `offset` and `length` in bytes, `queue`, `tag`. | | `Handler` | `(request, data: mut bytes) => Result`. It gives the bytes done (0 for `FLUSH` and `DISCARD`), or a positive errno. The kernel sees `-errno`. A fault in the handler gives `-EIO` for that request only. | | `device.id`, `device.path` | The device number `N` and `/dev/ublkbN`. | | `device.stop(sys)` | Stops the device. `/dev/ublkbN` goes away and the queue threads end. Gives the number of requests served. | | `device.delete(sys)` | Deletes a stopped device. | | `remove(sys, id)` | Deletes device `id`: a stopped one, or one whose server process ended. | Queues: - Each queue runs on its own thread (`Task.thread`). `serve(own, queue)` runs on that thread first and makes the handler of the queue. What it opens, and the `Ref` values that it makes, belong to that thread. Thus `serve` must not capture `Ref` values of the caller. Use a crossing channel or a cell to reach the caller. - A queue serves one request at a time, in the order that the kernel completes them. - `data` is the buffer of the queue. It has `maxIo` bytes, and the queue uses it again for each request. For `WRITE`, its first `length` bytes are the data. For `READ`, write the data into it, or put a new value of at least the bytes done in `data`. - The memory of a queue is fixed: `depth * maxIo` bytes of native buffers, one `data` buffer and the descriptor map. A request allocates nothing in the queue loop. - The task that calls `create` owns the device. Stop the device before that task ends. If you do not, the task waits for the queue threads, and they wait for the kernel. - A fault that escapes a queue ends the process with exit code 70 (fail-stop). The kernel fails the requests in flight of a device only when its char device closes, that is, when the server process ends. Thus do not read or write a device from the process that serves it: when a request hangs, the reader waits in the kernel and the process can never end. Serve the device in its own process, and kill that process when it stops making progress. - An unknown `op`, or a request longer than `maxIo`, gets `-EOPNOTSUPP` and does not reach the handler. Access: - Every device is unprivileged (`UBLK_F_UNPRIVILEGED_DEV`). The process needs no capability. It needs read-write access to `/dev/ublk-control`, and to `/dev/ublkcN` and `/dev/ublkbN` of the devices that it makes. A udev rule gives this access. Without it, `create` fails with errno 13. - The kernel module `ublk_drv` must be loaded. Linux 6.5 or later is necessary. - `gizlang:system` gives the io_uring that carries the commands ([Rings](#rings)) and the read-only map of the descriptors of a queue (`System.mapShared`). Everything else is in `fs/ublk.giz`: the control commands (`ADD_DEV`, `SET_PARAMS`, `START_DEV`, `STOP_DEV`, `DEL_DEV`), `FETCH_REQ` and `COMMIT_AND_FETCH_REQ`, and the request decoding. - Without Linux, every call fails with errno 38 (`ENOSYS`). - An isolated process cannot make a device: its seccomp filter refuses io_uring ([Isolation](#isolation)). - `tests/fs/block` needs the `ublk` capability. It serves a device from a file on two queues in a child process, writes and reads it with `O_DIRECT` from the test process, flushes, reads a block whose handler faults (EIO, and the device keeps serving), stops and deletes it. Then a handler hangs: a watchdog kills the server after 5 s with no progress, the reader gets EIO and `remove` deletes the device. ## Live state `Live.state(name, initial)` from `gizlang:live` returns the managed object (`Ref`) bound to `name`. The first time, it creates the object from `initial`. Under `giz live`, every generation sees the same named state. ```gizlang import * as Live from "gizlang:live"; import * as Memory from "gizlang:memory"; type Counter = { count: i64; label: string; }; // label added while running function step(): i64 { const state = Live.state("counter", Counter(0, "new")); const c = Memory.load(state); Memory.store(state, Counter(c.count + 1, c.label)); return c.count; } ``` When a generation changes the type of the state, the stored value moves into a new object: - Each field whose name and type are unchanged keeps its value. A field that `giz edit` renamed keeps its value through its `@was` doc tag. - Each other field gets its value from `initial`. - A field whose value is not kept (removed, or with a changed type) is named on stderr: `gizlang: live state NAME: field F not kept: removed, or its type changed`. Thus a value is never dropped silently. - A deployed host (`giz live --accept`, [CLI guide](../guides/command-line.md#deploy-to-a-live-host)) keeps every field: it refuses a change that drops one. The bind fails, the stored value stays, and the generation that asked is withdrawn (`runtime/state.c`, `runtime/live.c`). `Live.generation()` is the number of the newest generation (1, 2, ...; 0 outside a live host). A program that keeps a value made at its start compares it to see that a new generation took over (`Web.Generation` does). `Live.reject()` withdraws the generation that runs when its startup failed: true on a deployed host (the one before it serves again), false on a development host, which keeps the generation so the next edit can fix it. A function is replaced only by a function with the same name and the same parameter and result types, names included. Thus a call that already runs in old code continues to call old code, and never receives a value of a changed type. ### WebSocket compression `Http.webSocket` accepts `permessage-deflate` (RFC 7692) when the client offers it. It uses no context between messages (`*_no_context_takeover`), and sends a message of 128 bytes or more compressed (RSV1). `Http.openWebSocket` offers it when its headers carry `Sec-WebSocket-Extensions: permessage-deflate`; `send` and `receive` then compress and inflate on their own. ### gRPC and protobuf (`gizlang:grpc`, `gizlang:protobuf`) `gizlang:protobuf` is the wire format: a message is a `Vec`; `encode` and `decode` convert it, `varintField`, `stringField`, `messageField`, `packedField`, `fixed32Field` and `fixed64Field` make fields, and `last`, `all` and `unpacked` read them. The schema stays in your code: you read the numbers you know and skip the rest. `gizlang:grpc` runs gRPC on HTTP/2. A server is `Http.serve(listener, Grpc.service([Grpc.Method(path, handler)]))` over h2c, or over TLS that chose h2. A handler gets the request and its messages, sends replies with `replies.send` as it makes them, and returns a `Status`; the status goes out in the trailers (`Http.Exchange.trailers`). A client is `Grpc.dial(sys, host, port)` (or `Grpc.use` on a TLS stream) and `channel.call(path, messages, metadata)`, which sends the request messages and returns all replies: unary, client-streaming and server-streaming calls fit. A call whose requests and replies depend on each other while it runs is not supported. Messages are not compressed. `Http.openH2` is the HTTP/2 client under it. ### OCSP stapling `credentials.staple()` asks the OCSP responder named in the certificate (its AIA extension) for the certificate's status, checks the answer (signed by the issuer, good, current) and keeps it: a client that asks for the status gets the answer in the TLS handshake. The issuer must be the second certificate of the chain. It returns the seconds until the answer's next update; `Tls.keepStapled` asks again at half that time, in a task of its own. Replacing the credentials drops the answer, since the certificate changed. ### HTTP/3 `gizlang:http3` serves HTTP/3 (RFC 9114) over the QUIC server of OpenSSL 3.5 or later. The runtime finds that server at run time, so a host with an older OpenSSL still runs everything else; `Http3.supported()` says whether HTTP/3 is available (`bin/giz openssl` builds OpenSSL 3.5.4 into `$GIZLANG_CACHE/openssl`, and the runtime, the CLI and the live host link and rpath to it). `Http3.open(credentials, host, port)` listens on UDP (host `""` is every address, port 0 any) with the certificate of the TLS credentials, and `server.serve(sys, reply)` answers every request with `reply: (Http.Request) => Http.ReplyPlan`, the same reply plans as HTTP/2 (`Http.replyOn`), until `server.close()`. A request body streams through `head` (`Http.RequestHead`) like an HTTP/2 upload, or is read whole up to `Http.MOST_BODY` without it. Header fields use QPACK with the static table only (the server announces a dynamic table of capacity 0). `Web.listen` with a domain opens HTTP/3 on the same port number over UDP when it is available, and the TLS answers carry `alt-svc: h3=":PORT"; ma=86400`. The peer address of an HTTP/3 request is not available (OpenSSL has no getter for it). A certificate that is renewed (`Credentials.replace`) reaches an open listener: the next handshake offers it. A test needs an HTTP/3 client: `tests/sdk/http3` runs the client in `GIZLANG_H3_CLIENT` (quic-go's `http3.Transport`) and says so when it or QUIC is missing. ### Gateway `gizlang:gateway` is a reverse proxy for a host: `giz tools/gateway/main.giz CONFIG.json`. The JSON file names the ports (`http`, `https`, `host`), a `metrics` address, an absolute `accessLog` path (JSON lines, reopened every 30 s), a `challenges` folder (HTTP-01 files for ACME, and every other plain-HTTP request is redirected to HTTPS when `https` is set), `headers` added to every answer, `certs` (`names`, `chain`, `key`, or `names` and `acme` (`folder`, `directory`, `contact`) for a certificate that the gateway issues and renews itself by HTTP-01 on the plain port: files `chain.pem` and `domain.key` in the folder; the first is issued before the ports open, the others while the gateway serves (also one that a reload adds, which then loads with no restart); each is checked every 6 h, after a failure too, and renewed in the last third of its life; the first is the default and the others are picked by the TLS server name, exact or `*.` wildcard, for TLS and for QUIC), `limit` (`requests`, `windowMs`, `proxies`: at most that many requests from one client in the window, before any route; the client is the peer, or the last `X-Forwarded-For` hop when the peer is one of `proxies`, whose `X-Forwarded-Proto` is also kept; over it, 429 with `Retry-After`; each serving thread counts on its own, and a reload keeps the counts while the limit stays the same) and `routes` (`name`, `hosts`, `prefix`, `stripPrefix`, `upstreams`, `standby`, `holdMs`, `keepHost`, `headers`, `priority`, `responseMs`, `retry`, `errors`, `redirect`, `pathRegex`, `hostRegex`, `basicAuth`). A route is chosen by `priority` (higher first), then host fit (exact 2, wildcard 1, any 0), then the longest prefix, on whole path segments. `standby` upstreams get a request only when no upstream can take it. An upstream that refuses the connection is held back, and the request goes to the next one, whatever its method, since nothing was sent. `responseMs` limits the wait for a response head. `retry` (`attempts`, `statuses`) sends an idempotent request again, to the next upstream, when an upstream answers with a listed status. `errors` (`statuses`, `upstream`) answers with the page the upstream serves at `/`, under the original status, for a listed status or when no upstream can be reached (502). `redirect` (`to`, `status`) replaces the upstreams. `pathRegex` and `hostRegex` must also match the path and the host. `basicAuth` (`realm`, `users`: htpasswd lines with `$2y$` (bcrypt, `Crypto.bcrypt`; checked once a password, then remembered by its SHA-256), `$apr1$`, `$1$` or `{SHA}` hashes; `Crypto.md5Crypt`) answers 401 without a listed login. The file is read every 2 s: a changed file replaces the routes, a bad one is refused and the old routes stay; a changed certificate file is reloaded. Each answer is relayed as the upstream sends it (`Http.Balancer.relay`: status and headers at once, then each body piece, no size bound); an `Upgrade` request (a WebSocket) becomes a byte tunnel after the upstream's 101. The upstream gets `X-Forwarded-For` (the peer appended to the client's chain), `X-Forwarded-Proto` and `X-Forwarded-Host`; other client-sent forwarding fields are dropped (`Http.forwarded`). A request body that is chunked or of 256 KiB or more goes to the first upstream that can be reached as it arrives (`Http.Balancer.forward`; chunked stays chunked, nothing is held, no size bound; a body sent cannot be sent again, so a failure after its first byte is a 502); a smaller body is read whole before it is sent on. The metrics are `gateway_requests_total` (route, status class), `gateway_request_seconds`, `gateway_unrouted_total` and `gateway_limited_total`. `tests/sdk/gateway` covers the file checks, routing, the limit, the relay, the tunnel, the log, the metrics, the certificate choice and the reload.