Tools

All tools are Gizlang programs in tools/. bin/giz builds the toolchain (the compiler and the C runtime). It then runs the Gizlang command line, tools/cli/main.giz, for every command. A compiler edit starts from the build folder's last compiler, which is a fixed point of the previous sources. If that compiler emits the same IR, it is linked again. If not, the new IR is linked, and that compiler must compile its sources to the same IR (or link once more and do so). New syntax that the last compiler cannot compile, or no fixed point, starts from the seed (bootstrap/compiler.ll.xz). giz bootstrap always starts from the seed. On a shared server, a comment takes 6 s (the seed took 55 s). A code change compiles the one unit of each changed function and links in about 1.3 s (ir_objects, a unit per function; llvm-split's parts all missed the ThinLTO cache, 28-37 s). The tools and the prebuilt SDK are keyed by the compiler's IR (compiler_ir), so a comment change rebuilds neither.

Contents

1. Commands
2. REPL
3. Formatter
4. C and TypeScript bindings
5. JSON codecs
6. Language server
7. Debugger
8. Private native release construction

Commands

The command table in tools/cli/commands.giz is the one source of the commands. Each entry has the name, aliases, flags, operands, summary, examples, exit codes, the JSON output schema and the access (public, private or internal). These all come from it:

• Dispatch. gizlang runs only a command in the table. An unknown command exits with status 2.
• giz help lists the commands by group; giz help --all adds the commands that work on this repository. giz help COMMAND and giz COMMAND --help show one command.
• giz completion bash.
• giz capabilities (gizlang.capabilities.v1).
• The commands of giz doc --json, and thus the command page of the site.

tests/tools/doc runs each example of the table as a test.

Reference

giz doc assembles the reference of the SDK and the commands, as one versioned JSON (gizlang.reference.v1). The developer site, LSP hover, llms.txt and MCP read it. They do not parse sources.

The compiler owns the facts. The outline gives each public declaration, its types, its effects and its doc comment. giz doc adds:

• The signature, from the declared source span, without the body, export or the native binding. A record with a hidden field (_name) shows only its name. A _ name is not in the reference.
• refs: each [Name] in a doc text, and each @see name, resolved to a declaration id (gizlang:json#Json.get). A link [text](url), a code span, a fenced block and an index (v[i]) are not references.
• The checks of --check:

tests/tools/doc holds the coverage ratchet of the SDK. Raise its numbers when you add docs. A release carries the reference as share/gizlang/reference.json, so an installed giz doc needs no SDK sources. Packages and registries are in LANGUAGE.

REPL

giz repl reads, checks, runs and prints (tools/repl/main.giz).

• Each entry is checked and run as part of one program.
• Declarations (import, function, type, interface, declare) stay in the session.
• Statements run in main(sys: System), after the earlier statements of the session. The earlier statements run again with their output hidden, so their bindings stay visible.
• The REPL prints the value of an expression.
• An entry continues on the next line while its brackets and strings are open.
• Commands: :show (the program of the session), :reset and :quit.

Catalogs of messages

giz i18n DIR [--add TAG]... [--check] brings the catalogs locales/TAG.json of the .ghtml files under DIR up to date: each new message is added with an empty translation, a key no file shows is removed, and each catalog prints how much of it is translated. --check writes nothing and fails on a catalog that is behind or a translation that does not fit (MULTILINGUAL-ARCHITECTURE).

Formatter

giz fmt [--check | --bytes] FILE... lays out each line by its bracket depth: 4 spaces for each level, and one more level for a continued statement. It puts spaces between tokens by rule:

• after , ; :;
• around binary operators and =>;
• before {;
• between keywords and (.

It keeps the lines of the author, the comments, the inside of braces, and the spacing of < and >.

Before the layout, fmt rewrites old spellings to the modern ones that the checker proves the same (Option.some(x) to x, x.unwrap() to x!, join(v, s) to v.join(s), string + chains to template strings, and unused imports removed; COMPILER). --check does not rewrite. It changes only the files it is given; the files of one program share each round's one check. --bytes makes only one rewrite: a string's s.length, s[i], s.slice(a, b), s.indexOf(x) and s.lastIndexOf(x) become s.byteLength, s.byteAt(i), s.byteSlice(a, b), s.byteIndexOf(x) and s.byteLastIndexOf(x), which always count UTF-8 bytes; it lays out nothing. A land needs no fmt: wt.giz land lays out the changed files that must stay laid out (giz ci layout); fmt is the owner's sweep of its paths.

The formatter writes a file only when the lexer of the compiler (--tokens) finds the same tokens before and after. Thus it can format files that do not check yet, and it cannot change the meaning of a file. --check lists the files that are not formatted and exits with 1.

C and TypeScript bindings

C headers

giz bind HEADER LIBRARY NAME... prints a declare line for each named C function of HEADER. It reads the JSON AST of clang through the shared owner tools/bind/bindings.giz. It resolves typedefs. It maps types as the table in SYSTEM shows:

• A function that it cannot declare (variadic, a struct or function pointer by value, long double) becomes a comment that gives the reason.
• clang shows an array parameter as a pointer. Thus a line with mut scalars ends with a comment that names them. Make sure that C writes one value there.
• The output is source that you must review and paste. Its effects are the default {io, fail}. The manifest must still grant the library.

Reader selection:

• The repository reader uses GIZLANG_CLANG if it is set. If not, it uses clang.
• An installed binding uses the admitted release backend, with the same target, relocated sysroot and Clang resource directory as native compilation. It ignores reader overrides from the environment.
• A reader must exit successfully before its JSON is admitted. Valid JSON from a reader that failed is still an error.
• The native entry tools/bind/native.giz imports the shared binding owner. Its import closure does not include the repository TypeScript adapter.

Tests:

• tests/tools/bind/run.giz binds zlib from its header, and checks it against the crc32 of gizlang:compress and a round trip.
• tests/tools/bind/native/run.giz specifies the selection of the native reader, the refusal of .d.ts, and source-disclosure admission.

TypeScript declarations

The repository adapter tools/bind/main.giz supports a .d.ts HEADER. It runs the checker of TypeScript on the VM in its own process (tools/bind/typescript.js, through gizvm). The typescript package is found from the folder of the file or from NODE_PATH. LIBRARY is the JavaScript module (node:path, npm:slugify).

The output is a Gizlang module: for each name, a function that takes sys: System first, loads LIBRARY in the JavaScript realm and calls the export (SYSTEM). Each function throws string.

• Of the overloads of a function, the adapter binds the first one whose parameters and result it can map: string, number (f64), boolean (bool), void, string[] (Vec<string>) and number[] (Vec<f64>). Any other type is a gizvm Value. A Promise result is awaited.
• Optional and rest parameters are left out, with a comment.
• A bigint type makes the function a comment that gives the reason.
• tests/tools/bind/run.giz binds node:path from @types/node and calls it.

The native entry, which keeps its source private, returns E_BIND_TYPESCRIPT_UNAVAILABLE for .d.ts. It contains no TypeScript checker. Native TypeScript binding is not available.

JSON codecs

giz codec FILE TYPE... prints JSON functions for the named record types of FILE, and for the records that they contain (tools/codec/main.giz).

• For each named type: encodeT(value): string and decodeT(text): Result<T, JsonError>.
• For all types: writeT and readT, on the writer and pull reader of gizlang:json.

Fields can be:

• integers (i8 to i64, u8 to u32), f64, bool and string;
• records of the file;
• Vec or Option of these.

Decode rules:

• Integers are read exactly (also above 2^53) and checked against their width.
• A missing field, a value of a different kind, and trailing text are decode errors, with their offset.
• An Option field can be absent or null.
• Unknown keys are skipped.

Decoding builds each record with a literal that names every field. Thus, when a field is added or renamed, the old functions do not compile. Run the command again to replace them (tests/tools/codec/run.giz).

Language server

giz lsp runs the Gizlang language server (tools/lsp/main.giz) over stdin and stdout.

• The source CLI builds the server and keeps it in the cache. An installed release uses its admitted prebuilt helper.
• The protocol output contains only JSON-RPC framed with Content-Length.
• The server binary also accepts the absolute path of the compiler binary as its first argument.
• System.exit(code) ends the server without a print of a main result. It keeps the protocol framing and the exit status, and runs the cleanup of files and temporary directories.

Supported protocol

The server implements the LSP 3.17 protocol:

• initialize, shutdown and exit;
• full document sync;
• diagnostics on open and change;
• document symbols;
• hover, with declaration signatures and inferred effects;
• definition lookup and references;
• completion: after value., the fields and methods of its type (COMPILER), also while the edit does not check; elsewhere, the declarations of the last outline and the keywords.
• signature help: the parameters of the called function and the argument at the cursor, for a method from the compiler's completion of its receiver, for a function from the last outline.
• formatting: the layout of giz fmt, as one edit of the whole buffer, and none when the layout would change a token;
• rename of a declaration and its uses, by the compiler's rename transaction on a snapshot of the package (COMPILER). The program must check after it, or the rename is refused. Each changed file is one edit of the whole file. prepareRename gives the name at the cursor.
• semantic tokens: each name the outline binds in the file, typed by the declaration it names (type, enum for a union, interface, function, method, variable for a constant), with the modifier declaration at the declaration. Locals have none; the grammar colors them;
• folding ranges of brackets and block comments over more than one line;
• workspace symbols: the declarations of the programs of the open documents whose name contains the query, in any case.
• code actions: the quick fix of each error whose diagnostic has a fix (COMPILER), such as let for a const that is assigned;
• inlay hints: the type of each local whose type is not written, after its name (COMPILER). A local of a generic function gets none, because its type differs by instance.

Behavior:

• Positions use UTF-16, also for astral text.
• When a document closes, its diagnostics are cleared. Older document versions are ignored.
• A file of a package is checked by one resident compiler of that package (gizlang --serve). Each change writes the package snapshot again and sends edit FILE, so a changed function body is checked alone. A single file is checked by a child process. Both use isolated temporary copies of the editor buffers and never change a source file.
• The outline is made only for a symbol, hover, definition, reference or completion request, once for each valid buffer.
• Headers have a limit of 8 KiB. Bodies have a limit of 16 MiB.
• Incremental sync and range formatting are not advertised.

Editor buffers and the checked closure

• Full buffers become immutable overlays through the source snapshot owner of the CLI.
• Relative imports keep their original layout.
• A source below a package manifest uses its declared source closure. Root source pins can be stale in a working project. Restored dependencies must first agree with their original pins.
• Manifest, import and native grants come from disk, never from editor buffers. Preview hashes exist only in the temporary snapshot. User sources and manifests are never written.
• The compiler outline gives checked file spans and resolved declaration bindings (the compiler internals). Definitions, hover and references use those bindings across the checked closure. An unresolved or unsupported binding returns no result.
• Open buffers invalidate the affected contexts. Their checked closure can share one outline.
• When a document closes, its overlay is discarded, and the remaining documents are read from disk again.
• A maximum of 128 documents can be open. Their combined buffers must fit the source capacity of the compiler.
• Document versions are signed LSP integers. Stale changes are ignored.
• Only local .giz file URIs are admitted, with percent decoding and rejection of canonical-path collisions.
• Checks stay synchronous. These operations are read-only, so they need no edit transaction.

VS Code

The extension is in tools/vscode of the repository, and each toolchain release carries it as share/gizlang/vscode/gizlang.vsix; giz vscode installs that file. It starts giz lsp and adds the Test Explorer, debugging (Debugger), the giz task type and the Run, Debug, Check and Test File commands. To build it from a checkout:

cd tools/vscode
npm install
npm run package
code --install-extension gizlang-0.3.0.vsix

Set gizlang.path to the giz command when it is not on PATH. In a Gizlang checkout, set it to /absolute/path/to/gizlang/bin/giz, and set GIZLANG_BUILD in the environment of VS Code.

Neovim

For Neovim 0.11+, put this in init.lua. You do not need the lspconfig plugin.

vim.filetype.add({ extension = { giz = 'gizlang' } })
vim.lsp.config('gizlang', {
  cmd = { '/absolute/path/to/gizlang/bin/giz', 'lsp' },
  cmd_env = { GIZLANG_BUILD = '/absolute/build/directory' },
  filetypes = { 'gizlang' },
  root_markers = { 'package.json', '.git' },
})
vim.lsp.enable('gizlang')

Test

Build the server before you run bin/giz test tools/lsp. The test starts that binary and drives a complete framed session through pipes. The test and the launcher inherit GIZLANG_BUILD.

Debugger

giz build --debug FILE builds the program at -O0 with DWARF for its Gizlang sources: line tables, a subprogram for each function, and the arguments and locals with their types (tools/cli/debuginfo.giz). gdb or lldb then stops at a .giz line (break main.giz:12) or a function (break twice), steps by source line, bt shows Gizlang names and lines, and info locals shows numbers, bool, records, and string, bytes and Vec as {data, length}.

The VS Code extension (tools/vscode/src/debug.ts) builds the file with giz build --debug and runs it under gdb -i=dap (gdb 14 or later) or lldb-dap. gdb 15 starts the program when it receives launch, before VS Code sends the breakpoints, so the extension holds launch until configurationDone. tests/tools/debug checks the metadata and the binary's DWARF; the test hosts have no debugger. tests/tools/vscode/manifest.test.giz checks the debugger contribution.

Private native release construction

The source-checkout entry has a private construction adapter. Its source and specifications are implemented.

Construct a candidate

Run the adapter through the script command, with the normal source-launcher environment and a private request file:

GIZLANG_BUILD=/absolute/private/build bin/giz tools/distribution/build.giz /absolute/private/request.json

The request uses the release metadata object of tools/distribution/release.giz. There is no other release schema.

{
  "operation": "construct",
  "source": "/absolute/source/gizlang",
  "output": "/absolute/private/candidate-payload",
  "metadataOutput": "/absolute/private/candidate.unsigned.json",
  "release": "INSERT_THE_EXISTING_RELEASE_METADATA_OBJECT_HERE",
  "inputs": {
    "libexec/llvm/clang": "/absolute/reviewed/clang"
  },
  "probes": [
    {"label": "reviewed private implementation", "fragment": "EXACT_REVIEWED_PRIVATE_SOURCE_FRAGMENT"}
  ]
}

This excerpt shows only the placement of the fields. To make a real request:

1. Replace the release placeholder with the actual metadata object.
2. Put every prebuilt backend, runtime and sysroot input from its files in inputs. Each key is an exact payload path. Each value is an explicitly reviewed absolute source file.
3. Get the sizes and hashes from the release artifact audit. Never invent them.
4. Add reviewed probes of core implementation and paths. The Builder also keeps its mandatory probe of the known private TypeScript helper.

Rules:

• release.files contain only pinned prebuilt inputs.
• Generated payload paths must not be in the input mapping.
• Every source must agree with its File size and SHA-256 before construction accepts it.
• The Builder owns the target layout, the generated SDK, compiler and tool artifacts, the source disclosure audit, compiler semantics and linking.
• The request is a regular UTF-8 file, limited by Release.MOST_METADATA, and parsed with SDK Json.parseUnique. Input mapping, target and File admission stay the contracts of the Builder and Release owners.

Environment and output:

• Common.environment(sys, true) stays the one owner of the root, build, cache, compiler and CLI identities.
• Construction requires distribution.none, an explicit source root that agrees, absolute tool, cache and build paths, and existing private output parents.
• The output and the metadata must not exist, and must be physically outside the source checkout. The metadata also stays outside the payload.
• The metadata file uses the atomic publication of the SDK that does not replace. Thus existing or concurrent outputs cannot be overwritten.
• A failed build can leave its private candidate for inspection. Use new output paths. Do not use an incomplete candidate again.

The adapter gives machine JSON with signed:false and verified:false. It writes only the unsigned canonical Release.metadata to metadataOutput.

• The supplied evidence digest stays an input of the release identity. It is not a new proof that construction makes.
• Signing, an installer bound to the signed envelope, actual target execution and verified public publication have their own owners.

Construct an installer

After the signed envelope exists, the same private script accepts operation:"installer":

GIZLANG_BUILD=/absolute/private/build bin/giz tools/distribution/build.giz /absolute/private/installer-request.json

{
  "operation": "installer",
  "source": "/absolute/private/source-checkout",
  "output": "/absolute/private/gizlang-installer",
  "candidate": {
    "root": "/absolute/private/payload",
    "metadata": "/absolute/private/unsigned-release.json"
  },
  "envelope": "/absolute/private/signed-envelope.json",
  "binding": {
    "target": "x86_64-unknown-linux-gnu",
    "product": "toolchain",
    "origin": "https://gizstack.com/",
    "signedMetadataSha256": "<exact SHA-256 of signed-envelope bytes>",
    "trusted": [{ "id": "<reviewed release key id>", "publicKey": "<64 lowercase hex characters>" }],
    "httpsAuthority": "/absolute/reviewed/public-ca.pem"
  },
  "probes": [{ "label": "reviewed private implementation", "fragment": "<actual reviewed private implementation fragment>" }]
}

The placeholders are not admissible on purpose. Supply the actual signed envelope digest, the public release keys and the reviewed implementation probes.

• The candidate metadata is the unsigned Release.metadata of the earlier construction. It is not a second manifest.
• The adapter reads the candidate metadata (maximum 2 MiB), the signed envelope bytes (maximum 4 MiB) and the public CA bytes (maximum 65,536 bytes) through Dir.readBounded. Then it gives admission and construction to their owners.
• The output must not exist. It must be outside the source and the candidate, and separate from all request inputs.
• Both operations use Common.environment(sys, true) and the same request parser and path admission.

A successful installer output is machine JSON with operation:"installer", status:"constructed", signed:false, verified:false, and installer:{source:<absolute output>,file:<existing Release.File JSON>}.

• You can give its observed path, hash, size and executable pin directly to the private publication owner. Construction does not approve that publication.
• It never signs, makes a catalog, changes download readiness, or certifies native or OS acceptance.
• The flags refer to the installer artifact. The flags do not deny the authenticated release envelope that was an input.

Installer trust

Builder.constructInstaller(sys, env, binding, candidate, signedMetadata, output, reviewedProbes) authenticates the exact signed envelope through Installer.admitted. It compares the canonical Release.metadata identity with the constructed candidate.

• Binding.httpsAuthority contains the reviewed public PEM CA bytes. They are embedded with the public release keys and the immutable envelope digest.
• Installer.httpsAuthoritySha256 identifies those exact bytes. Release.trustedKeysSha256 identifies the canonical public key set.
• Arguments at run time cannot change either trust source.
• Metadata and every payload request use the same explicit HTTPS trust, independently of the certificate stores of the host.

Installer link

• Installer construction copies candidate bytes through Native.auditTo into private pinned snapshots.
• It then uses the native link owner with the candidate backend, sysroot, startup, SDK and runtime, and the exact static OpenSSL and sqlite archives from Common.sdkLibraryInputs.
• It does no host lookup of -lssl, -lcrypto, -lsqlite3 or a separate -luv.
• The final private installer output is published without replacement, through the atomic file owner.

Specifications

• tests/tools/distribution/build.test.giz specifies strict inputs, consistency of mapping and pins, explicit installer authority, key and path fields, private root separation, and refusal in an installed environment.
• tests/runtime/system/publication/run.giz specifies atomic file conflicts, cleanup after a failed stream, unchanged ordinary replacement, and tree publication.
• These specifications have not run yet.