Benchmarks

The released toolchain on a clean user account. giz init hello, then in hello: the first giz main.giz (it compiles, links and runs, with an empty cache), the second (from the cache) and giz test main.test.giz.

A clean user account.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

Measure it again with the toolchain: giz bench run tools/firstrun, or one row on a small input: giz bench compare tools/firstrun/first --size quick.

The same threaded realtime comments app on GizWeb and on Next.js 16 (App Router, server actions, node:sqlite): the same pages, actions and session token, and 60 seeded comments. Each server is one process on 8 server CPUs; the load (bench/web/lib/load.c, 32 keep-alive connections) is on 8 other CPUs. GizWeb runs on one server CPU, then on a thread for each server CPU, with GizDB in the process. Throughput is the best of 3 runs.

GizDB in the process (`Data.database`).

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

This suite needs the GizStack sources, so the release does not carry it yet.

The same todos app on GizWeb and in the examples/react-todos that nextrs measures itself with: the same three todos, page, detail page, filter, JSON API and live updates. Each server runs on one CPU, then on the 8 server CPUs (GizWeb with a thread for each CPU, the tokio of nextrs with a thread for each CPU). The load comes from 8 other CPUs, with 50 connections, on the two routes that the own benchmark of nextrs loads.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

This suite needs the GizStack sources, so the release does not carry it yet.

The scored profiles of HttpArena with the load tools and arguments of the board, scaled to 8 server CPUs and 8 load CPUs (pinned rates / 8), best of 3 runs (latency profiles: 2). actix, Bun and dusty ran at the same time on the same CPUs. score.giz projects the result onto the composite of the board through the references measured at the same time. A latency profile shows the CPU time of each request at the fixed rate.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

This suite needs the GizStack sources, so the release does not carry it yet.

Each server gets one worker for each server CPU, with the same files, upstream and certificate. The load runs on 8 other CPUs (load.c, or wrk for TLS) with 256 connections; the proxy upstream is an nginx with two workers on two more CPUs. The value is the median rate of 3 rounds, with the server order turned in each round.

2 workers on 2 CPUs, a quiet machine. 4 workers on 4 cores, a shared machine (load average 4 to 26); run A.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

This suite needs the GizStack sources, so the release does not carry it yet.

10 million rows of ClickBench's hits, loaded once into a GizDB folder and a DuckDB 1.5.6 file. bench/db/engine/hot.giz runs each query of bench/db/clickbench/queries.sql 4 times in one session of each engine on the same 12 CPUs: one warm run, then the best of 3 (psql's \timing for GizDB over the PG protocol, DuckDB's .timer), in ms. Row counts must match. The geometric mean is of GizDB's time over DuckDB's, over the queries DuckDB takes 1 ms or more for.

29 of 40 queries at or below DuckDB's time. Measured alone at 0486c83b under other loads on the host (DuckDB 300 ms, 244 at 523f9210 on a quieter host).

This suite needs the GizStack sources, so the release does not carry it yet.

CALL dbgen(sf=10) (60 million lineitem rows), loaded once into a GizDB folder and a DuckDB 1.5.6 file. bench/db/engine/hot.giz runs each query of bench/db/tpch/queries.sql 4 times in one session of each engine on the same 12 CPUs: one warm run, then the best of 3 (psql's \timing for GizDB over the PG protocol, DuckDB's .timer), in ms. Row counts must match. The geometric mean is of GizDB's time over DuckDB's, over the queries DuckDB takes 1 ms or more for.

17 of 22 queries at or below DuckDB's time. A second pass 30 minutes later: 0.757, 17 of 22.

This suite needs the GizStack sources, so the release does not carry it yet.

CALL dbgen(sf=1) (6 million lineitem rows), loaded into GizDB and PostgreSQL by psql's \copy and analyzed. Each query runs once, then 5 more times; the value is the median of those, in ms. GizDB and PostgreSQL are timed at the client over the PG protocol, DuckDB by its own timer. A query stops at 5 minutes. Every answer must equal DuckDB's.

Servers and DuckDB on 8 benchmark CPUs (6-9, 18-21), the runner on 4 others. DuckDB 1.5.6, PostgreSQL 16.15.

This suite needs the GizStack sources, so the release does not carry it yet.

f(id PK, c, a, d, s) with 10 million rows from generate_series (c 100k values, a 1000, d 365 days, s 100 texts) and dim(id PK, r) with 100k rows. Each query runs once, then 5 more times; the value is the median of those, in ms. Every answer must equal DuckDB's.

Both engines on 8 benchmark CPUs (6-9, 18-21), the runner on 4 others. DuckDB 1.5.6.

Measure it again with the toolchain: giz bench run db/olap, or one row on a small input: giz bench compare db/olap/Q1 --size quick.

pgbench -i -s 10 (1 million accounts), then the built-in TPC-B script with -M prepared for 8 s. GizDB runs with --threads 8; fsync is on in both, so every commit waits for its sync. The load step is timed alone: pgbench -i -s 10 -q (CREATE, COPY of 1 million accounts from the client, VACUUM, primary keys).

Servers on 8 benchmark CPUs (6-9, 18-21), pgbench on 4 others (10, 11, 22, 23). PostgreSQL 16.15.

Measure it again with the toolchain: giz bench run db/pgbench, or one row on a small input: giz bench compare db/pgbench/tpcb1 --size quick.

Each loop starts the server on the same folder, runs 4 connections that commit 3-row transactions (and in one loop of 7 a CHECKPOINT), kills the server with SIGKILL after 5 to 150 ms, restarts it and checks the loop's rows: an acknowledged transaction must be there whole, one in flight whole or absent, and nothing else. One loss fails the run.

1,000 kills; 4,000 transactions in flight at the kills, each whole or absent; 56,625 rows checked.

This suite needs the GizStack sources, so the release does not carry it yet.

One server on one CPU, redis-benchmark on other CPUs: 50 connections, 200,000 requests, 64-byte values, data on disk. Each mode gives the same guarantee on both servers: no persistence (GizDB still writes its log and never syncs it), a sync every second, or a sync before each answer. Server CPU counts the plain SETs.

1 shard on 1 CPU (6), 1 I/O thread each; Redis 7.4.1. Load from 8 other benchmark CPUs.

This suite needs the GizStack sources, so the release does not carry it yet.

run.giz writes a generated web app, starts giz live on it and edits a literal in a handler body 20 times. Each edit is timed from the save of the file (a rename, as an editor saves) to the line on the app's stdout that shows the response of the new code. The value is the p50.

Load average 9 to 23.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

This suite needs the GizStack sources, so the release does not carry it yet.

Five deployments of a changed title to a page that was already open, timed from Deploy to the change in the page. The text and focus in a form field stayed. Measured by hand.

Five deployments.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

This suite needs the GizStack sources, so the release does not carry it yet.

LangArena: 50 realistic programs, written safe and idiomatic in each language. Each test alternated with Rust on the same machine, then the formula of the board was applied against its runs of 2026-10-05.

Against the runs of the board of 2026-10-05.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

Measure it again with the toolchain: giz bench run compiler/langarena, or one row on a small input: giz bench compare compiler/langarena/score --size quick.

main.giz spawns N tasks that each pause ROUNDS times, then joins them; main.go does the same with goroutines on one OS thread (GOMAXPROCS(1)), as a Gizlang World runs its tasks. The switch time is 100 tasks that pause 10,000 times (1,000,000 switches). The memory is the peak RSS of 100,000 tasks, divided by the task count.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

Measure it again with the toolchain: giz bench run runtime/tasks, or one row on a small input: giz bench compare runtime/tasks/switch --size quick.

The three engines run in turn on the same pinned CPUs, with the order turned in each round. A process loads the packages fast-deep-equal, ms and cookie and exits; the value is the median wall time of 31 processes.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

Measure it again with the toolchain: giz bench run compiler/js, or one row on a small input: giz bench compare compiler/js/start --size quick.

Qwen3.8-27B, linear layers in 4 bits and the output layer in 6 bits, on one NVIDIA RTX 4070 Ti SUPER (16 GB). run.giz speed warms every kernel shape, then times a 1024-token prefill and 128 tokens of greedy decode for each batch size, with MTP drafts. The decode rate is the tokens of all sequences together.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

Measure it again with the toolchain: giz bench run ai/llm, or one row on a small input: giz bench compare ai/llm/decode1 --size quick.

Qwen3-4B-Instruct-2507 in BF16 (weights and keys and values) on one NVIDIA RTX 4070 Ti SUPER (16 GB), contexts of 2048 tokens, each server's prefix cache on. One client (bench/ai/llm/load.giz) holds 1, 4 or 16 chats at once with each server's OpenAI chat API, streamed and greedy: a chat starts with a prompt of 512 words and asks twice for more, 256 tokens an answer at most. The first token is the median of the first turns; tokens/s of an answer is the median after its first token. The servers write their answers' tokens with the same greedy rule, so their lengths differ a little.

Measure it again with the toolchain: giz bench run ai/serving, or one row on a small input: giz bench compare ai/serving/c1-rate --size quick.

FLUX.3 klein, 1024 x 1024, against ComfyUI on the same NVIDIA RTX 4080 SUPER with the same BF16 checkpoint. Gizlang's step runs on INT8 matrix cores; its image matches ComfyUI's by eye. The step cache (First Block Cache) is on at 0.3 by default, and 0 turns it off. INT4 products are an option, a little softer. ComfyUI's edit times include its PNG save.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

Measure it again with the toolchain: giz bench run ai/diffusion, or one row on a small input: giz bench compare ai/diffusion/step --size quick.

MiniMax H3, 832 x 480, 124 frames, 6 steps, its W4A8 checkpoint with the turbo LoRA, against WanGP on the same request, on one NVIDIA RTX 4070 Ti SUPER (16 GB). Our total is the sum of its measured parts (6 passes, the video and audio decoders), warm.

These numbers come from our measurement notes of that date. The suite has not recorded them again yet.

Measure it again with the toolchain: giz bench run ai/diffusion, or one row on a small input: giz bench compare ai/diffusion/pass --size quick.

Terminal-Bench 2.1, its 89 tasks with one trial each, on gpt-6-luna through the OpenAI Responses API at reasoning effort medium. Each harness works in the task's Docker container until it finishes or the task's time limit ends, then the task's tests score it 1 or 0. Gizlang runs gizai:agent with one bash tool and a short instruction; Codex CLI runs codex exec with the same effort. Input tokens include the cached ones. Codex counts its tokens from the turns that completed, so a trial that timed out adds none: its totals are a lower bound. A run whose results do not count the cached input tokens takes them from its logs. Cost: uncached input $0.10, cached input $0.01 and output $0.50 per million tokens. A value that cannot be measured from a run says why (README.md).

Measure it again with the toolchain: giz bench run agent/terminal, or one row on a small input: giz bench compare agent/terminal/medium-pass --size quick.

The same tasks, model, harnesses and counting as the medium table, at reasoning effort xhigh.

Measure it again with the toolchain: giz bench run agent/terminal, or one row on a small input: giz bench compare agent/terminal/xhigh-pass --size quick.