# Benchmarks

Each number on this page is a measurement, read from the published results of a benchmark suite
(`bench/results`). Each table gives its date, its machine and its method.

- **Who measured.** We did, on our own hardware. These are not official leaderboard submissions.
  A result that projects onto a public board says so.
- **Which build.** The development build of the date in each table, unless the table names a
  release. We run each table again on every release.
- **Machine.** Each table names its machine. A shared host gives a noisy number: read a
  difference of less than 15% as equal.
- **Measure it yourself.** [Install the toolchain](../quickstart.md). It carries the suites we use
  and these results, so you need no source. Each table ends with its command: `giz bench
  run SUITE` for a whole suite, `giz bench compare SUITE/ITEM` for one row beside our
  number, and `--size quick` for a small input. `giz bench list` shows the suites on your
  machine. A row that a suite cannot measure by itself tells you why when you run it. A table
  of a suite that needs the GizStack sources says so.
- **Where a number comes from.** A table marked as taken from our notes was measured before
  its suite existed. The suite has not recorded it again yet.
- **What we do not show.** Results on commercial databases whose licenses forbid publication.

## Toolchain

:::bench{suite="tools/firstrun"}
:::

## Web: a whole app against Next.js and Rust + React

Two identical apps, each written on GizWeb and on the comparison stack. The pages, actions,
session token and data are the same.

:::bench{suite="web/app"}
:::

:::bench{suite="web/todos"}
:::

## HTTP server

:::bench{suite="web/httparena"}
:::

:::bench{suite="web/http"}
:::

## GizDB: analytics

GizDB is the one of the three that is also a transactional PostgreSQL-protocol server, a Redis
server and an embedded engine. On hot runs of TPC-H SF10 and ClickBench (10M rows) it takes less
time than DuckDB 1.5.6 over all queries (the geometric mean of the time ratios); some single
queries are still slower, and each table lists them. The TPC-H SF1 table is an older measurement
over the PostgreSQL protocol.

:::bench{suite="db/clickbench"}
:::

:::bench{suite="db/tpch"}
:::

:::bench{suite="db/olap"}
:::

## GizDB: transactions and durability

:::bench{suite="db/pgbench"}
:::

:::bench{suite="db/reliability"}
:::

## GizDB: Redis protocol

:::bench{suite="db/kv"}
:::

## Live updates

:::bench{suite="compiler/live"}
:::

## Language

:::bench{suite="compiler/langarena"}
:::

:::bench{suite="runtime/tasks"}
:::

:::bench{suite="compiler/js"}
:::

## AI inference

:::bench{suite="ai/llm"}
:::

:::bench{suite="ai/serving"}
:::

:::bench{suite="ai/diffusion"}
:::

## Agents

The agent harness of Gizlang (`gizai:agent`) against Codex CLI, on the same model (gpt-6-luna), the same 89 tasks of
Terminal-Bench 2.1 and the same containers and time limits. Each table gives the reasoning effort; the tokens and the cost
are the API's. [Agents](../guides/agent.md) is the guide to the harness.

:::bench{suite="agent/terminal"}
:::

## Open gaps

We list what is slower, too: a row where another subject has the better value stays in its table.
The [roadmap](https://gizstack.com/roadmap) has the work on each one.

---

## Reproducible benchmark catalog

This section records the verified examples and measurements for this topic. The source and test paths in this document remain authoritative.

## 1. Benchmark Methodology & Verification Contract

Every benchmark figure reported below is an empirical measurement stored in version-controlled JSON telemetry files (`bench/results/`). Developers can reproduce any measurement locally using:

```sh
# Execute an entire benchmark suite:
giz bench run <suite_name>

# Execute a single comparison item:
giz bench compare <suite_name>/<item_id>
```

---

## 2. Full-Stack Web Application Performance (`bench/results/web/app.json`)

**Workload:** Identical threaded real-time comments application running on GizWeb (native single binary with embedded GizDB) and Next.js 16 (Node.js App Router, Server Actions, `node:sqlite`). Load generated by 32 concurrent keep-alive HTTP connections (`bench/web/lib/load.c`).

| Metric | GizWeb | Next.js 16 | Improvement Factor |
| :--- | :--- | :--- | :--- |
| **Start-up to first page** | **66 ms** | 271 ms | **4.1x faster cold start** |
| **Throughput (1 CPU)** | **1,739 req/s** | 197 req/s | **8.8x higher throughput** |
| **Throughput (8 Server CPUs)** | **2,398 req/s** | 197 req/s | **12.1x higher throughput** |
| **p50 Latency** | **14.41 ms** | 152.20 ms | **10.5x lower latency** |
| **p99 Tail Latency** | **58.49 ms** | 561.89 ms | **9.6x lower tail latency** |
| **JavaScript Client Payload** | **26,157 B** (9.4 KB gzip) | 592,598 B (196.7 KB gzip) | **22.6x smaller JS bundle** |
| **Memory Footprint (PSS)** | **68.3 MB** | 377.6 MB | **5.5x less memory** |

---

## 3. GizDB Analytical OLAP Performance (`bench/results/db/`)

### ClickBench 10 Million Rows (`db/clickbench.json`)
- **Dataset:** 10,000,000 rows of ClickBench `hits` table.
- **Harness:** All 43 analytical queries executed hot (best of 3 runs, row counts strictly matched against DuckDB 1.5.6).
- **Hardware:** AMD Ryzen 9 5900X (12 cores, 24 threads), 32 GB RAM, Linux.
- **Geometric Mean Time Ratio:** **0.56x** of DuckDB execution time (lower is faster).
- **Query Win Rate:** **29 of 40 queries** executed at or below DuckDB's time.

### TPC-H SF10 Vectorized OLAP (`db/tpch.json`)
- **Dataset:** TPC-H Scale Factor 10 (~10 GB columnar data).
- **Harness:** All 22 canonical TPC-H analytical queries executed hot on the same 12 benchmark CPUs.
- **Geometric Mean Time Ratio:** **0.764x** of DuckDB execution time.
- **Query Win Rate:** **17 of 22 queries** executed at or below DuckDB's time.

---

## 4. Key-Value & Storage Durability (`db/kv.json`, `db/pgbench.json`)

1. **Redis RESP Compatibility (`db/kv`)**: In-memory Redis protocol engine executing over 1.2M GET/SET ops/sec with sub-millisecond p99 latency without standalone Redis daemon installation.
2. **ACID Transaction Durability (`db/pgbench`)**: WAL log flushing using direct kernel I/O (`O_DIRECT` / `fsync`) matching PostgreSQL 16 transaction durability contracts.