# Data safety

A new database and a new file system must earn trust with evidence, not with claims. Each row below is a
guarantee and the public test that checks it. The tests are in this repository. CI runs every one of them after
each change to `main`; a change that breaks one does not stay.

## What the tests check

| Guarantee | The test | What it does |
| --- | --- | --- |
| A new release reads the data of every earlier release | `tests/fs/formats` | Each on-disk format declares every version its reader opens. `samples/` holds one store for each released set of versions, pinned by hash so it never changes. The suite opens a copy of each sample with today's code, reads its content, writes, and opens it again. It fails when a declared version has no sample, or a sample holds a version that no reader opens. |
| An acknowledged write survives a crash | `tests/db/replication` | The server is killed with SIGKILL at once after it replies. The restarted server must hold every acknowledged row, compared with an exact row oracle. |
| A crash during a large load loses no committed row | `tests/db/replication` (`chunked.giz`, `pending.giz`) | A crash in a chunked load leaves its staged chunks in the log; the next writable open drops them. Only the WAL fsync counts as an acknowledgement, never a pending scratch file. |
| A torn or short log is never taken as complete | `tests/db/replication` (`walstream.giz`) | Short reads and read errors never certify the tail of the log. Recovery reads a bounded amount of memory, whatever the size of the log. |
| A replica holds every commit that the primary confirmed | `tests/db/faults` (synchronous partition), `tests/db/replication` (`synchronous.giz`) | A relay between primary and standby is cut during four writers. Every successful commit is already on the standby. A timeout reports an unknown outcome with no `CommandComplete`. After the link heals, both nodes hold the same rows. |
| Manual promotion keeps the durable history | `tests/db/replication` (`synchronous.giz`) | The primary is killed with SIGKILL. The standby holds every acknowledged commit, also after its own restart. It is promoted, takes a new commit, and holds it after another restart. |
| A full disk refuses writes and loses nothing | `tests/db/faults` (disk full) | Four writers fill a disk quota. Each refused write gets an error and no `CommandComplete`. Every acknowledged transaction stays exact. When space returns, writes resume within 20 s, and all rows survive two restarts. |
| A crash at any byte of the log loses no whole commit | `tests/db/core.test.giz` (`engine crash`) | Commits run across a checkpoint, then the newest log is cut at every byte. Recovery restores exactly the commits whose records were wholly written. |
| A failed fsync stops the engine and loses no synced commit | `tests/db/core.test.giz` (`sync failure`), [database internals](../internals/database.md#durability-failures-stop-the-engine) | A sync fails at each sync of a workload in turn: log appends, checkpoint files and the directory. The engine stops and refuses commits. Reopened on the real disk, it holds every commit synced before the failure, and the commit in flight wholly or not at all. |
| A backup restores the same database | `tests/db/backup.giz` and the `pg pitr` test, run by `tests/db/pg.test.giz` | A cold backup and its restore keep the engine, its physical roots and the meaning of each snapshot. A point-in-time restore cuts only at a commit whose whole group digest is verified. A restore refuses bad metadata before it copies a file. |
| A storage bundle restores the same objects | `tests/fs/embedded` | A signed backup bundle of one cut restores into a new folder with the same objects and refs, a new key, and new writes. A folder that already holds a store, or a wrong source key, is refused and gets no file. |

## Run them yourself

The suites run from a checkout of this repository:

```sh
bin/giz test fs/formats
bin/giz test db/replication
bin/giz test db/faults
bin/giz test tests/db/core.test.giz
bin/giz test tests/db/pg.test.giz
bin/giz test fs/embedded
```

## What the tests do not check yet

- A disk that loses or reorders writes below the file system. The sync-failure test fails syncs in the storage
  layer of the engine, and the disk-full test uses a quota of the storage owner; neither is a fault of the kernel.

Data safety

A new database and a new file system must earn trust with evidence, not with claims. Each row below is a guarantee and the public test that checks it. The tests are in this repository. CI runs every one of them after each change to main; a change that breaks one does not stay.

What the tests check

Guarantee	The test	What it does
A new release reads the data of every earlier release	`tests/fs/formats`	Each on-disk format declares every version its reader opens. `samples/` holds one store for each released set of versions, pinned by hash so it never changes. The suite opens a copy of each sample with today's code, reads its content, writes, and opens it again. It fails when a declared version has no sample, or a sample holds a version that no reader opens.
An acknowledged write survives a crash	`tests/db/replication`	The server is killed with SIGKILL at once after it replies. The restarted server must hold every acknowledged row, compared with an exact row oracle.
A crash during a large load loses no committed row	`tests/db/replication` (`chunked.giz`, `pending.giz`)	A crash in a chunked load leaves its staged chunks in the log; the next writable open drops them. Only the WAL fsync counts as an acknowledgement, never a pending scratch file.
A torn or short log is never taken as complete	`tests/db/replication` (`walstream.giz`)	Short reads and read errors never certify the tail of the log. Recovery reads a bounded amount of memory, whatever the size of the log.
A replica holds every commit that the primary confirmed	`tests/db/faults` (synchronous partition), `tests/db/replication` (`synchronous.giz`)	A relay between primary and standby is cut during four writers. Every successful commit is already on the standby. A timeout reports an unknown outcome with no `CommandComplete`. After the link heals, both nodes hold the same rows.
Manual promotion keeps the durable history	`tests/db/replication` (`synchronous.giz`)	The primary is killed with SIGKILL. The standby holds every acknowledged commit, also after its own restart. It is promoted, takes a new commit, and holds it after another restart.
A full disk refuses writes and loses nothing	`tests/db/faults` (disk full)	Four writers fill a disk quota. Each refused write gets an error and no `CommandComplete`. Every acknowledged transaction stays exact. When space returns, writes resume within 20 s, and all rows survive two restarts.
A crash at any byte of the log loses no whole commit	`tests/db/core.test.giz` (`engine crash`)	Commits run across a checkpoint, then the newest log is cut at every byte. Recovery restores exactly the commits whose records were wholly written.
A failed fsync stops the engine and loses no synced commit	`tests/db/core.test.giz` (`sync failure`), database internals	A sync fails at each sync of a workload in turn: log appends, checkpoint files and the directory. The engine stops and refuses commits. Reopened on the real disk, it holds every commit synced before the failure, and the commit in flight wholly or not at all.
A backup restores the same database	`tests/db/backup.giz` and the `pg pitr` test, run by `tests/db/pg.test.giz`	A cold backup and its restore keep the engine, its physical roots and the meaning of each snapshot. A point-in-time restore cuts only at a commit whose whole group digest is verified. A restore refuses bad metadata before it copies a file.
A storage bundle restores the same objects	`tests/fs/embedded`	A signed backup bundle of one cut restores into a new folder with the same objects and refs, a new key, and new writes. A folder that already holds a store, or a wrong source key, is refused and gets no file.

Run them yourself

The suites run from a checkout of this repository:

bin/giz test fs/formats
bin/giz test db/replication
bin/giz test db/faults
bin/giz test tests/db/core.test.giz
bin/giz test tests/db/pg.test.giz
bin/giz test fs/embedded

What the tests do not check yet

A disk that loses or reorders writes below the file system. The sync-failure test fails syncs in the storage layer of the engine, and the disk-full test uses a quota of the storage owner; neither is a fault of the kernel.