GizStack
View .mdOpen in ChatGPTOpen in Claude

gizmodel/store

the database a web app keeps its records in, by its location. Every database is a gizlang/sql Database: GizDB, the default, is a folder whose engine a gizdb process or serve of gizdb/local serves, and every process, the serving one too, reaches it as a PostgreSQL server at the folder's unix socket with its password. This module is a client only, so a program that opens a store links no engine (the engine was 5 of the 12 million IR lines of a client platform, 10/23). DATABASE_URL selects another SQL database: PostgreSQL (a server; each worker process holds a pool) or a SQLite file (embedded; every worker process opens it). Apps, the feed and the model write one SQL for all (ON CONFLICT, RETURNING, $N, TRUE and FALSE); what differs, DDL types and locks, is chosen here or by the SQL Dialect (model/sql/types/dialect.giz). Here are the folder's layout, the move of a SQLite file into a database with its proof, and the typed reads and writes of a Model over SQL text. Typed records (model/data.giz) reach a served engine through db/access.giz.

gizlang
import * as Store from "gizmodel/store";

Types

Place

gizlang
type Place = { path: string; secret: string; }

A GizDB folder: its absolute path and its password.

  • path: string
  • secret: string

Moved

gizlang
type Moved = { same: bool; lines: Vec<string>; }

What a move found: same when every table is equal, and a line for each table.

  • same: bool
  • lines: Vec<string>

Source

gizlang
type Source = { columns: string[]; cells: Vec<Option<string>>; }

A source row of a copy: its cells by column name. A SQL NULL is undefined.

  • columns: Vec<string>
  • cells: Vec<_gizlang_sdk_core.Option<string>>

Plan

gizlang
type Plan<T> = { name: string; table: string; key: string; changed: string; read: (row: Source) => T[]; where?: Tree.Expression<bool>; }

One table's copy into the stored type T: the rows of table in the order of key (unique, never null), each read by read into the records it becomes (none skips the row). changed is a column that grows on each write of a row (such as updatedAt), so follow copies the rows written after the bulk copy; "" when the table has none. where copies only the rows it admits (one account's for a rehearsal), and the copy's proof keeps it.

Opaque: create and use it through this module's functions.

Progress

gizlang
type Progress = { plan: string; after: string; read: i64; written: i64; checksum: string; changed: string; changedKey: string; done: bool; filter: string; }

Where a plan stands, kept in the target's gizdb_copies: the last key copied, the source rows read, the records written, the checksum of the source rows read (gizmodel/copy), the changed value and key that follow has reached, and whether the bulk copy has ended.

  • plan: string
  • after: string
  • read: i64
  • written: i64
  • checksum: string
  • changed: string
  • changedKey: string
  • done: bool
  • filter: string

Functions

open

gizlang
function open(location: string): Sql.Database throws SqlError

The database at location: a postgres:// address or a SQLite file, as gizlang/sql opens them, else a GizDB folder, which is the PostgreSQL server at its unix socket (gizdbLocation). A folder must be served already, by gizdb or by gizdb/local serve in some process: this opens a connection only. STORE_QUOTA, when set, is the most bytes a GizDB or SQLite store holds: past it a write fails and reads go on.

Effects: fail, read, write, console, io, alloc.

dialect

gizlang
function dialect(db: Sql.Database): Dialect

The SQL dialect of DB.

Effects: fail, alloc.

isFolder

gizlang
function isFolder(location: string): bool

Whether location names a GizDB folder: not a postgres://, mysql:// or odbc: address and not a SQLite file.

Effects: fail.

gizdbLocation

gizlang
function gizdbLocation(folder: string, password: string): string

The postgres:// location of the GizDB server of folder (absolute) with password.

Effects: fail, read, console, io, alloc.

gizdbSocket

gizlang
function gizdbSocket(folder: string): string

The path to listen and connect at for the socket folder GIZDB_RUN of folder (absolute).

Effects: fail, read, console, io, alloc.

socketFolder

gizlang
function socketFolder(folder: string): string

The path of folder (absolute) for its unix-domain sockets (GIZDB_RUN, a hosted app's platform socket). The socket files are always in the folder (a hosted app reaches them there), but a socket path may be at most 107 bytes. For a deep folder this is /tmp/giz-HASH (the folder's SHA-256), a link to it that this makes.

Effects: fail, read, console, io, alloc.

placeOf

gizlang
function placeOf(location: string): Place throws SqlError

The GizDB folder location, made when missing, with its password, made on its first open: gizdb without one would accept every client.

Effects: fail, read, io, alloc.

socketFile

gizlang
function socketFile(socket: string, port: i64): string

<socket>/.s.PGSQL.<port>: the unix-domain socket of a server on port, where libpq looks.

Effects: fail, alloc.

serving

gizlang
function serving(folder: string): bool

Whether a socket connect to the GizDB folder folder succeeds now: some process, any process, accepts it. This is a client's probe, not the answer to whether a server of one program runs; a program's own servers are asked there.

Effects: fail, read, console, io, alloc.

movedIn

gizlang
function movedIn(location: string, folder: string): bool throws SqlError

Moves the SQLite file folder.db into the GizDB database at LOCATION, whose folder is folder, on the folder's first start, when there is one (move, which proves the copy; the file stays, as the way back), so an app that kept its records in data.db finds them in data. GIZDB_MOVED records that the start did it, or found none. Only a move opens a connection: its sign-in was half of the start of a folder with nothing to move (35 of 60 ms).

Effects: fail, read, write, console, io, alloc.

moveSchemas

gizlang
function moveSchemas(old: string, database: Sql.Database): bool throws SqlError

The schemas of the SQLite file OLD moved into the same schemas of DATABASE. A SQLite schema is a file beside it, <old>-<schema>.db, which the app attached (data.giz attachedIn): an app's people are in <old>-auth.db.

Effects: fail, read, write, console, io, alloc.

moveInto

gizlang
function moveInto(src: Sql.Database, dst: Sql.Database, schema: string): Moved throws SqlError

As move, the SQLite database SRC into schema SCHEMA of DST, made when it is not there: one transaction whose search path is SCHEMA, so the tables, rows and indexes of SRC find their names there.

Effects: fail, read, write, console, io, alloc.

move

gizlang
function move(src: Sql.Database, dst: Sql.Database, verify: bool): Moved throws SqlError

Move the SQLite database src into dst, which has none of its tables, and prove the copy; with verify, only compare an earlier copy.

Effects: fail, read, write, console, io, alloc.

progress

gizlang
function progress<T>(dst: Sql.Database, plan: Plan<T>): Progress throws SqlError

Where PLAN stands in DST before a run, so a caller can report what the run itself copied and changed.

copy

gizlang
function copy<T>(src: Sql.Database, dst: Sql.Database, plan: Plan<T>, rows: i64): Progress throws SqlError

Copies the next rows source rows of plan from src into dst and answers where the plan stands; a plan whose bulk copy has ended copies nothing. A run that stops resumes after the last key it wrote: each batch and its progress are one transaction, and a record that is written again replaces itself.

follow

gizlang
function follow<T>(src: Sql.Database, dst: Sql.Database, plan: Plan<T>, rows: i64): Progress throws SqlError

Copies the next rows source rows of plan written after the point it has reached (its changed column, then its key), and answers where it stands; nothing for a plan without a changed column. The point is compared as the source reads its own text of the column (PostgreSQL types an untyped parameter by the column). A row deleted at the source is not seen here: the cutover's count shows it.

recount

gizlang
function recount<T>(src: Sql.Database, dst: Sql.Database, plan: Plan<T>, rows: i64): bool throws SqlError

Reads the source rows of plan up to the last key copied, rows at a time, and answers whether their count and checksum are still the ones its bulk copy recorded (the proof of a copy before follow changes rows). A PostgreSQL source sums its rows itself (sourceSlice): a table of 33M rows came to the client to be summed.

migrate

gizlang
function migrate(db: Sql.Database, shape: Model.Shape): Rows throws SqlError

Bring the table up to the shape, additively: create it, add each missing column (a required one with an empty, zero or false default for the rows already there) and the indexes. A column the shape no longer has keeps its data: dropping or converting is an explicit, separate step. One transaction under the table's lock, so worker processes starting together migrate one after another (PostgreSQL's CREATE TABLE IF NOT EXISTS is not safe to race). The result's affected counts the statements that changed the schema.

Effects: fail, read, write, console, io, alloc.

additions

gizlang
function additions(shape: Model.Shape, dialect: Dialect, existing: Vec<string>): Vec<string> throws SqlError

The statements that bring a table with the columns EXISTING to SHAPE: @was renames, then the missing columns, a NOT NULL one with its type's zero as the default, so old rows read it so. The engine's records migrate by it too (model/data.giz engineInstall).

Effects: fail, read, write, console, io, alloc.

migrateLocked

gizlang
function migrateLocked(tx: Sql.Database, shape: Model.Shape): Rows throws SqlError

Within the caller's locked transaction: @was, additive columns, then indexes.

Effects: fail, read, write, console, io, alloc.

fetch

gizlang
function fetch(db: Sql.Database, tree: Tree.Select): Rows throws SqlError

The rows of the query TREE (a syntax tree, as Model and Query build it: an aggregate, a join), its values bound as parameters (render.giz bound): the one way a typed read of a Store reaches SQL text.

Effects: fail, read, write, console, io, alloc.

apply

gizlang
function apply(db: Sql.Database, command: Tree.Statement, params: Vec<Value>): Rows throws SqlError

COMMAND (a syntax tree, as Model and Query build it) with its parameters $1.. PARAMS, run on the database with its values bound (render.giz boundMutation).

Effects: fail, read, write, console, io, alloc.

ping

gizlang
function ping(db: Sql.Database): bool throws SqlError

Whether the database answers a query.

Effects: fail, read, write, console, io, alloc.

rowCount

gizlang
function rowCount(db: Sql.Database, name: string): i64 throws SqlError

The rows of the table NAME (Sql.Database.tables names them).

Effects: fail, read, write, console, io, alloc.

sample

gizlang
function sample(db: Sql.Database, name: string, rows: i64): Rows throws SqlError

The first ROWS rows of the table NAME with its column names, in the order the table keeps them.

Effects: fail, read, write, console, io, alloc.

Methods

Source.get

gizlang
function Source.get(self: Source, name: string): string | undefined

The cell of column name: undefined for NULL or for a column the row does not have.

Effects: fail.

Constants

GIZDB_FILES

gizlang
const GIZDB_FILES = ".gizdb"

The engine's files.

GIZDB_RUN

gizlang
const GIZDB_RUN = ".gizdb.run"

The folder of its one way in, the unix socket .s.PGSQL.5432 (libpq's name); no TCP port is open.

GIZDB_PASSWORD

gizlang
const GIZDB_PASSWORD = ".gizdb.pw"

Its password: gizdb without one would accept every client.

GIZDB_MOVED

gizlang
const GIZDB_MOVED = ".gizdb.moved"

The record that its first start moved the folder's SQLite file in, or found none.