# gizdb:value Values, types and schemas, and their one binary encoding (used by the log and by segment files alike). ```gizlang import * as Value from "gizdb:value"; ``` ## Types ### `Type` ```gizlang type Type = | { kind: "int"; } | { kind: "float"; } | { kind: "text"; } | { kind: "blob"; } | { kind: "boolean"; } | { kind: "vector"; dim: i64; } | { kind: "decimal"; scale: i64; } | { kind: "temporal"; unit: i64; } ``` A column's type. `vector` has a fixed dimension; `text` may carry a full-text index; `decimal` holds exact decimals at `scale` digits after the point (DECIMAL(p, s)), or at each value's own scale when `scale` is -1 (NUMERIC without one); `temporal` holds dates and times of `unit` (DATE, TIMESTAMP, TIME, TIMESTAMPTZ) as integers, in an integer column as booleans are. - `int` - `float` - `text` - `blob` - `boolean` - `vector` - `decimal` - `temporal` ### `Value` ```gizlang type Value = | { kind: "null"; } | { kind: "int"; v: i64; } | { kind: "float"; v: f64; } | { kind: "text"; v: string; } | { kind: "blob"; v: bytes; } | { kind: "boolean"; v: bool; } | { kind: "vector"; v: Vec; } | { kind: "decimal"; d: Dec; } | { kind: "temporal"; v: i64; unit: i64; } ``` One value of a row or an expression. `decimal` is exact (decimal.giz); `temporal` is `v` of `unit`: days since 1970-01-01 (DATE), microseconds since its midnight (TIMESTAMP, and TIMESTAMPTZ in UTC) or since the day's start (TIME). - `null` - `int` - `float` - `text` - `blob` - `boolean` - `vector` - `decimal` - `temporal` ### `ColumnDef` ```gizlang type ColumnDef = { name: string; ty: Type; nullable: bool; declared: string; default: string; fill: Value; } ``` A column of a table. `ty` is how it is stored; `declared` is the type as the defining SQL spelled it (the SQL layer reads dates, decimals, JSON and blobs from it, and catalogs show it); `default` is the SQL of its default value ("" for none); `fill` is what rows written before the column was added read. - `name`: `string` - `ty`: `M55.Type` - `nullable`: `bool` - `declared`: `string` - `default`: `string` - `fill`: `M55.Value` ### `IndexDef` ```gizlang type IndexDef = { name: string; columns: Vec; unique: bool; method: string; options: string; } ``` An index: `btree` (lookups and uniqueness; the primary key is always one), `text` (an inverted index for BM25 on one text column) or `vector` (a nearest-neighbour graph on one vector column; `options` names the metric: l2, cosine or ip). A btree made by a UNIQUE constraint has `options` "constraint" (catalogs list it as one). - `name`: `string` - `columns`: `Vec` - `unique`: `bool` - `method`: `string` - `options`: `string` ### `Schema` ```gizlang type Schema = { name: string; columns: Vec; keys: Vec; indexes: Vec; } ``` A table's schema: its columns, the primary key's columns (none for a table without one) and its other indexes. - `name`: `string` - `columns`: `Vec` - `keys`: `Vec` - `indexes`: `Vec` ### `DecodeCredit` ```gizlang type DecodeCredit = { maxValues: i64; maxBytes: i64; values: i64; bytes: i64; refused: bool; } ``` Decode work credits: 64 bytes per Value slot, payload bytes, and schema vector slots. Inspection and materialization share the same parser; inspection never grows vectors. - `maxValues`: `i64` - `maxBytes`: `i64` - `values`: `i64` - `bytes`: `i64` - `refused`: `bool` ## Functions ### `fitsInt` ```gizlang function fitsInt(x: i64, bytes: i64): bool ``` Whether `x` fits in a signed integer of `bytes` bytes. Effects: fail. ### `storageType` ```gizlang function storageType(declared: string, affinity: bool, dateIsTimestamp: bool): Type ``` How a column of declared type `declared` stores its values; SQLite (affinity) has no decimals, its NUMERIC holds reals. Effects: fail. ### `temporalUnit` ```gizlang function temporalUnit(d: string, affinity: bool, dateIsTimestamp: bool): i64 ``` The temporal unit (value.giz DATE, TIMESTAMP, TIME, TIMESTAMPTZ) a declared type (upper case) stores, or -1: Oracle's DATE holds a time as well; SQLite keeps dates as text. Effects: fail. ### `timetzDeclared` ```gizlang function timetzDeclared(d: string): bool ``` Whether a declared type (upper case) is time with time zone, which keeps the text of its value (types.giz timetzParts). Effects: fail. ### `offsetText` ```gizlang function offsetText(east: i64): string ``` A zone offset of `east` seconds east of UTC as PostgreSQL prints it: +HH, +HH:MM or +HH:MM:SS. Effects: fail. ### `precisionWritten` ```gizlang function precisionWritten(declared: string): i64 ``` The precision p of DECIMAL(p,s) or NUMBER(p) as written; 0 when none is. Effects: fail. ### `scaleOf` ```gizlang function scaleOf(declared: string): i64 ``` The scale of DECIMAL(p,s) (0 when none is written); -1 for NUMERIC without arguments. Effects: fail. ### `isTz` ```gizlang function isTz(u: i64): bool ``` Whether a unit is a timestamptz of either kind. ### `typeUnit` ```gizlang function typeUnit(u: i64): i64 ``` The unit a column or an expression type has: every zone code is the type ZONED. ### `fixedZone` ```gizlang function fixedZone(east: i64): i64 ``` The unit of a value with the fixed offset `east` seconds east of UTC (whole minutes). Effects: fail. ### `fixedEast` ```gizlang function fixedEast(u: i64): Option ``` The offset in seconds east of UTC that a fixed zone's unit names, or none for a region. Effects: fail. ### `micros` ```gizlang function micros(v: i64, unit: i64): i64 ``` V in UNIT as microseconds since the epoch (a TIME since midnight), the one scale on which the units compare: DATE '2024-01-01' equals TIMESTAMP '2024-01-01 00:00:00'. Effects: fail. ### `formatDate` ```gizlang function formatDate(days: i64): string ``` The text of the day number DAYS: 'YYYY-MM-DD', with ' BC' before year 1, or 'infinity'. Effects: fail. ### `formatTime` ```gizlang function formatTime(us: i64): string ``` The text of a TIME of US microseconds: 'HH:MM:SS[.ffffff]'. Effects: fail. ### `clockText` ```gizlang function clockText(s: i64, frac: i64): string ``` 'HH:MM:SS[.ffffff]' of whole seconds and microseconds; the parts keep i64's least value in range. Effects: fail. ### `formatTimestamp` ```gizlang function formatTimestamp(us: i64): string ``` 'YYYY-MM-DD HH:MM:SS[.ffffff]' of microseconds since the epoch. Effects: fail. ### `stampText` ```gizlang function stampText(us: i64, zone: string): string ``` formatTimestamp with `zone` after the time, and BC last, as PostgreSQL prints them. Effects: fail. ### `temporalText` ```gizlang function temporalText(v: i64, unit: i64): string ``` A temporal value as PostgreSQL prints it (DateStyle ISO, TimeZone UTC). Effects: fail. ### `intBytes` ```gizlang function intBytes(declared: string): i64 ``` The bytes of an integer column as its declared type names them: 2 (smallint, int2), 4 (int4) or 8 (every other declaration). INTEGER, INT and SERIAL are 8 bytes, as in SQLite and GizWeb: apps store millisecond times in INTEGER columns (stack/app/host.giz docs.updated). Catalogs and results still name them int4, as PostgreSQL does (db/sql/catalog/pgcatalog.giz, pgIntBytes). Effects: fail. ### `pgIntBytes` ```gizlang function pgIntBytes(declared: string): i64 ``` The bytes of an integer as PostgreSQL types it: INTEGER, INT and SERIAL are int4 (GizDB stores them in 8 bytes, intBytes; a value past int4 goes out in 8). Literals in int4 range are INT4 (db/sql/bind.giz intDeclared). Effects: fail. ### `unitName` ```gizlang function unitName(unit: i64): string ``` The SQL name of a temporal unit ("DATE", "TIMESTAMP", ...). ### `encodeTuple` ```gizlang function encodeTuple(values: Vec): bytes ``` Equality keys are opaque bytes, equal exactly when the tuple's values compare equal. Effects: fail. ### `sameTuple` ```gizlang function sameTuple(a: Vec, b: Vec): bool ``` Whether two tuples have the same width and `compareValues` finds each pair equal. Effects: fail. ### `keyOfRow` ```gizlang function keyOfRow(schema: Schema, row: Vec): Value ``` The key of `row` in a table of `schema`. Effects: fail. ### `keyValue` ```gizlang function keyValue(values: Vec): Value ``` The key of a row with the key column VALUES: the value itself for one column, else the tuple bytes. Effects: fail. ### `keyValueBounded` ```gizlang function keyValueBounded(values: Vec, maxBytes: i64): Result ``` Preserve native single-column identities while admitting canonical tuple bytes before growth. Effects: fail. ### `encodeTupleBounded` ```gizlang function encodeTupleBounded(values: Vec, maxBytes: i64): Result ``` Canonical equality keys with one allocation bound shared by all consumers. Effects: fail. ### `coerce` ```gizlang function coerce(value: Value, ty: Type): Result ``` The value converted to a column's type, or a reason it cannot be. Integers widen to floats and decimals; a decimal or float goes into a decimal column at its scale; nothing else narrows silently. Effects: fail. ### `convertUnit` ```gizlang function convertUnit(v: i64, from0: i64, to0: i64): Option ``` A temporal value of unit `from` in unit `to`: a DATE is its midnight, a TIMESTAMP's date its day (as PostgreSQL's casts), a TIMESTAMP's TIME its time of day; None between DATE and TIME. Effects: fail. ### `parseVector` ```gizlang function parseVector(text: string): Option> ``` "[1, 2.5, -3]" as a vector; pgvector's sparsevec text {1:2.5,3:-3}/5 too, as its dense elements. Effects: fail. ### `compareValues` ```gizlang function compareValues(a: Value, b: Value): i64 ``` Total order for sorting and comparing values; NULL first, then booleans, numbers (int and float compared exactly, NaN above every number), text, vectors (element by element, then shorter first), dates and times (as microseconds). Effects: fail. ### `valueWireSize` ```gizlang function valueWireSize(value: Value, limit: i64): Option ``` Exact size of this module's current wire encoding, before allocating payload bytes. UTF-8 string lengths are bytes; variable lengths must fit their u32 wire prefix. Effects: fail. ### `writeValueBounded` ```gizlang function writeValueBounded(out: mut bytes, value: Value, limit: i64): bool ``` Appends only after the complete value fits; rejection leaves out unchanged. Effects: fail. ### `writeValue` ```gizlang function writeValue(out: mut bytes, value: Value): void ``` Appends a value: a tag byte, then its payload. Effects: fail. ### `decodeCredit` ```gizlang function decodeCredit(maxValues: i64, maxBytes: i64): DecodeCredit ``` A credit of MAXVALUES values and MAXBYTES bytes for a decode; a negative limit refuses every charge. ### `rowDecodeCredit` ```gizlang function rowDecodeCredit(values: i64, wireBytes: i64): Option ``` Wire size bounds payloads, not fixed Value slots: NULL occupies one wire byte. Resource owners still admit the resulting row; this bounds materialization work. Effects: fail. ### `writeType` ```gizlang function writeType(out: mut bytes, ty: Type): void ``` Appends the encoding of the type TY. Effects: fail. ### `typeWireSize` ```gizlang function typeWireSize(ty: Type): Option ``` Exact current type wire width; reject parameters the canonical reader cannot represent. Effects: fail. ### `schemaWireSize` ```gizlang function schemaWireSize(schema: Schema, limit: i64): Option ``` Schema producer admission counts the existing wire layout without allocating output. Effects: fail. ### `writeSchemaBounded` ```gizlang function writeSchemaBounded(out: mut bytes, schema: Schema, limit: i64): Result ``` Rejection leaves the caller's existing output unchanged. Effects: fail. ### `writeSchema` ```gizlang function writeSchema(out: mut bytes, schema: Schema): void ``` Appends the encoding of SCHEMA. Effects: fail. ### `orderValues` ```gizlang function orderValues(values: Vec): bytes ``` Bytes that order as `compareValues` orders the tuple, and are equal exactly when the values compare equal (1 and 1.0 alike): the keys of ordered indexes. Effects: fail. ### `orderValue` ```gizlang function orderValue(out: mut bytes, v: Value): void ``` Appends the order key of V: bytes that compare as the values do. Effects: fail. ### `intervalSpan` ```gizlang function intervalSpan(x: string): Vec ``` Whole days and the microseconds left in [0, one day). Effects: fail. ## Methods ### `IndexDef.nullsEqual` ```gizlang function IndexDef.nullsEqual(self: IndexDef): bool ``` Whether a unique index takes its NULLs as equal values (NULLS NOT DISTINCT, the end of its options). Effects: fail. ### `Type.name` ```gizlang function Type.name(self: Type): string ``` The SQL name of the type ("BIGINT", "TEXT", "VECTOR(3)"). Effects: fail. ### `Type.same` ```gizlang function Type.same(self: Type, other: Type): bool ``` Whether the two types have the same name. Effects: fail. ### `Type.fits` ```gizlang function Type.fits(self: Type, other: Type): bool ``` Same, or two vectors of which one has no declared dimensions ('[1,2]'::vector, subvector). Effects: fail. ### `Schema.find` ```gizlang function Schema.find(self: Schema, name: string): i64 ``` The index of the column NAME, or -1. Effects: fail. ### `Value.declared` ```gizlang function Value.declared(self: Value): Type ``` The type a value has by itself (NULL has none: text stands for it). Effects: fail. ### `Value.isNull` ```gizlang function Value.isNull(self: Value): bool ``` Whether the value is NULL. ### `Value.binary` ```gizlang function Value.binary(self: Value): Option ``` Project opaque bytes without a textual encoding or an implicit conversion. ### `Value.integer` ```gizlang function Value.integer(self: Value): Option ``` The integer of an int value. ### `Value.show` ```gizlang function Value.show(self: Value): string ``` The value as SQL prints it. Effects: fail. ### `DecodeCredit.charge` ```gizlang function DecodeCredit.charge(self: mut DecodeCredit, n: i64): bool ``` Takes N bytes from the credit; false, and the credit refuses from then on, when they do not fit. Effects: fail. ### `DecodeCredit.reserve` ```gizlang function DecodeCredit.reserve(self: mut DecodeCredit, n: i64, width: i64): bool ``` Reserve fixed decoded work before growing a container; byte multiplication is overflow-safe. Effects: fail. ### `Reader.valueTextWithin` ```gizlang function Reader.valueTextWithin(self: mut Reader, credit: mut DecodeCredit, materialize: bool): string ``` The next text value of the reader, charged to CREDIT; when MATERIALIZE is false it is checked but not kept. Effects: fail, read, write, console, io. ### `Reader.value` ```gizlang function Reader.value(self: mut Reader): Value ``` The next value of the reader, with no decode limit. Effects: fail, read, write, console, io. ### `Reader.valueWithin` ```gizlang function Reader.valueWithin(self: mut Reader, credit: mut DecodeCredit, materialize: bool): Value ``` The next value of the reader, charged to CREDIT; when MATERIALIZE is false it is checked but not kept. Effects: fail, read, write, console, io. ### `Reader.columnType` ```gizlang function Reader.columnType(self: mut Reader): Type ``` The next column type of the reader. Effects: fail, read, write, console, io. ### `Reader.schema` ```gizlang function Reader.schema(self: mut Reader): Schema ``` The next schema of the reader, with no decode limit. Effects: fail, read, write, console, io. ### `Reader.schemaWithin` ```gizlang function Reader.schemaWithin(self: mut Reader, credit: mut DecodeCredit, materialize: bool): Schema ``` The next schema of the reader, charged to CREDIT; when MATERIALIZE is false it is checked but not kept. Effects: fail, read, write, console, io. ### `Schema.indexedBy` ```gizlang function Schema.indexedBy(self: Schema, c: i64, method: string): bool ``` Whether column `c` has an index of `method` (text or vector) on it alone. Effects: fail. ### `Schema.indexOptions` ```gizlang function Schema.indexOptions(self: Schema, c: i64, method: string): string ``` The options of the index with METHOD on the column C, or "" when there is none. Effects: fail. ### `Value.accepts` ```gizlang function Value.accepts(self: Value, ty: Type): bool ``` Canonical scalar storage admission, shared by logical writes and physical decoders. Effects: fail. ## Constants ### `DATE` ```gizlang const DATE = 0 ``` The unit of a DATE value: days since 1970-01-01. ### `TIMESTAMP` ```gizlang const TIMESTAMP = 1 ``` The unit of a TIMESTAMP value: microseconds since 1970-01-01, without a zone. ### `TIME` ```gizlang const TIME = 2 ``` The unit of a TIME value: microseconds since midnight. ### `TIMESTAMPTZ` ```gizlang const TIMESTAMPTZ = 3 ``` The unit of a TIMESTAMPTZ value: microseconds since 1970-01-01 UTC. ### `ZONED` ```gizlang const ZONED = 4 ``` Oracle's TIMESTAMP WITH TIME ZONE, a timestamptz that keeps the zone it was written in. As a column type its unit is ZONED. A value's unit is ZONED + a zone code: 0 is +00:00, 1 to 2879 is minutes east of UTC plus 1440 less 1439 (fixedZone), and REGION_BASE and up is a tz database region (types.giz regionUnit). Its number is still the instant in microseconds, so values compare as timestamptz do. ### `REGION_BASE` ```gizlang const REGION_BASE = 4096 ``` The first zone code of a tz database region in a ZONED unit. ### `SECOND_US` ```gizlang const SECOND_US = 1000000 ``` Microseconds in a second. ### `MINUTE_US` ```gizlang const MINUTE_US = 60000000 ``` Microseconds in a minute. ### `HOUR_US` ```gizlang const HOUR_US = 3600000000 ``` Microseconds in an hour. ### `DAY_US` ```gizlang const DAY_US = 86400000000 ``` Microseconds in a day. ### `TS_INFINITY` ```gizlang const TS_INFINITY = 9223372036854775807 ``` PostgreSQL's infinity: the largest timestamp, and the day past every date (dates end before it, types.giz dayInRange). -infinity is their negation. ### `DATE_INFINITY` ```gizlang const DATE_INFINITY = 106751991 ``` The day number of a DATE 'infinity'. Its negation is '-infinity'.