GizStack
View .mdOpen in ChatGPTOpen in Claude

gizfs:codec

The one binary encoding every stored byte of GizDB goes through: the log, segment files, the manifest and key-value snapshots. Little-endian words, u32-length-prefixed blobs, and a masked CRC-32 that seals a whole file or frame. Reads never fault on short or malformed input: they return a zero value and set bad, which the caller checks once after decoding.

gizlang
import * as Codec from "gizfs:codec";

Types

ReadResources

gizlang
type ReadResources = { scope: Ref<Resources>; windowBytes: i64; fence: Option<ResourceLease>; }

The owning actor passes one shared scope; this record issues no credit or default limit.

  • scope: Ref<M37.Resources>
  • windowBytes: i64
  • fence: _gizlang_sdk_core.Option<M37.ResourceLease>

EncodedParts

gizlang
type EncodedParts = { data: Vec<bytes>; length: i64; }

Encoded bytes in chunks of ENCODED_CHUNK_BYTES, and their length.

  • data: Vec<bytes>
  • length: i64

Format

gizlang
type Format = { name: string; versions: Vec<bytes>; }

An on-disk format: its NAME and every VERSION (its magic) that its reader opens, oldest first. Its writer writes the last one. A test opens a committed store sample of every version.

  • name: string
  • versions: Vec<bytes>

Reader

gizlang
type Reader = { data: bytes; at: i64; bad: bool; source: Option<File>; base: i64; length: i64; windowAt: i64; owned: bool; error: Option<IoError>; windowBytes: i64; admission: Option<ReadAdmission>; closed: bool; parts: Option<EncodedParts>; }

A reader of encoded values, from bytes in memory, encoded parts or a window of a file. A read past the end or of a bad value marks it bad.

  • data: bytes
  • at: i64
  • bad: bool
  • source: _gizlang_sdk_core.Option<M37.File>
  • base: i64
  • length: i64
  • windowAt: i64
  • owned: bool
  • error: _gizlang_sdk_core.Option<gizlang_system.IoError>
  • windowBytes: i64
  • admission: _gizlang_sdk_core.Option<M38.ReadAdmission>
  • closed: bool
  • parts: _gizlang_sdk_core.Option<M38.EncodedParts>

Functions

writePagePosition

gizlang
function writePagePosition(out: mut bytes, position: PagePosition): void

Physical descriptors have one codec. Semantic path prefixes and admission remain with FS/Object/DB; this codec grants no publication authority.

Effects: fail.

readResources

gizlang
function readResources(scope: Ref<Resources>, windowBytes: i64): Result<ReadResources, IoError>

The read resources of SCOPE with a window of WINDOWBYTES (1 byte to 1 MiB).

Effects: fail.

readResourcesContext

gizlang
function readResourcesContext(context: ResourceContext, windowBytes: i64): Result<ReadResources, IoError>

Reader descendants preserve the same issued fence. A serial ledger cannot be copied into a long-lived shared reader and is explicitly refused here.

Effects: fail, read.

encodedParts

gizlang
function encodedParts(): EncodedParts

Encoded parts with no bytes.

reader

gizlang
function reader(data: bytes): Reader

A reader of DATA.

fileReader

gizlang
function fileReader(file: File, base: i64, length: i64, owned: bool): Reader

The same decoder over a bounded physical window; length is the declared verified range. Direct .data indexing remains valid only for memory readers. New backends use take/bytesAt.

budgetedFileReader

gizlang
function budgetedFileReader(file: File, base: i64, length: i64, owned: bool, scope: Ref<Resources>, windowBytes: i64): Result<Reader, IoError>

Same-task admission against the caller's existing ledger; no hidden limits or fallback. On factory failure file ownership remains with the caller, including owned=true.

Effects: fail, read, write, console, io.

fileReaderResources

gizlang
function fileReaderResources(file: File, base: i64, length: i64, owned: bool, resources: ReadResources): Result<Reader, IoError>

A reader of LENGTH bytes of FILE from BASE, read in windows charged to RESOURCES; it closes FILE when OWNED.

Effects: fail, read, write, console, io.

writeFlag

gizlang
function writeFlag(out: mut bytes, value: bool): void

Appends VALUE as one byte, 1 or 0.

Effects: fail.

writeOptionalInt

gizlang
function writeOptionalInt(out: mut bytes, value: Option<i64>): void

Appends VALUE as a flag and, when it is there, its integer.

Effects: fail.

blobSize

gizlang
function blobSize(length: i64, limit: i64): Option<i64>

The exact u32-length-prefixed size, checked without allocating or overflowing.

Effects: fail.

writeInt

gizlang
function writeInt(out: mut bytes, n: i64): void

Appends N as a 64-bit little-endian integer.

Effects: fail.

writeFloat

gizlang
function writeFloat(out: mut bytes, x: f64): void

Appends X as a 64-bit little-endian float.

Effects: fail.

writeBlob

gizlang
function writeBlob(out: mut bytes, b: bytes): void

Appends B with its 32-bit length before it.

Effects: fail.

writeText

gizlang
function writeText(out: mut bytes, text: string): void

Appends TEXT as a blob of UTF-8.

Effects: fail.

writeInts

gizlang
function writeInts(out: mut bytes, ids: Vec<i64>): void

IDS as a u64 count and one i64 each (Reader.ints reads them).

Effects: fail.

checksum

gizlang
function checksum(data: bytes): u64

LevelDB's masked CRC-32: a CRC stored inside checksummed data does not checksum to itself.

Effects: fail.

checksumAppend

gizlang
function checksumAppend(crc: u64, part: bytes): u64

Fold bounded ranges without joining a potentially multi-gigabyte WAL frame.

Effects: fail.

checksumFinish

gizlang
function checksumFinish(crc: u64): u64

The masked checksum of a CRC that checksumAppend folded.

Effects: fail.

seal

gizlang
function seal(out: mut bytes): void

Appends the checksum of everything in out.

Effects: fail.

unseal

gizlang
function unseal(data: bytes): Option<bytes>

The data before a seal, or none when it is short or does not verify.

Effects: fail.

orderI64

gizlang
function orderI64(out: mut bytes, n: i64): void

An i64 as 8 big-endian bytes with the sign bit flipped.

Effects: fail.

orderF64

gizlang
function orderF64(out: mut bytes, x: f64): void

An f64 as 8 bytes ordering as the numbers do (-0 before +0; NaN after +infinity).

Effects: fail.

readOrderI64

gizlang
function readOrderI64(d: bytes, at: i64): i64

The i64 of orderI64 at AT of D.

Effects: fail.

readOrderF64

gizlang
function readOrderF64(d: bytes, at: i64): f64

The f64 of orderF64 at AT of D.

Effects: fail.

orderBytes

gizlang
function orderBytes(out: mut bytes, b: bytes): void

Bytes that order as b does and end unambiguously, so a key can continue after them: each 0 becomes 0 255, and 0 0 ends it.

Effects: fail.

Methods

Reader.pagePosition

gizlang
function Reader.pagePosition(self: mut Reader): PagePosition

The next page position: its offset, length and CRC. A bad position marks the reader bad.

Effects: fail, read, write, console, io.

ReadResources.context

gizlang
function ReadResources.context(self: ReadResources): ResourceContext

The resource context that the reads charge: the fence's, or the scope's.

ReadResources.require

gizlang
function ReadResources.require(self: ReadResources): Result<bool, IoError>

Validate the original issuer without creating or dropping a payload claim.

Effects: fail, read.

EncodedParts.append

gizlang
function EncodedParts.append(self: mut EncodedParts, data: bytes): void

Appends DATA.

Effects: fail.

EncodedParts.takeTail

gizlang
function EncodedParts.takeTail(self: mut EncodedParts): bytes

The last chunk, taken out to be written in place (EncodedParts.putTail); a new one when the last is full: an operation encoded into a fresh buffer and then copied was a fifth of a KV SET (Ops.push).

Effects: fail.

EncodedParts.putTail

gizlang
function EncodedParts.putTail(self: mut EncodedParts, tail: bytes): void

Puts back the chunk takeTail gave, with what was written to it; the part past a chunk's size starts new chunks.

Effects: fail.

EncodedParts.copyTo

gizlang
function EncodedParts.copyTo(self: EncodedParts, out: mut bytes, at: i64, n: i64): void

Appends bytes [AT, AT + N) to OUT, chunk by chunk.

Effects: fail.

EncodedParts.truncate

gizlang
function EncodedParts.truncate(self: mut EncodedParts, n: i64): void

Cuts the parts to their first N bytes.

Effects: fail.

EncodedParts.reader

gizlang
function EncodedParts.reader(self: EncodedParts, at: i64, n: i64): Reader

A reader of N bytes of the parts from AT; a bad reader when the range is outside them.

Effects: fail.

Reader.close

gizlang
function Reader.close(self: mut Reader): Result<bool, IoError>

Closes the reader's file when it owns it, and gives back its charges.

Effects: fail, read, write, console, io.

Reader.view

gizlang
function Reader.view(self: mut Reader, at: i64, n: i64): Reader

Independent bounded cursor over the same owned handle. Closing a view never closes its parent file; separate windows keep WAL offset tables and payload scans sequential.

Effects: fail, read, write, console, io.

Reader.fits

gizlang
function Reader.fits(self: mut Reader, n: i64): bool

Whether n more bytes remain; sets bad when not.

Effects: fail.

Reader.bytesAt

gizlang
function Reader.bytesAt(self: mut Reader, at: i64, n: i64): bytes

N bytes at AT of the reader's range, without moving it.

Effects: fail, read, write, console, io.

Reader.take

gizlang
function Reader.take(self: mut Reader, n: i64): bytes

The next N bytes.

Effects: fail, read, write, console, io.

Reader.byte

gizlang
function Reader.byte(self: mut Reader): u64

The next byte.

Effects: fail, read, write, console, io.

Reader.word32

gizlang
function Reader.word32(self: mut Reader): u64

The next 32-bit little-endian word.

Effects: fail, read, write, console, io.

Reader.word64

gizlang
function Reader.word64(self: mut Reader): u64

The next 64-bit little-endian word.

Effects: fail, read, write, console, io.

Reader.int64

gizlang
function Reader.int64(self: mut Reader): i64

The next 64-bit little-endian integer.

Effects: fail, read, write, console, io.

Reader.flag

gizlang
function Reader.flag(self: mut Reader): bool

Canonical booleans reject every representation other than zero and one.

Effects: fail, read, write, console, io.

Reader.optionalInt

gizlang
function Reader.optionalInt(self: mut Reader): Option<i64>

The next optional integer (writeOptionalInt).

Effects: fail, read, write, console, io.

Reader.float64

gizlang
function Reader.float64(self: mut Reader): f64

The next 64-bit little-endian float.

Effects: fail, read, write, console, io.

Reader.varint

gizlang
function Reader.varint(self: mut Reader): i64

The next zigzag LEB128 integer.

Effects: fail, read, write, console, io.

Reader.count

gizlang
function Reader.count(self: mut Reader, size: i64): i64

A u32 count of items of at least size bytes each that the remaining input can hold.

Effects: fail, read, write, console, io.

Reader.count64

gizlang
function Reader.count64(self: mut Reader, size: i64): i64

A u64 count, as count.

Effects: fail, read, write, console, io.

Reader.blob

gizlang
function Reader.blob(self: mut Reader): bytes

The next blob (a 32-bit length and its bytes).

Effects: fail, read, write, console, io.

Reader.blobWithin

gizlang
function Reader.blobWithin(self: mut Reader, maxBytes: i64): bytes

Refuse the declared payload length before slicing or allocating its bytes.

Effects: fail, read, write, console, io.

Reader.text

gizlang
function Reader.text(self: mut Reader): string

The next text (a blob of UTF-8).

Effects: fail, read, write, console, io.

Reader.textWithin

gizlang
function Reader.textWithin(self: mut Reader, maxBytes: i64): string

Bound encoded UTF-8 before materializing it, using the same blob reader.

Effects: fail, read, write, console, io.

Reader.ints

gizlang
function Reader.ints(self: mut Reader, max: i64): Vec<i64>

The ids of writeInts; more than MAX of them makes the input bad.

Effects: fail, read, write, console, io.

Reader.done

gizlang
function Reader.done(self: Reader): bool

Whether the reader is at its end.

Reader.orderedBlob

gizlang
function Reader.orderedBlob(self: mut Reader): bytes

The inverse of orderBytes: consumes one prefix-free component, preserving raw bytes. Malformed escapes and missing terminators set bad before a caller reads the next field.

Effects: fail, read, write, console, io.

Constants

WORD_BYTES

gizlang
const WORD_BYTES = 8

The bytes of an encoded integer or float.

ORDERED_END_BYTES

gizlang
const ORDERED_END_BYTES = 2

The bytes of the end mark of an ordered key.

BLOB_LENGTH_BYTES

gizlang
const BLOB_LENGTH_BYTES = 4

The bytes of the length before a blob.

ENCODED_CHUNK_BYTES

gizlang
const ENCODED_CHUNK_BYTES = 65536

Encoded scratch is chunked independently of operation boundaries; it is not a new store. Fixed chunks bound COW append/savepoint copies, not total retained transaction memory.