gizfs:codec
The one binary encoding every stored byte of GizDB goes through: the log, segment files, the manifest and key-value snapshots. Little-endian words, u32-length-prefixed blobs, and a masked CRC-32 that seals a whole file or frame. Reads never fault on short or malformed input: they return a zero value and set bad, which the caller checks once after decoding.
import * as Codec from "gizfs:codec";Types
ReadResources
type ReadResources = { scope: Ref<Resources>; windowBytes: i64; fence: Option<ResourceLease>; }The owning actor passes one shared scope; this record issues no credit or default limit.
scope:Ref<M37.Resources>windowBytes:i64fence:_gizlang_sdk_core.Option<M37.ResourceLease>
EncodedParts
type EncodedParts = { data: Vec<bytes>; length: i64; }Encoded bytes in chunks of ENCODED_CHUNK_BYTES, and their length.
data:Vec<bytes>length:i64
Format
type Format = { name: string; versions: Vec<bytes>; }An on-disk format: its NAME and every VERSION (its magic) that its reader opens, oldest first. Its writer writes the last one. A test opens a committed store sample of every version.
name:stringversions:Vec<bytes>
Reader
type Reader = { data: bytes; at: i64; bad: bool; source: Option<File>; base: i64; length: i64; windowAt: i64; owned: bool; error: Option<IoError>; windowBytes: i64; admission: Option<ReadAdmission>; closed: bool; parts: Option<EncodedParts>; }A reader of encoded values, from bytes in memory, encoded parts or a window of a file. A read past the end or of a bad value marks it bad.
data:bytesat:i64bad:boolsource:_gizlang_sdk_core.Option<M37.File>base:i64length:i64windowAt:i64owned:boolerror:_gizlang_sdk_core.Option<gizlang_system.IoError>windowBytes:i64admission:_gizlang_sdk_core.Option<M38.ReadAdmission>closed:boolparts:_gizlang_sdk_core.Option<M38.EncodedParts>
Functions
writePagePosition
function writePagePosition(out: mut bytes, position: PagePosition): voidPhysical descriptors have one codec. Semantic path prefixes and admission remain with FS/Object/DB; this codec grants no publication authority.
Effects: fail.
readResources
function readResources(scope: Ref<Resources>, windowBytes: i64): Result<ReadResources, IoError>The read resources of SCOPE with a window of WINDOWBYTES (1 byte to 1 MiB).
Effects: fail.
readResourcesContext
function readResourcesContext(context: ResourceContext, windowBytes: i64): Result<ReadResources, IoError>Reader descendants preserve the same issued fence. A serial ledger cannot be copied into a long-lived shared reader and is explicitly refused here.
Effects: fail, read.
encodedParts
function encodedParts(): EncodedPartsEncoded parts with no bytes.
reader
function reader(data: bytes): ReaderA reader of DATA.
fileReader
function fileReader(file: File, base: i64, length: i64, owned: bool): ReaderThe same decoder over a bounded physical window; length is the declared verified range. Direct .data indexing remains valid only for memory readers. New backends use take/bytesAt.
budgetedFileReader
function budgetedFileReader(file: File, base: i64, length: i64, owned: bool, scope: Ref<Resources>, windowBytes: i64): Result<Reader, IoError>Same-task admission against the caller's existing ledger; no hidden limits or fallback. On factory failure file ownership remains with the caller, including owned=true.
Effects: fail, read, write, console, io.
fileReaderResources
function fileReaderResources(file: File, base: i64, length: i64, owned: bool, resources: ReadResources): Result<Reader, IoError>A reader of LENGTH bytes of FILE from BASE, read in windows charged to RESOURCES; it closes FILE when OWNED.
Effects: fail, read, write, console, io.
writeFlag
function writeFlag(out: mut bytes, value: bool): voidAppends VALUE as one byte, 1 or 0.
Effects: fail.
writeOptionalInt
function writeOptionalInt(out: mut bytes, value: Option<i64>): voidAppends VALUE as a flag and, when it is there, its integer.
Effects: fail.
blobSize
function blobSize(length: i64, limit: i64): Option<i64>The exact u32-length-prefixed size, checked without allocating or overflowing.
Effects: fail.
writeInt
function writeInt(out: mut bytes, n: i64): voidAppends N as a 64-bit little-endian integer.
Effects: fail.
writeFloat
function writeFloat(out: mut bytes, x: f64): voidAppends X as a 64-bit little-endian float.
Effects: fail.
writeBlob
function writeBlob(out: mut bytes, b: bytes): voidAppends B with its 32-bit length before it.
Effects: fail.
writeText
function writeText(out: mut bytes, text: string): voidAppends TEXT as a blob of UTF-8.
Effects: fail.
writeInts
function writeInts(out: mut bytes, ids: Vec<i64>): voidIDS as a u64 count and one i64 each (Reader.ints reads them).
Effects: fail.
checksum
function checksum(data: bytes): u64LevelDB's masked CRC-32: a CRC stored inside checksummed data does not checksum to itself.
Effects: fail.
checksumAppend
function checksumAppend(crc: u64, part: bytes): u64Fold bounded ranges without joining a potentially multi-gigabyte WAL frame.
Effects: fail.
checksumFinish
function checksumFinish(crc: u64): u64The masked checksum of a CRC that checksumAppend folded.
Effects: fail.
seal
function seal(out: mut bytes): voidAppends the checksum of everything in out.
Effects: fail.
unseal
function unseal(data: bytes): Option<bytes>The data before a seal, or none when it is short or does not verify.
Effects: fail.
orderI64
function orderI64(out: mut bytes, n: i64): voidAn i64 as 8 big-endian bytes with the sign bit flipped.
Effects: fail.
orderF64
function orderF64(out: mut bytes, x: f64): voidAn f64 as 8 bytes ordering as the numbers do (-0 before +0; NaN after +infinity).
Effects: fail.
readOrderI64
function readOrderI64(d: bytes, at: i64): i64The i64 of orderI64 at AT of D.
Effects: fail.
readOrderF64
function readOrderF64(d: bytes, at: i64): f64The f64 of orderF64 at AT of D.
Effects: fail.
orderBytes
function orderBytes(out: mut bytes, b: bytes): voidBytes that order as b does and end unambiguously, so a key can continue after them: each
0 becomes 0 255, and 0 0 ends it.
Effects: fail.
Methods
Reader.pagePosition
function Reader.pagePosition(self: mut Reader): PagePositionThe next page position: its offset, length and CRC. A bad position marks the reader bad.
Effects: fail, read, write, console, io.
ReadResources.context
function ReadResources.context(self: ReadResources): ResourceContextThe resource context that the reads charge: the fence's, or the scope's.
ReadResources.require
function ReadResources.require(self: ReadResources): Result<bool, IoError>Validate the original issuer without creating or dropping a payload claim.
Effects: fail, read.
EncodedParts.append
function EncodedParts.append(self: mut EncodedParts, data: bytes): voidAppends DATA.
Effects: fail.
EncodedParts.takeTail
function EncodedParts.takeTail(self: mut EncodedParts): bytesThe last chunk, taken out to be written in place (EncodedParts.putTail); a new one when the last is full: an operation encoded into a fresh buffer and then copied was a fifth of a KV SET (Ops.push).
Effects: fail.
EncodedParts.putTail
function EncodedParts.putTail(self: mut EncodedParts, tail: bytes): voidPuts back the chunk takeTail gave, with what was written to it; the part past a chunk's size starts new chunks.
Effects: fail.
EncodedParts.copyTo
function EncodedParts.copyTo(self: EncodedParts, out: mut bytes, at: i64, n: i64): voidAppends bytes [AT, AT + N) to OUT, chunk by chunk.
Effects: fail.
EncodedParts.truncate
function EncodedParts.truncate(self: mut EncodedParts, n: i64): voidCuts the parts to their first N bytes.
Effects: fail.
EncodedParts.reader
function EncodedParts.reader(self: EncodedParts, at: i64, n: i64): ReaderA reader of N bytes of the parts from AT; a bad reader when the range is outside them.
Effects: fail.
Reader.close
function Reader.close(self: mut Reader): Result<bool, IoError>Closes the reader's file when it owns it, and gives back its charges.
Effects: fail, read, write, console, io.
Reader.view
function Reader.view(self: mut Reader, at: i64, n: i64): ReaderIndependent bounded cursor over the same owned handle. Closing a view never closes its parent file; separate windows keep WAL offset tables and payload scans sequential.
Effects: fail, read, write, console, io.
Reader.fits
function Reader.fits(self: mut Reader, n: i64): boolWhether n more bytes remain; sets bad when not.
Effects: fail.
Reader.bytesAt
function Reader.bytesAt(self: mut Reader, at: i64, n: i64): bytesN bytes at AT of the reader's range, without moving it.
Effects: fail, read, write, console, io.
Reader.take
function Reader.take(self: mut Reader, n: i64): bytesThe next N bytes.
Effects: fail, read, write, console, io.
Reader.byte
function Reader.byte(self: mut Reader): u64The next byte.
Effects: fail, read, write, console, io.
Reader.word32
function Reader.word32(self: mut Reader): u64The next 32-bit little-endian word.
Effects: fail, read, write, console, io.
Reader.word64
function Reader.word64(self: mut Reader): u64The next 64-bit little-endian word.
Effects: fail, read, write, console, io.
Reader.int64
function Reader.int64(self: mut Reader): i64The next 64-bit little-endian integer.
Effects: fail, read, write, console, io.
Reader.flag
function Reader.flag(self: mut Reader): boolCanonical booleans reject every representation other than zero and one.
Effects: fail, read, write, console, io.
Reader.optionalInt
function Reader.optionalInt(self: mut Reader): Option<i64>The next optional integer (writeOptionalInt).
Effects: fail, read, write, console, io.
Reader.float64
function Reader.float64(self: mut Reader): f64The next 64-bit little-endian float.
Effects: fail, read, write, console, io.
Reader.varint
function Reader.varint(self: mut Reader): i64The next zigzag LEB128 integer.
Effects: fail, read, write, console, io.
Reader.count
function Reader.count(self: mut Reader, size: i64): i64A u32 count of items of at least size bytes each that the remaining input can hold.
Effects: fail, read, write, console, io.
Reader.count64
function Reader.count64(self: mut Reader, size: i64): i64A u64 count, as count.
Effects: fail, read, write, console, io.
Reader.blob
function Reader.blob(self: mut Reader): bytesThe next blob (a 32-bit length and its bytes).
Effects: fail, read, write, console, io.
Reader.blobWithin
function Reader.blobWithin(self: mut Reader, maxBytes: i64): bytesRefuse the declared payload length before slicing or allocating its bytes.
Effects: fail, read, write, console, io.
Reader.text
function Reader.text(self: mut Reader): stringThe next text (a blob of UTF-8).
Effects: fail, read, write, console, io.
Reader.textWithin
function Reader.textWithin(self: mut Reader, maxBytes: i64): stringBound encoded UTF-8 before materializing it, using the same blob reader.
Effects: fail, read, write, console, io.
Reader.ints
function Reader.ints(self: mut Reader, max: i64): Vec<i64>The ids of writeInts; more than MAX of them makes the input bad.
Effects: fail, read, write, console, io.
Reader.done
function Reader.done(self: Reader): boolWhether the reader is at its end.
Reader.orderedBlob
function Reader.orderedBlob(self: mut Reader): bytesThe inverse of orderBytes: consumes one prefix-free component, preserving raw bytes. Malformed escapes and missing terminators set bad before a caller reads the next field.
Effects: fail, read, write, console, io.
Constants
WORD_BYTES
const WORD_BYTES = 8The bytes of an encoded integer or float.
ORDERED_END_BYTES
const ORDERED_END_BYTES = 2The bytes of the end mark of an ordered key.
BLOB_LENGTH_BYTES
const BLOB_LENGTH_BYTES = 4The bytes of the length before a blob.
ENCODED_CHUNK_BYTES
const ENCODED_CHUNK_BYTES = 65536Encoded scratch is chunked independently of operation boundaries; it is not a new store. Fixed chunks bound COW append/savepoint copies, not total retained transaction memory.