# gizfs:codec The one binary encoding every stored byte of GizDB goes through: the log, segment files, the manifest and key-value snapshots. Little-endian words, u32-length-prefixed blobs, and a masked CRC-32 that seals a whole file or frame. Reads never fault on short or malformed input: they return a zero value and set `bad`, which the caller checks once after decoding. ```gizlang import * as Codec from "gizfs:codec"; ``` ## Types ### `ReadResources` ```gizlang type ReadResources = { scope: Ref; windowBytes: i64; fence: Option; } ``` The owning actor passes one shared scope; this record issues no credit or default limit. - `scope`: `Ref` - `windowBytes`: `i64` - `fence`: `_gizlang_sdk_core.Option` ### `EncodedParts` ```gizlang type EncodedParts = { data: Vec; length: i64; } ``` Encoded bytes in chunks of ENCODED_CHUNK_BYTES, and their length. - `data`: `Vec` - `length`: `i64` ### `Format` ```gizlang type Format = { name: string; versions: Vec; } ``` An on-disk format: its NAME and every VERSION (its magic) that its reader opens, oldest first. Its writer writes the last one. A test opens a committed store sample of every version. - `name`: `string` - `versions`: `Vec` ### `Reader` ```gizlang type Reader = { data: bytes; at: i64; bad: bool; source: Option; base: i64; length: i64; windowAt: i64; owned: bool; error: Option; windowBytes: i64; admission: Option; closed: bool; parts: Option; } ``` A reader of encoded values, from bytes in memory, encoded parts or a window of a file. A read past the end or of a bad value marks it bad. - `data`: `bytes` - `at`: `i64` - `bad`: `bool` - `source`: `_gizlang_sdk_core.Option` - `base`: `i64` - `length`: `i64` - `windowAt`: `i64` - `owned`: `bool` - `error`: `_gizlang_sdk_core.Option` - `windowBytes`: `i64` - `admission`: `_gizlang_sdk_core.Option` - `closed`: `bool` - `parts`: `_gizlang_sdk_core.Option` ## Functions ### `writePagePosition` ```gizlang function writePagePosition(out: mut bytes, position: PagePosition): void ``` Physical descriptors have one codec. Semantic path prefixes and admission remain with FS/Object/DB; this codec grants no publication authority. Effects: fail. ### `readResources` ```gizlang function readResources(scope: Ref, windowBytes: i64): Result ``` The read resources of SCOPE with a window of WINDOWBYTES (1 byte to 1 MiB). Effects: fail. ### `readResourcesContext` ```gizlang function readResourcesContext(context: ResourceContext, windowBytes: i64): Result ``` Reader descendants preserve the same issued fence. A serial ledger cannot be copied into a long-lived shared reader and is explicitly refused here. Effects: fail, read. ### `encodedParts` ```gizlang function encodedParts(): EncodedParts ``` Encoded parts with no bytes. ### `reader` ```gizlang function reader(data: bytes): Reader ``` A reader of DATA. ### `fileReader` ```gizlang function fileReader(file: File, base: i64, length: i64, owned: bool): Reader ``` The same decoder over a bounded physical window; length is the declared verified range. Direct .data indexing remains valid only for memory readers. New backends use take/bytesAt. ### `budgetedFileReader` ```gizlang function budgetedFileReader(file: File, base: i64, length: i64, owned: bool, scope: Ref, windowBytes: i64): Result ``` Same-task admission against the caller's existing ledger; no hidden limits or fallback. On factory failure file ownership remains with the caller, including owned=true. Effects: fail, read, write, console, io. ### `fileReaderResources` ```gizlang function fileReaderResources(file: File, base: i64, length: i64, owned: bool, resources: ReadResources): Result ``` A reader of LENGTH bytes of FILE from BASE, read in windows charged to RESOURCES; it closes FILE when OWNED. Effects: fail, read, write, console, io. ### `writeFlag` ```gizlang function writeFlag(out: mut bytes, value: bool): void ``` Appends VALUE as one byte, 1 or 0. Effects: fail. ### `writeOptionalInt` ```gizlang function writeOptionalInt(out: mut bytes, value: Option): void ``` Appends VALUE as a flag and, when it is there, its integer. Effects: fail. ### `blobSize` ```gizlang function blobSize(length: i64, limit: i64): Option ``` The exact u32-length-prefixed size, checked without allocating or overflowing. Effects: fail. ### `writeInt` ```gizlang function writeInt(out: mut bytes, n: i64): void ``` Appends N as a 64-bit little-endian integer. Effects: fail. ### `writeFloat` ```gizlang function writeFloat(out: mut bytes, x: f64): void ``` Appends X as a 64-bit little-endian float. Effects: fail. ### `writeBlob` ```gizlang function writeBlob(out: mut bytes, b: bytes): void ``` Appends B with its 32-bit length before it. Effects: fail. ### `writeText` ```gizlang function writeText(out: mut bytes, text: string): void ``` Appends TEXT as a blob of UTF-8. Effects: fail. ### `writeInts` ```gizlang function writeInts(out: mut bytes, ids: Vec): void ``` IDS as a u64 count and one i64 each (Reader.ints reads them). Effects: fail. ### `checksum` ```gizlang function checksum(data: bytes): u64 ``` LevelDB's masked CRC-32: a CRC stored inside checksummed data does not checksum to itself. Effects: fail. ### `checksumAppend` ```gizlang function checksumAppend(crc: u64, part: bytes): u64 ``` Fold bounded ranges without joining a potentially multi-gigabyte WAL frame. Effects: fail. ### `checksumFinish` ```gizlang function checksumFinish(crc: u64): u64 ``` The masked checksum of a CRC that checksumAppend folded. Effects: fail. ### `seal` ```gizlang function seal(out: mut bytes): void ``` Appends the checksum of everything in `out`. Effects: fail. ### `unseal` ```gizlang function unseal(data: bytes): Option ``` The data before a `seal`, or none when it is short or does not verify. Effects: fail. ### `orderI64` ```gizlang function orderI64(out: mut bytes, n: i64): void ``` An i64 as 8 big-endian bytes with the sign bit flipped. Effects: fail. ### `orderF64` ```gizlang function orderF64(out: mut bytes, x: f64): void ``` An f64 as 8 bytes ordering as the numbers do (-0 before +0; NaN after +infinity). Effects: fail. ### `readOrderI64` ```gizlang function readOrderI64(d: bytes, at: i64): i64 ``` The i64 of orderI64 at AT of D. Effects: fail. ### `readOrderF64` ```gizlang function readOrderF64(d: bytes, at: i64): f64 ``` The f64 of orderF64 at AT of D. Effects: fail. ### `orderBytes` ```gizlang function orderBytes(out: mut bytes, b: bytes): void ``` Bytes that order as `b` does and end unambiguously, so a key can continue after them: each 0 becomes 0 255, and 0 0 ends it. Effects: fail. ## Methods ### `Reader.pagePosition` ```gizlang function Reader.pagePosition(self: mut Reader): PagePosition ``` The next page position: its offset, length and CRC. A bad position marks the reader bad. Effects: fail, read, write, console, io. ### `ReadResources.context` ```gizlang function ReadResources.context(self: ReadResources): ResourceContext ``` The resource context that the reads charge: the fence's, or the scope's. ### `ReadResources.require` ```gizlang function ReadResources.require(self: ReadResources): Result ``` Validate the original issuer without creating or dropping a payload claim. Effects: fail, read. ### `EncodedParts.append` ```gizlang function EncodedParts.append(self: mut EncodedParts, data: bytes): void ``` Appends DATA. Effects: fail. ### `EncodedParts.takeTail` ```gizlang function EncodedParts.takeTail(self: mut EncodedParts): bytes ``` The last chunk, taken out to be written in place (EncodedParts.putTail); a new one when the last is full: an operation encoded into a fresh buffer and then copied was a fifth of a KV SET (Ops.push). Effects: fail. ### `EncodedParts.putTail` ```gizlang function EncodedParts.putTail(self: mut EncodedParts, tail: bytes): void ``` Puts back the chunk takeTail gave, with what was written to it; the part past a chunk's size starts new chunks. Effects: fail. ### `EncodedParts.copyTo` ```gizlang function EncodedParts.copyTo(self: EncodedParts, out: mut bytes, at: i64, n: i64): void ``` Appends bytes [AT, AT + N) to OUT, chunk by chunk. Effects: fail. ### `EncodedParts.truncate` ```gizlang function EncodedParts.truncate(self: mut EncodedParts, n: i64): void ``` Cuts the parts to their first N bytes. Effects: fail. ### `EncodedParts.reader` ```gizlang function EncodedParts.reader(self: EncodedParts, at: i64, n: i64): Reader ``` A reader of N bytes of the parts from AT; a bad reader when the range is outside them. Effects: fail. ### `Reader.close` ```gizlang function Reader.close(self: mut Reader): Result ``` Closes the reader's file when it owns it, and gives back its charges. Effects: fail, read, write, console, io. ### `Reader.view` ```gizlang function Reader.view(self: mut Reader, at: i64, n: i64): Reader ``` Independent bounded cursor over the same owned handle. Closing a view never closes its parent file; separate windows keep WAL offset tables and payload scans sequential. Effects: fail, read, write, console, io. ### `Reader.fits` ```gizlang function Reader.fits(self: mut Reader, n: i64): bool ``` Whether `n` more bytes remain; sets `bad` when not. Effects: fail. ### `Reader.bytesAt` ```gizlang function Reader.bytesAt(self: mut Reader, at: i64, n: i64): bytes ``` N bytes at AT of the reader's range, without moving it. Effects: fail, read, write, console, io. ### `Reader.take` ```gizlang function Reader.take(self: mut Reader, n: i64): bytes ``` The next N bytes. Effects: fail, read, write, console, io. ### `Reader.byte` ```gizlang function Reader.byte(self: mut Reader): u64 ``` The next byte. Effects: fail, read, write, console, io. ### `Reader.word32` ```gizlang function Reader.word32(self: mut Reader): u64 ``` The next 32-bit little-endian word. Effects: fail, read, write, console, io. ### `Reader.word64` ```gizlang function Reader.word64(self: mut Reader): u64 ``` The next 64-bit little-endian word. Effects: fail, read, write, console, io. ### `Reader.int64` ```gizlang function Reader.int64(self: mut Reader): i64 ``` The next 64-bit little-endian integer. Effects: fail, read, write, console, io. ### `Reader.flag` ```gizlang function Reader.flag(self: mut Reader): bool ``` Canonical booleans reject every representation other than zero and one. Effects: fail, read, write, console, io. ### `Reader.optionalInt` ```gizlang function Reader.optionalInt(self: mut Reader): Option ``` The next optional integer (writeOptionalInt). Effects: fail, read, write, console, io. ### `Reader.float64` ```gizlang function Reader.float64(self: mut Reader): f64 ``` The next 64-bit little-endian float. Effects: fail, read, write, console, io. ### `Reader.varint` ```gizlang function Reader.varint(self: mut Reader): i64 ``` The next zigzag LEB128 integer. Effects: fail, read, write, console, io. ### `Reader.count` ```gizlang function Reader.count(self: mut Reader, size: i64): i64 ``` A u32 count of items of at least `size` bytes each that the remaining input can hold. Effects: fail, read, write, console, io. ### `Reader.count64` ```gizlang function Reader.count64(self: mut Reader, size: i64): i64 ``` A u64 count, as `count`. Effects: fail, read, write, console, io. ### `Reader.blob` ```gizlang function Reader.blob(self: mut Reader): bytes ``` The next blob (a 32-bit length and its bytes). Effects: fail, read, write, console, io. ### `Reader.blobWithin` ```gizlang function Reader.blobWithin(self: mut Reader, maxBytes: i64): bytes ``` Refuse the declared payload length before slicing or allocating its bytes. Effects: fail, read, write, console, io. ### `Reader.text` ```gizlang function Reader.text(self: mut Reader): string ``` The next text (a blob of UTF-8). Effects: fail, read, write, console, io. ### `Reader.textWithin` ```gizlang function Reader.textWithin(self: mut Reader, maxBytes: i64): string ``` Bound encoded UTF-8 before materializing it, using the same blob reader. Effects: fail, read, write, console, io. ### `Reader.ints` ```gizlang function Reader.ints(self: mut Reader, max: i64): Vec ``` The ids of writeInts; more than MAX of them makes the input bad. Effects: fail, read, write, console, io. ### `Reader.done` ```gizlang function Reader.done(self: Reader): bool ``` Whether the reader is at its end. ### `Reader.orderedBlob` ```gizlang function Reader.orderedBlob(self: mut Reader): bytes ``` The inverse of orderBytes: consumes one prefix-free component, preserving raw bytes. Malformed escapes and missing terminators set bad before a caller reads the next field. Effects: fail, read, write, console, io. ## Constants ### `WORD_BYTES` ```gizlang const WORD_BYTES = 8 ``` The bytes of an encoded integer or float. ### `ORDERED_END_BYTES` ```gizlang const ORDERED_END_BYTES = 2 ``` The bytes of the end mark of an ordered key. ### `BLOB_LENGTH_BYTES` ```gizlang const BLOB_LENGTH_BYTES = 4 ``` The bytes of the length before a blob. ### `ENCODED_CHUNK_BYTES` ```gizlang const ENCODED_CHUNK_BYTES = 65536 ``` Encoded scratch is chunked independently of operation boundaries; it is not a new store. Fixed chunks bound COW append/savepoint copies, not total retained transaction memory.