GizStack
View .mdOpen in ChatGPTOpen in Claude

gizmodel/core/kernels

The vector loops over column storage: the one owner of SIMD in the GizDB executor. The batch kernels (db/sql/exec/vm.giz), single-group aggregates (db/sql/exec/exec.giz) and zone maps (db/core/segment.giz) call these instead of looping row by row; vector distances are db/core/vector.giz's. 64-bit lanes go 8 at a time (one AVX-512 register, two AVX2 or four NEON ones: the compiler picks), bytes 32 at a time, f32 16 at a time; a scalar tail finishes each loop. Every kernel means exactly what its scalar tail does, which is what the callers' row loops did before, so results do not depend on the lane count.

Comparison ops are 0 =, 1 <>, 2 <, 3 <=, 4 >, 5 >= (db/sql/exec/vm.giz's EQ..GE less EQ). Floats compare as compareValues orders them: NaN equals NaN and is above every number.

gizlang
import * as Kernels from "gizmodel/core/kernels";

Types

Sums

gizlang
type Sums = { count: i64; sum: i64; min: i64; max: i64; ok: bool; fsum: f64; wrapped: bool; }

The non-null count, integer sum and float sum. ok proves every ordered prefix fits i64; false selects the row fold, since cancellation can hide overflow in a lane-reordered sum. wrapped: a value plus sumInts' plus passes its width, which the caller's expression reports as the dialect does.

  • count: i64
  • sum: i64
  • min: i64
  • max: i64
  • ok: bool
  • fsum: f64
  • wrapped: bool

WideSums

gizlang
type WideSums = { count: i64; th: i64; tl: u64; lh: i64; ll: u64; hh: i64; hl: u64; fsum: f64; }

A run of integers summed exactly: the count, the total and the least and greatest prefix sums (the empty one, 0, included) as 128-bit two's complement words (high, low), and the float sum in row order. It is the state func.giz's stepInteger leaves after the run one value at a time (foldIntegers).

  • count: i64
  • th: i64
  • tl: u64
  • lh: i64
  • ll: u64
  • hh: i64
  • hl: u64
  • fsum: f64

Bounds

gizlang
type Bounds = { lo: i64; hi: i64; }

The least and greatest of v[from..to], which must hold at least one value (zone maps).

  • lo: i64
  • hi: i64

Truth

gizlang
type Truth = { v: Vec<i64>; nulls: bytes; }

A truth column: not 0 is true, with its nulls (empty: none).

  • v: Vec<i64>
  • nulls: bytes

Tri

gizlang
type Tri = { t: bytes; nul: bytes; }

A three-valued result over rows: t the true rows, nul the null rows, one bit each.

  • t: bytes
  • nul: bytes

Functions

compareInts

gizlang
function compareInts(op: i64, x: Vec<i64>, y: Vec<i64>, scalar: bool, s: i64, n: i64): Vec<i64>

0 or 1 per row: x[i] op (y[i], or s when scalar).

Effects: fail, alloc.

compareFloats

gizlang
function compareFloats(op: i64, x: Vec<f64>, y: Vec<f64>, scalar: bool, s: f64, n: i64): Vec<i64>

compareInts for floats.

Effects: fail, alloc.

arithInts

gizlang
function arithInts(op: i64, x: Vec<i64>, xs: bool, xv: i64, y: Vec<i64>, ys: bool, yv: i64, nulls: bytes, n: i64, width: i64): Option<Vec<i64>>

x op y per row, op 0 +, 1 -, 2 * (each side a column, or its scalar when xs/ys); none when a row that is not null overflows i64, or the bounds of width (a product is refused from 9.2e18, as the scalar path's check). Null rows hold 0.

Effects: fail, alloc.

arithFloats

gizlang
function arithFloats(op: i64, x: Vec<f64>, xs: bool, xv: f64, y: Vec<f64>, ys: bool, yv: f64, n: i64): Vec<f64>

arithInts for floats (no overflow; null rows hold whatever their lanes compute).

Effects: fail, alloc.

intsAsFloats

gizlang
function intsAsFloats(v: Vec<i64>, n: i64): Vec<f64>

Each int as a float.

Effects: fail, alloc.

orNulls

gizlang
function orNulls(a: bytes, b: bytes, n: i64): bytes

The union of two nulls arrays of n rows (an empty one has no nulls).

Effects: fail, alloc.

nullFlags

gizlang
function nullFlags(nulls: bytes, n: i64, negate: bool): Vec<i64>

1 per null row (0 per null row when negate), from a nulls array.

Effects: fail, alloc.

selectedRows

gizlang
function selectedRows(v: Vec<i64>, nulls: bytes, n: i64): Vec<u32>

The rows whose flag is not 0 and that are not null.

Effects: fail, alloc.

rankedRows

gizlang
function rankedRows(out: mut Vec<u32>, v: Vec<i64>, nulls: bytes, from: i64, n: i64, bits: Vec<u64>, klo: i64, top: i64): void

The rows FROM..FROM+N (row numbers) of V, pushed onto OUT, whose value is a key of a ranked build: bit j of BITS for the value KLO + j, for j from 0 to TOP. A null row is no key. One loop with few live values: in the row loop of exec.giz joinFirst each row was 25 instructions, 7 of them reloads (TPC-H Q11: 38% of its time).

Effects: fail, alloc.

countSelectedRows

gizlang
function countSelectedRows(v: Vec<i64>, nulls: bytes, n: i64): i64

selectedRows(v, nulls, n).length without the rows: a count(*) over a filter needs no gather.

Effects: fail.

sumInts

gizlang
function sumInts(v: Vec<i64>, nulls: bytes, n: i64, plus: i64, size: i64): Sums

Sums of the non-NULL values of v, each plus plus (sum(x + 1) with no column made for x + 1: ClickBench Q29's 90 sums), whose result is an integer of size bytes (arithInts' width: 4 and 2 are PostgreSQL's integer and smallint, 0 an i64).

Effects: fail.

wideSums

gizlang
function wideSums(v: Vec<i64>, nulls: bytes, n: i64, local: Vec<i64>, k: i64, plus: i64): Vec<WideSums>

WideSums of the non-NULL values of v per group: row i is in group local[i] of k (every row in group 0 when local is empty). For runs whose i64 sum may overflow (sumInts' ok false): ClickBench's UserID is near 2^62, so every batch of its AVG took the row loop, a boxed value and a decimal add a row. Each value is taken plus plus, which the caller found does not wrap (sumInts).

Effects: fail, alloc.

sumFloats

gizlang
function sumFloats(v: Vec<f64>, nulls: bytes, n: i64): Sums

The count, sum, least and greatest of the first N values of V that are not null.

Effects: fail.

extremeInts

gizlang
function extremeInts(v: Vec<i64>, nulls: bytes, n: i64, greatest: bool): i64

The first row holding the greatest (else least) non-null value; -1 when every row is null.

Effects: fail.

extremeFloats

gizlang
function extremeFloats(v: Vec<f64>, nulls: bytes, n: i64, greatest: bool): i64

extremeInts for floats, in compareValues' order: NaN is the greatest value.

Effects: fail.

boundsInts

gizlang
function boundsInts(v: Vec<i64>, from: i64, to: i64): Bounds

The least and greatest of V from FROM up to TO.

Effects: fail.

countNulls

gizlang
function countNulls(nulls: bytes, n: i64): i64

How many of n rows are null.

Effects: fail.

logic3

gizlang
function logic3(and: bool, a: Vec<i64>, an: bytes, b: Vec<i64>, bn: bytes, n: i64): Truth

SQL's three-valued AND (else OR) of two truth columns: false (for OR, true) decides even against a null; otherwise a null makes the result null.

Effects: fail, alloc.

bitsCmpInts

gizlang
function bitsCmpInts(op: i64, x: Vec<i64>, from: i64, n: i64, s: i64): bytes

The rows whose x[from + r] is op s.

Effects: fail, alloc.

bitsCmpFloats

gizlang
function bitsCmpFloats(op: i64, x: Vec<f64>, from: i64, n: i64, s: f64): bytes

bitsCmpInts for floats, ordered as compareFloats orders them.

Effects: fail, alloc.

bitsCmpIntCols

gizlang
function bitsCmpIntCols(op: i64, x: Vec<i64>, y: Vec<i64>, from: i64, n: i64): bytes

x[from + r] op y[from + r] for two integer columns.

Effects: fail, alloc.

bitsCodeRange

gizlang
function bitsCodeRange(codes: Vec<u32>, from: i64, n: i64, lo: u64, hi: u64): bytes

The rows whose dictionary code is in lo..hi.

Effects: fail, alloc.

bitsCodeTable

gizlang
function bitsCodeTable(codes: Vec<u32>, table: bytes, from: i64, n: i64): bytes

The rows whose dictionary code table marks (one byte per code, 1 for true).

Effects: fail, alloc.

bitsOfBytes

gizlang
function bitsOfBytes(flags: bytes, from: i64, n: i64): bytes

The rows whose flag byte (a nulls array) is not 0.

Effects: fail, alloc.

bitsOfFlags

gizlang
function bitsOfFlags(v: Vec<i64>, from: i64, n: i64): bytes

The rows whose 0/1 flag is not 0 (a boolean column).

Effects: fail, alloc.

bitsAll

gizlang
function bitsAll(n: i64): bytes

n rows all set.

Effects: fail, alloc.

bitsNone

gizlang
function bitsNone(n: i64): bytes

A bitmap of N rows with no bit set.

Effects: fail, alloc.

triLogic

gizlang
function triLogic(and: bool, a: Tri, b: Tri, n: i64): Tri

SQL AND (else OR) of two truths over n rows.

Effects: fail, alloc.

triNot

gizlang
function triNot(a: Tri, n: i64): Tri

SQL NOT.

Effects: fail, alloc.

bitsAnd

gizlang
function bitsAnd(a: bytes, b: bytes): bytes

The rows that are true in both of a, b (no nulls to carry): a conjunction of filters.

Effects: fail, alloc.

bitsCount

gizlang
function bitsCount(a: bytes): i64

How many of the n rows are set.

Effects: fail.

bitsRows

gizlang
function bitsRows(a: bytes): Vec<u32>

The set rows, in order.

Effects: fail, alloc.

onesAt

gizlang
function onesAt(a: bytes): Vec<i64>

The rows a mask of one byte a row sets (each byte 0 or 1, so bitsCount counts them), in order. A block of 32 clear rows costs one test.

Effects: fail, alloc.

bitsMinus

gizlang
function bitsMinus(a: bytes, b: bytes): bytes

The rows set in a and not in b.

Effects: fail, alloc.

blockBytes

gizlang
function blockBytes(width: i64): i64

The bytes packBlock writes for 128 values of WIDTH bits: 8 lanes of 16 values, each lane in whole 32-bit words.

Effects: fail.

packBlock

gizlang
function packBlock(out: mut bytes, v: Vec<u32>, from: i64, width: i64): void

Appends the 128 values V[FROM, FROM + 128), each below 2^WIDTH (0 to 32), packed by lanes: value 8s + l is lane l's value s, at bit s * WIDTH of the lane, and word k of the lanes is the 32 bytes from 32k. So unpackBlock reads 8 values with one load and shift, where a value at a time cost five times as long (2026-10-29).

Effects: fail, alloc.

unpackBlock

gizlang
function unpackBlock(data: bytes, at: i64, width: i64, plus: u64, out: mut Vec<u32>, to: i64): void

The 128 values packBlock wrote at AT of DATA, each plus PLUS, into OUT from TO.

Effects: fail, alloc.

unpackGaps

gizlang
function unpackGaps(data: bytes, at: i64, width: i64, last: u64, out: mut Vec<u32>, to: i64): void

The 128 ascending values whose gaps less one packBlock wrote at AT of DATA, after LAST (0xffffffff before the first value 0), into OUT from TO: each step adds its 8 gaps in the register (a prefix sum of three shifts) and carries the last.

Effects: fail, alloc.