# gizml:tensor n-dimensional f32 arrays on the CPU's vector lanes and cores. One Tensor type over one storage (row-major Vec), numpy broadcasting, elementwise chains fused into one pass (Lazy: a small tensor IR run 1024 elements at a time in L1), reductions with a policy for their order, and matmul on packed 6x32 FMA tiles across cores. ```gizlang import * as Tensor from "gizml:tensor"; ``` ## Types ### `Tensor` ```gizlang type Tensor ``` A tensor: in host memory (`data`, f32) or on a GPU (`_gpu`, with `data` empty). Operations run where their operands are; `on(gpu)` and `host()` move one. Operands of one operation are in one place. - `shape`: `Vec` - `data`: `Vec` ### `OnGpu` ```gizlang type OnGpu = { gpu: Gpu; buffer: Buffer; dtype: i64; } ``` A GPU tensor's memory: `dtype` elements (F32, F16, BF16 or FP8). - `gpu`: `M10.Gpu` - `buffer`: `M10.Buffer` - `dtype`: `i64` ### `Reduction` ```gizlang type Reduction = | { kind: "strict"; } | { kind: "deterministic"; } | { kind: "fast"; } ``` The order a reduction adds in: `strict` is one fixed order and f64 partial sums on every machine (bitwise reproducible), `deterministic` the same result each run on one machine, `fast` any order across cores. - `strict` - `deterministic` - `fast` ### `Key` ```gizlang type Key = { k0: u64; k1: u64; } ``` A random key: two 32-bit words. `key(seed)` makes one; `split` derives new ones. - `k0`: `u64` - `k1`: `u64` ### `Lazy` ```gizlang type Lazy ``` An elementwise expression over tensors, evaluated by `eval`. With a `_product`, input 0 is that product, not yet computed: on a GPU the expression runs as the product is stored (its epilogue), so C is written once. - `shape`: `Vec` - `inputs`: `Vec` - `steps`: `Vec` ### `Product` ```gizlang type Product = { a: Tensor; ta: bool; b: Tensor; tb: bool; m: i64; k: i64; n: i64; compute: i64; low: Vec; lowAt: i64; patch: Vec; } ``` A times B as matmulT takes them: m x k times k x n, on a GPU with the operands rounded to `compute` (-1: as they are; lazyMatmulIn). `low` is [D, U] of a low-rank product D Uᵀ (D m x r, U n x r, both F16 or BF16 on a GPU) the chain adds as its input `lowAt`: on NVIDIA's matrix cores the product's epilogue computes it from the stage memory the main loop is done with, so D Uᵀ is never stored (a LoRA's up projection); elsewhere lowApart. - `a`: `M32.Tensor` - `ta`: `bool` - `b`: `M32.Tensor` - `tb`: `bool` - `m`: `i64` - `k`: `i64` - `n`: `i64` - `compute`: `i64` - `low`: `Vec` - `lowAt`: `i64` - `patch`: `Vec` ### `Named` ```gizlang type Named = { name: string; tensor: Tensor; } ``` A named tensor, as a safetensors file holds them. - `name`: `string` - `tensor`: `M32.Tensor` ## Functions ### `vexp` ```gizlang function vexp(x: f32x16): f32x16 ``` e^x per lane, within 2 ulp (Cephes expf): 2^n by the exponent bits, the rest by a degree-6 polynomial on |r| <= ln2/2. Overflow gives inf, underflow 0. ### `vlog` ```gizlang function vlog(x: f32x16): f32x16 ``` ln x per lane, within 2 ulp (Cephes logf); negative gives NaN, 0 gives -inf. ### `bytesOf` ```gizlang function bytesOf(dtype: i64, count: i64): i64 ``` Bytes of `count` elements of dtype: I4 packs two a byte. Effects: fail. ### `fromData` ```gizlang function fromData(shape: Vec, data: Vec): Tensor ``` A host tensor of `shape` over `data` (as many as the shape has). Effects: fail. ### `placed` ```gizlang function placed(shape: Vec, gpu: Gpu, buffer: Buffer, dtype: i64): Tensor ``` A tensor of SHAPE whose values are in BUFFER on GPU, of the element type DTYPE. ### `elements` ```gizlang function elements(shape: Vec): i64 ``` The number of elements of a tensor of SHAPE. Effects: fail. ### `tensor` ```gizlang function tensor(shape: Vec, values: Vec): Tensor ``` A tensor of `shape` holding `values` (as many as the shape has, rounded to f32). Effects: fail. ### `full` ```gizlang function full(shape: Vec, value: f64): Tensor ``` A host tensor of SHAPE with every value VALUE. Effects: fail. ### `zeros` ```gizlang function zeros(shape: Vec): Tensor ``` A host tensor of SHAPE with every value 0. Effects: fail. ### `ones` ```gizlang function ones(shape: Vec): Tensor ``` A host tensor of SHAPE with every value 1. Effects: fail. ### `scalar` ```gizlang function scalar(value: f64): Tensor ``` A 0-d tensor: a scalar that broadcasts to any shape. Effects: fail. ### `arange` ```gizlang function arange(n: i64): Tensor ``` 0, 1, ..., n-1. Effects: fail. ### `key` ```gizlang function key(seed: u64): Key ``` The random key of SEED. Effects: fail. ### `philox` ```gizlang function philox(k: Key, c0: u64, c1: u64, out: mut Vec, at: i64): void ``` Four 32-bit words for counter (c0, c1), written to out[at..at+4) as far as it reaches. Effects: fail. ### `uniform` ```gizlang function uniform(k: Key, shape: Vec, low: f64, high: f64): Tensor ``` Uniform in [low, high): the top 24 bits of each word, so every value is exact in f32. Effects: fail. ### `normal` ```gizlang function normal(k: Key, shape: Vec): Tensor ``` Standard normal (mean 0, deviation 1) by Box-Muller on pairs of words, in f64. Effects: fail. ### `onDevice` ```gizlang function onDevice(shape: Vec, gpu: Gpu, buffer: Buffer, dtype: i64): Tensor ``` The tensor of `shape` whose `dtype` elements (row-major) are device memory `buffer` of `gpu`: a kernel's result, or weights uploaded as their bytes. ### `broadcastShape` ```gizlang function broadcastShape(a: Vec, b: Vec): Vec ``` The shape two shapes broadcast to (numpy: aligned at the right, 1 stretches). Effects: fail. ### `lazy` ```gizlang function lazy(t: Tensor): Lazy ``` T as a lazy expression with no steps yet. Effects: fail. ### `lazyMatmul` ```gizlang function lazyMatmul(x: Tensor, transposeA: bool, y: Tensor, transposeB: bool): Lazy ``` x times y as matmulT, not yet computed: the steps on it run as the product is stored. Effects: fail, read, write, console, io. ### `lazyMatmulIn` ```gizlang function lazyMatmulIn(x: Tensor, transposeA: bool, y: Tensor, transposeB: bool, compute: i64): Lazy ``` As lazyMatmul, with operands on a GPU rounded to `compute` (F16, BF16 or FP8) as Tensor.cast rounds them; in host memory they stay f32. The matrix cores' kernel rounds an F32 operand of an F16 or BF16 product as it copies it, so the cast takes no pass of its own (the bits are those of the cast and then the product). FP8 operands are scaled to its range first (fp8Scaled), and the product's store multiplies by both scales. Effects: fail, read, write, console, io. ### `lazyProduct` ```gizlang function lazyProduct(shape: Vec, product: Product): Lazy ``` The expression of PRODUCT alone, of `shape`: its input 0 stands for the product and has no values. Effects: fail. ### `fp8Scaled` ```gizlang function fp8Scaled(t: Tensor): Vec ``` `t` scaled to the range of FP8 as tensorwise scaling (Transformer Engine) does it: [t / s as FP8, s], with s = max |t| / 448 a tensor of shape [] where t is. Effects: fail, read, write, console, io. ### `fp4Blocks` ```gizlang function fp4Blocks(t: Tensor, across: bool): Vec ``` `t` (on a GPU) in NVFP4 blocks with its rows along k, as NVIDIA's Transformer Engine makes them: [q, s]. q is of type FP4 and shape [rows, k]: rows x k / 2 bytes of E2M1 pairs (row-major, the first in the low half), then k / 16 E4M3 block scales for each of `lines` (rows rounded up to 4; zeros past rows), in words of 4 (64 of k) by columns: word w of row r at w lines + r, so the words of a warp's mma lie together and copy in 16 bytes. s (F32, shape []) is max |t| / (6 x 448). The value at row r, p is its E2M1 times the scale of its block times s. `across`: t is [k, rows], else its last axis is k. Effects: fail, read, write, console, io. ### `productType` ```gizlang function productType(a: i64, b: i64): i64 ``` The type of a product of operands of types a and b, as in torch: theirs when both are F16 or both BF16 (the sums are f32, rounded once at the end), else F32. ### `matmulT` ```gizlang function matmulT(x: Tensor, transposeA: bool, y: Tensor, transposeB: bool): Tensor ``` A times B with either read transposed in place, no copy made: A is (..., m, k) or, with transposeA, a 2-d (k, m); B is (k, n) or, with transposeB, (n, k). Each sum adds a[i][p] b[p][j] by fma for p in order from 0, so the result is the same bits however the work is split. B is packed once in panels of 32 columns; rows go in blocks across the cores once there are 2^21 or more multiply-adds, as many blocks as cores (in multiples of 6 rows, at most 96), and within a block 6x32 FMA tiles stay in registers. The result is of productType. Effects: fail, read, write, console, io. ### `gpuOf` ```gizlang function gpuOf(xs: Vec): Gpu ``` The GPU of the first operand there, every other operand moved to it. Effects: fail. ### `keyOf` ```gizlang function keyOf(shape: Vec): string ``` The text of SHAPE as a key ("[2,3,]"). Effects: fail. ### `blocks256` ```gizlang function blocks256(threads: i64): i64 ``` The blocks of 256 threads that cover THREADS threads. Effects: fail. ### `longs` ```gizlang function longs(xs: Vec): bytes ``` XS as 64-bit little-endian words, for kernel arguments. Effects: fail. ### `concat` ```gizlang function concat(xs: Vec, axis: i64): Tensor ``` The tensors joined along `axis` (equal elsewhere), in order. Effects: fail, read, write, console, io. ### `batchMatmul` ```gizlang function batchMatmul(x: Tensor, transposeA: bool, y: Tensor, transposeB: bool): Tensor ``` Batched matmul: x (b, m, k) (or (b, k, m) with transposeA) times y (b, k, n) (or (b, n, k) with transposeB), or one y (k, n) for every batch. Effects: fail, read, write, console, io. ### `within` ```gizlang function within(buffer: Buffer, gpu: Gpu, shape: Vec, dtype: i64): Tensor ``` A tensor of `shape` and `dtype` on the first bytes of `buffer` (work memory a caller reuses). Effects: fail. ### `toSafetensors` ```gizlang function toSafetensors(tensors: Vec): bytes ``` TENSORS as a safetensors file of F32 data, in their order. Effects: fail, read, write, console, io. ### `fromSafetensors` ```gizlang function fromSafetensors(data: bytes): Vec throws string ``` The tensors of a safetensors file, in the header's order, as f32 (gizlang:safetensors decoded). Effects: fail. ### `placedOn` ```gizlang function placedOn(t: Tensor, gpu: Option): Tensor ``` Tensor t on `gpu` when there is one, else as it is: where a model runs. Effects: fail, read, write, console, io. ### `storedOn` ```gizlang function storedOn(sys: System, s: Stored, gpu: Option, linear: bool): Tensor throws string ``` A stored tensor where it runs: a linear as BF16 on a GPU, anything else as f32 (BF16, F16, F32 and E4M3 bytes go to the GPU as they are stored and are cast there). Effects: fail, read, write, console, io. ## Methods ### `Key.words` ```gizlang function Key.words(self: Key, n: i64): Vec ``` N random 32-bit words: word i from counter (i / 4, 0). Effects: fail. ### `Key.split` ```gizlang function Key.split(self: Key, n: i64): Vec ``` N keys independent of this one and of each other (counters (i, 1)). Effects: fail. ### `Tensor.text` ```gizlang function Tensor.text(self: Tensor): string ``` The values nested by shape, as numpy prints them: `[[1, 2], [3, 4]]`. A GPU tensor is copied to the host first. Effects: fail, read, write, console, io. ### `Tensor.size` ```gizlang function Tensor.size(self: Tensor): i64 ``` The number of values: the product of the shape. Effects: fail. ### `Tensor.gpu` ```gizlang function Tensor.gpu(self: Tensor): Option ``` Where the tensor is: none for host memory. ### `Tensor.isHost` ```gizlang function Tensor.isHost(self: Tensor): bool ``` Whether the tensor is in host memory. ### `Tensor.dtype` ```gizlang function Tensor.dtype(self: Tensor): i64 ``` The element type: F32 for a host tensor. ### `Tensor.on` ```gizlang function Tensor.on(self: Tensor, gpu: Gpu): Tensor ``` A copy on `gpu` (the same tensor when it is there). Effects: fail, read, write, console, io. ### `Tensor.host` ```gizlang function Tensor.host(self: Tensor): Tensor ``` A copy in host memory (the same tensor when it is there), as f32. Effects: fail, read, write, console, io. ### `Tensor.cast` ```gizlang function Tensor.cast(self: Tensor, dtype: i64): Tensor ``` The same values stored as `dtype` (F32, F16, BF16 or FP8; rounded to nearest even, FP8 saturated at ±448): on a GPU only, as host tensors are f32. Effects: fail, read, write, console, io. ### `Tensor.assign` ```gizlang function Tensor.assign(self: Tensor, x: Tensor): void ``` Writes x's values into this GPU tensor's memory (x of the same shape, on the same GPU or the host), so that the kernels of a recording (Gpu.record) that read this tensor read them. Effects: fail, read, write, console, io. ### `Tensor.buffer` ```gizlang function Tensor.buffer(self: Tensor): Buffer ``` The buffer of a GPU tensor. Effects: fail. ### `Tensor.onGpu` ```gizlang function Tensor.onGpu(self: Tensor): OnGpu ``` Where a GPU tensor is: its device, buffer and element type. Effects: fail. ### `Tensor.placement` ```gizlang function Tensor.placement(self: Tensor): Option ``` Where the tensor is: on a GPU (its device, buffer and element type), or none for host memory. ### `Tensor.deviceBuffer` ```gizlang function Tensor.deviceBuffer(self: Tensor): Buffer ``` A GPU tensor's device memory, for kernels made elsewhere (gizml:tile): its `dtype()` elements in row-major order. Effects: fail. ### `Tensor.rank` ```gizlang function Tensor.rank(self: Tensor): i64 ``` The number of axes. ### `Tensor.get` ```gizlang function Tensor.get(self: Tensor, index: Vec): f64 ``` The element at a full index (R001 outside). Effects: fail, read, write, console, io. ### `Tensor.values` ```gizlang function Tensor.values(self: Tensor): Vec ``` The values in row-major order, as f64, copied to the host when the tensor is on a GPU. Effects: fail, read, write, console, io. ### `Tensor.reshape` ```gizlang function Tensor.reshape(self: Tensor, shape: Vec): Tensor ``` The same elements in another shape of the same size; one -1 is inferred. Effects: fail. ### `Lazy.pending` ```gizlang function Lazy.pending(self: Lazy): bool ``` Whether input 0 is a product not yet evaluated (`_product`): a step over it evaluates that product first. ### `Lazy.expand` ```gizlang function Lazy.expand(self: Lazy, shape: Vec): Lazy ``` The expression broadcast to SHAPE. Effects: fail, read, write, console, io. ### `Lazy.add` ```gizlang function Lazy.add(self: Lazy, y: Lazy): Lazy ``` SELF + Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Lazy.sub` ```gizlang function Lazy.sub(self: Lazy, y: Lazy): Lazy ``` SELF - Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Lazy.mul` ```gizlang function Lazy.mul(self: Lazy, y: Lazy): Lazy ``` SELF * Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Lazy.div` ```gizlang function Lazy.div(self: Lazy, y: Lazy): Lazy ``` SELF / Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Lazy.maximum` ```gizlang function Lazy.maximum(self: Lazy, y: Lazy): Lazy ``` The larger of SELF and Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Lazy.minimum` ```gizlang function Lazy.minimum(self: Lazy, y: Lazy): Lazy ``` The smaller of SELF and Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Lazy.greater` ```gizlang function Lazy.greater(self: Lazy, y: Lazy): Lazy ``` 1 where self > y, else 0. Effects: fail, read, write, console, io. ### `Lazy.neg` ```gizlang function Lazy.neg(self: Lazy): Lazy ``` -SELF, element by element. Effects: fail. ### `Lazy.abs` ```gizlang function Lazy.abs(self: Lazy): Lazy ``` The absolute value of SELF, element by element. Effects: fail. ### `Lazy.sqrt` ```gizlang function Lazy.sqrt(self: Lazy): Lazy ``` The square root of SELF, element by element. Effects: fail. ### `Lazy.exp` ```gizlang function Lazy.exp(self: Lazy): Lazy ``` e to the power SELF, element by element. Effects: fail. ### `Lazy.log` ```gizlang function Lazy.log(self: Lazy): Lazy ``` The natural logarithm of SELF, element by element. Effects: fail. ### `Lazy.tanh` ```gizlang function Lazy.tanh(self: Lazy): Lazy ``` The hyperbolic tangent of SELF, element by element. Effects: fail. ### `Lazy.sigmoid` ```gizlang function Lazy.sigmoid(self: Lazy): Lazy ``` The sigmoid 1 / (1 + e^-x) of SELF, element by element. Effects: fail. ### `Lazy.relu` ```gizlang function Lazy.relu(self: Lazy): Lazy ``` max(SELF, 0), element by element. Effects: fail. ### `Lazy.square` ```gizlang function Lazy.square(self: Lazy): Lazy ``` SELF squared, element by element. Effects: fail. ### `Lazy.roundedBf16` ```gizlang function Lazy.roundedBf16(self: Lazy): Lazy ``` Each value rounded to BF16 (nearest even) and kept as F32: where a pipeline stores BF16 between two steps that a product's epilogue now runs together. Effects: fail. ### `Lazy.gelu` ```gizlang function Lazy.gelu(self: Lazy): Lazy ``` GELU, tanh form (as GPT-2 and torch's approximate="tanh"). Effects: fail. ### `Lazy.geluGrad` ```gizlang function Lazy.geluGrad(self: Lazy, dy: Lazy): Lazy ``` The gradient through gelu: self is x, `dy` the output's gradient. Effects: fail, read, write, console, io. ### `Lazy.geluErf` ```gizlang function Lazy.geluErf(self: Lazy): Lazy ``` GELU, erf form (torch's default and BERT's "gelu"). Effects: fail. ### `Lazy.geluErfGrad` ```gizlang function Lazy.geluErfGrad(self: Lazy, dy: Lazy): Lazy ``` The gradient through geluErf: self is x, `dy` the output's gradient. Effects: fail, read, write, console, io. ### `Lazy.on` ```gizlang function Lazy.on(self: Lazy, gpu: Gpu): Lazy ``` The expression with its inputs moved to `gpu`, so `eval` runs it there. Effects: fail, read, write, console, io. ### `Lazy.like` ```gizlang function Lazy.like(self: Lazy): Tensor ``` A tensor of the expression's shape where it is evaluated (a GPU or host memory), with no values: for its shape and place (Tensor.filled, Tensor.gpu), never to read. Effects: fail. ### `Lazy.eval` ```gizlang function Lazy.eval(self: Lazy): Tensor ``` The expression's tensor: one pass, on every core when it is large, or one kernel on the GPU of an input. It is f32 but over a product, where it is of the product's type (productType) when every other input is of that type or has one element, as torch's type promotion; else f32. Effects: fail, read, write, console, io. ### `Lazy.evalIn` ```gizlang function Lazy.evalIn(self: Lazy, dtype: i64): Tensor ``` The expression's tensor stored as `dtype` (F32, F16, BF16 or FP8) on a GPU, so a cast takes no pass of its own; in host memory it is f32. Effects: fail, read, write, console, io. ### `Lazy.evalInto` ```gizlang function Lazy.evalInto(self: Lazy, target: Tensor): void ``` Stores the expression's values into GPU tensor `target` (of its shape) as `target`'s type: an update in place, so the kernels of a recording (Gpu.record) that read `target` read them. Its kernel writes there where nothing it reads can see the write (it reads `target` only at the element it writes); else it computes a new tensor and copies it. Effects: fail, read, write, console, io. ### `Lazy.gluIn` ```gizlang function Lazy.gluIn(self: Lazy, dtype: i64): Tensor ``` A product chain's rows [m, n] as SwiGLU pairs, `dtype` elements [m, n / 2]: column c is silu(v[2c]) times v[2c + 1], each after the chain (a gate and an up projection with their rows interleaved). On a GPU the product's store makes them, so C's n columns are never stored. Effects: fail, read, write, console, io. ### `Tensor.glu` ```gizlang function Tensor.glu(self: Tensor): Tensor ``` SwiGLU of the rows [m, n] of a host tensor as pairs, [m, n / 2]: column c is silu(v[2c]) times v[2c + 1], rounded to f32 (Lazy.gluIn on the host). Effects: fail. ### `Lazy.gluInto` ```gizlang function Lazy.gluInto(self: Lazy, target: Tensor): void ``` Lazy.gluIn into GPU tensor target ([m, n / 2]). Effects: fail, read, write, console, io. ### `Lazy.reads` ```gizlang function Lazy.reads(self: Lazy, t: Tensor): bool ``` Whether the expression reads GPU tensor t's memory (an input or a product's operand). Effects: fail. ### `Tensor.add` ```gizlang function Tensor.add(self: Tensor, y: Tensor): Tensor ``` SELF + Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.sub` ```gizlang function Tensor.sub(self: Tensor, y: Tensor): Tensor ``` SELF - Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.mul` ```gizlang function Tensor.mul(self: Tensor, y: Tensor): Tensor ``` SELF * Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.div` ```gizlang function Tensor.div(self: Tensor, y: Tensor): Tensor ``` SELF / Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.maximum` ```gizlang function Tensor.maximum(self: Tensor, y: Tensor): Tensor ``` The larger of SELF and Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.minimum` ```gizlang function Tensor.minimum(self: Tensor, y: Tensor): Tensor ``` The smaller of SELF and Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.neg` ```gizlang function Tensor.neg(self: Tensor): Tensor ``` -SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.abs` ```gizlang function Tensor.abs(self: Tensor): Tensor ``` The absolute value of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.sqrt` ```gizlang function Tensor.sqrt(self: Tensor): Tensor ``` The square root of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.exp` ```gizlang function Tensor.exp(self: Tensor): Tensor ``` e to the power SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.log` ```gizlang function Tensor.log(self: Tensor): Tensor ``` The natural logarithm of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.tanh` ```gizlang function Tensor.tanh(self: Tensor): Tensor ``` The hyperbolic tangent of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.sigmoid` ```gizlang function Tensor.sigmoid(self: Tensor): Tensor ``` The sigmoid 1 / (1 + e^-x) of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.relu` ```gizlang function Tensor.relu(self: Tensor): Tensor ``` max(SELF, 0), element by element. Effects: fail, read, write, console, io. ### `Tensor.square` ```gizlang function Tensor.square(self: Tensor): Tensor ``` SELF squared, element by element. Effects: fail, read, write, console, io. ### `Tensor.gelu` ```gizlang function Tensor.gelu(self: Tensor): Tensor ``` GELU of SELF (the tanh approximation), element by element. Effects: fail, read, write, console, io. ### `Tensor.geluErf` ```gizlang function Tensor.geluErf(self: Tensor): Tensor ``` GELU of SELF (the exact erf form), element by element. Effects: fail, read, write, console, io. ### `Tensor.expand` ```gizlang function Tensor.expand(self: Tensor, shape: Vec): Tensor ``` The tensor broadcast to `shape`, where it is. Effects: fail, read, write, console, io. ### `Tensor.filled` ```gizlang function Tensor.filled(self: Tensor, shape: Vec, value: f64): Tensor ``` `value` in every element of `shape`, where this tensor is (host memory or its GPU). Effects: fail, read, write, console, io. ### `Tensor.item` ```gizlang function Tensor.item(self: Tensor): f64 ``` The value of a one-element tensor. Effects: fail, read, write, console, io. ### `Tensor.sum` ```gizlang function Tensor.sum(self: Tensor, policy: Reduction): f64 ``` The sum of all values, in the order POLICY sets (strict: the same on any core count and device). Effects: fail, read, write, console, io. ### `Tensor.mean` ```gizlang function Tensor.mean(self: Tensor, policy: Reduction): f64 ``` The mean of all values, summed in the order POLICY sets. Effects: fail, read, write, console, io. ### `Tensor.max` ```gizlang function Tensor.max(self: Tensor): f64 ``` The largest value. Effects: fail, read, write, console, io. ### `Tensor.transpose` ```gizlang function Tensor.transpose(self: Tensor): Tensor ``` The 2-d transpose, in bands of 32 output rows on every core (Task.fill); columns past the last whole band one by one. Effects: fail, read, write, console, io. ### `Tensor.softmax` ```gizlang function Tensor.softmax(self: Tensor): Tensor ``` softmax over the last axis: max-subtracted so no exp overflows. Effects: fail, read, write, console, io. ### `Tensor.matmul` ```gizlang function Tensor.matmul(self: Tensor, y: Tensor): Tensor ``` self (..., m, k) times y (k, n): leading axes are rows too. Effects: fail, read, write, console, io. ### `Lazy.total` ```gizlang function Lazy.total(self: Lazy, policy: Reduction): Tensor ``` The sum of the expression's elements (Tensor.total) without its tensor: on a GPU, a strict sum of a small one is one kernel that computes each element as it adds it. Effects: fail, read, write, console, io. ### `Lazy.average` ```gizlang function Lazy.average(self: Lazy, policy: Reduction): Tensor ``` The mean of the expression's elements (Tensor.average), as Lazy.total. Effects: fail, read, write, console, io. ### `Tensor.total` ```gizlang function Tensor.total(self: Tensor, policy: Reduction): Tensor ``` The sum (Tensor.sum) as a tensor of shape [] where this one is: on a GPU it stays there, with no wait. Effects: fail, read, write, console, io. ### `Tensor.average` ```gizlang function Tensor.average(self: Tensor, policy: Reduction): Tensor ``` The mean (Tensor.mean) as a tensor of shape [] where this one is, as Tensor.total. Effects: fail, read, write, console, io. ### `Tensor.sumTo` ```gizlang function Tensor.sumTo(self: Tensor, shape: Vec): Tensor ``` Summed down to `shape` over the axes broadcasting stretched (the adjoint of broadcasting; a reduction over any axes that keeps them as 1). Over the last axis only, each row adds as `sum(Reduction.strict())` adds a part (f64 lanes); otherwise each element adds its sources in index order from 0 in f32. Effects: fail, read, write, console, io. ### `Tensor.maxTo` ```gizlang function Tensor.maxTo(self: Tensor, shape: Vec): Tensor ``` The largest element over the axes that `shape` keeps as 1 (NaN is ignored, as by `max`; -Infinity where every element is NaN or none is left). Effects: fail, read, write, console, io. ### `Tensor.take` ```gizlang function Tensor.take(self: Tensor, ids: Vec): Tensor ``` Rows `ids` of the tensor (its first axis): an embedding lookup. Effects: fail, read, write, console, io. ### `Tensor.addRows` ```gizlang function Tensor.addRows(self: Tensor, ids: Vec, rows: i64): Tensor ``` The adjoint of take: a (rows, ...) tensor whose row r adds the rows i of this one with ids[i] = r, in increasing i (on a GPU each output element adds its own list, so no two threads write one place and the order holds). Effects: fail, read, write, console, io. ### `Tensor.slice` ```gizlang function Tensor.slice(self: Tensor, axis: i64, start: i64, end: i64): Tensor ``` Elements start..end of `axis`. Effects: fail, read, write, console, io. ### `Tensor.permute` ```gizlang function Tensor.permute(self: Tensor, axes: Vec): Tensor ``` The axes reordered: output axis i is input axis axes[i]. Effects: fail, read, write, console, io. ### `Tensor.rows` ```gizlang function Tensor.rows(self: Tensor, r0: i64, r1: i64): Tensor ``` Rows [r0, r1) of the tensor ([rows, ...] on a GPU) on the same memory, with no copy (Tensor.slice copies). Effects: fail. ### `Tensor.address` ```gizlang function Tensor.address(self: Tensor): u64 ``` The device address of the tensor's first element (a kernel's buffer argument). Effects: fail. ### `Stored.upload` ```gizlang function Stored.upload(self: Stored, sys: System, gpu: Gpu): Buffer throws string ``` The stored tensor's raw bytes in a new buffer on `gpu`, read from its file straight to the device (Gpu.uploadFile). Effects: fail, read, write, io. ## Constants ### `F32` ```gizlang const F32 = 0 ``` The element type of 32-bit float GPU tensors (gizhw:kernel's). ### `F16` ```gizlang const F16 = 4 ``` The element type of IEEE half-precision GPU tensors. ### `BF16` ```gizlang const BF16 = 5 ``` The element type of bfloat16 GPU tensors. ### `FP8` ```gizlang const FP8 = 9 ``` FP8 E4M3 (gizhw:kernel E_E4M3): 4 exponent bits, 3 mantissa bits, at most ±448. A storage type: its products are F32 (on matrix cores where the GPU has them, hasFp8Mma) and it takes no scale; scale the values first. ### `FP4` ```gizlang const FP4 = 11 ``` NVFP4: as `compute` of lazyMatmulIn, each operand in blocks of 16 along k of FP4 E2M1 values, each block scaled by an E4M3 and the whole operand by an F32 (fp4Blocks). A tensor of this type is such an operand only. ### `I8` ```gizlang const I8 = 7 ``` Signed 8-bit integers (gizhw:kernel E_I8): a storage type whose products of two such operands are F32 sums of integers on the matrix cores (hasI8Mma; A by rows and B by columns along k). It takes no scale; scale the values first and multiply the product by the scales. ### `I4` ```gizlang const I4 = 13 ``` Signed 4-bit integers, two a byte (gizhw:kernel E_I4X4 holds four): as I8 on the matrix cores where hasI4Mma, k a multiple of 4. Made by int8Rows with 4 bits; an operand only. ### `OP_INPUT` ```gizlang const OP_INPUT = 0 ``` The step of a lazy expression that reads one of its inputs.