# gizai:tensor/core the Tensor and its storage, f32 vector math in lanes (Cephes polynomials), reductions with an order policy, layout, matmul on the CPU's FMA tiles, sums to a shape, views and files. ```gizlang import * as Core from "gizai:tensor/core"; ``` ## Types ### `Named` ```gizlang type Named = { name: string; tensor: Tensor; } ``` A named tensor, as a safetensors file holds them. - `name`: `string` - `tensor`: `_gizlang_lib_ai__tensor__storage.Tensor` ## Functions ### `matmulT` ```gizlang function matmulT(x: Tensor, transposeA: bool, y: Tensor, transposeB: bool): Tensor ``` A times B with either read transposed in place, no copy made: A is (..., m, k) or, with transposeA, a 2-d (k, m); B is (k, n) or, with transposeB, (n, k). Each sum adds a[i][p] b[p][j] by fma for p in order from 0, so the result is the same bits however the work is split. B is packed once in panels of 32 columns; rows go in blocks across the cores once there are 2^21 or more multiply-adds, as many blocks as cores (in multiples of 6 rows, at most 96), and within a block 6x32 FMA tiles stay in registers. The result is of productType. Effects: fail, read, write, console, io. ### `productEval` ```gizlang function productEval(x: Lazy, p: Product, dtype: i64, into: Option): Tensor ``` A product chain's tensor: on the GPU of any operand or input in one kernel, else the product in host memory and then the chain over it. Effects: fail, read, write, console, io. ### `longs` ```gizlang function longs(xs: Vec): bytes ``` XS as 64-bit little-endian words, for kernel arguments. Effects: fail. ### `concat` ```gizlang function concat(xs: Vec, axis: i64, dtype: i64 = F32): Tensor ``` The tensors joined along `axis` (equal elsewhere), in order, stored as `dtype` on a GPU. Effects: fail, read, write, console, io. ### `batchMatmul` ```gizlang function batchMatmul(x: Tensor, transposeA: bool, y: Tensor, transposeB: bool): Tensor ``` Batched matmul: x (b, m, k) (or (b, k, m) with transposeA) times y (b, k, n) (or (b, n, k) with transposeB), or one y (k, n) for every batch. Effects: fail, read, write, console, io. ### `within` ```gizlang function within(buffer: Buffer, gpu: Gpu, shape: Vec, dtype: i64): Tensor ``` A tensor of `shape` and `dtype` on the first bytes of `buffer` (work memory a caller reuses). Effects: fail. ### `toSafetensors` ```gizlang function toSafetensors(tensors: Vec): bytes ``` TENSORS as a safetensors file of F32 data, in their order. Effects: fail, read, write, console, io. ### `fromSafetensors` ```gizlang function fromSafetensors(data: bytes): Vec throws string ``` The tensors of a safetensors file, in the header's order, as f32 (gizlang:safetensors decoded). Effects: fail. ### `placedOn` ```gizlang function placedOn(t: Tensor, gpu: Option): Tensor ``` Tensor t on `gpu` when there is one, else as it is: where a model runs. Effects: fail, read, write, console, io. ### `storedOn` ```gizlang function storedOn(sys: System, s: Stored, gpu: Option, linear: bool): Tensor throws string ``` A stored tensor where it runs: a linear as BF16 on a GPU, anything else as f32 (BF16, F16, F32 and E4M3 bytes go to the GPU as they are stored and are cast there). Effects: fail, read, write, console, io. ### `textOf` ```gizlang function textOf(t: Tensor, axis: i64, at: i64): string ``` The values of tensor `t` along `axis` from flat offset `at`, nested as bracketed lists. Effects: fail. ### `onDevice` ```gizlang function onDevice(shape: Vec, gpu: Gpu, buffer: Buffer, dtype: i64): Tensor ``` The tensor of `shape` whose `dtype` elements (row-major) are device memory `buffer` of `gpu`: a kernel's result, or weights uploaded as their bytes. ## Methods ### `Tensor.text` ```gizlang function Tensor.text(self: Tensor): string ``` The values nested by shape, as numpy prints them: `[[1, 2], [3, 4]]`. A GPU tensor is copied to the host first. Effects: fail, read, write, console, io. ### `Tensor.size` ```gizlang function Tensor.size(self: Tensor): i64 ``` The number of values: the product of the shape. Effects: fail. ### `Tensor.gpu` ```gizlang function Tensor.gpu(self: Tensor): Option ``` Where the tensor is: none for host memory. ### `Tensor.isHost` ```gizlang function Tensor.isHost(self: Tensor): bool ``` Whether the tensor is in host memory. ### `Tensor.dtype` ```gizlang function Tensor.dtype(self: Tensor): i64 ``` The element type: F32 for a host tensor. ### `Tensor.on` ```gizlang function Tensor.on(self: Tensor, gpu: Gpu): Tensor ``` A copy on `gpu` (the same tensor when it is there). Effects: fail, read, write, console, io. ### `Tensor.host` ```gizlang function Tensor.host(self: Tensor): Tensor ``` A copy in host memory (the same tensor when it is there), as f32. Effects: fail, read, write, console, io. ### `Tensor.cast` ```gizlang function Tensor.cast(self: Tensor, dtype: i64): Tensor ``` The same values stored as `dtype` (F32, F16, BF16 or FP8; rounded to nearest even, FP8 saturated at ±448): on a GPU only, as host tensors are f32. Effects: fail, read, write, console, io. ### `Tensor.assign` ```gizlang function Tensor.assign(self: Tensor, x: Tensor): void ``` Writes x's values into this GPU tensor's memory (x of the same shape, on the same GPU or the host), so that the kernels of a recording (Gpu.record) that read this tensor read them. Effects: fail, read, write, console, io. ### `Tensor.buffer` ```gizlang function Tensor.buffer(self: Tensor): Buffer ``` The buffer of a GPU tensor. Effects: fail. ### `Tensor.onGpu` ```gizlang function Tensor.onGpu(self: Tensor): OnGpu ``` Where a GPU tensor is: its device, buffer and element type. Effects: fail. ### `Tensor.placement` ```gizlang function Tensor.placement(self: Tensor): Option ``` Where the tensor is: on a GPU (its device, buffer and element type), or none for host memory. ### `Tensor.deviceBuffer` ```gizlang function Tensor.deviceBuffer(self: Tensor): Buffer ``` A GPU tensor's device memory, for kernels made elsewhere (gizai:tile): its `dtype()` elements in row-major order. Effects: fail. ### `Tensor.rank` ```gizlang function Tensor.rank(self: Tensor): i64 ``` The number of axes. ### `Tensor.get` ```gizlang function Tensor.get(self: Tensor, index: Vec): f64 ``` The element at a full index (R001 outside). Effects: fail, read, write, console, io. ### `Tensor.values` ```gizlang function Tensor.values(self: Tensor): Vec ``` The values in row-major order, as f64, copied to the host when the tensor is on a GPU. Effects: fail, read, write, console, io. ### `Tensor.reshape` ```gizlang function Tensor.reshape(self: Tensor, shape: Vec): Tensor ``` The same elements in another shape of the same size; one -1 is inferred. Effects: fail. ### `Tensor.glu` ```gizlang function Tensor.glu(self: Tensor): Tensor ``` SwiGLU of the rows [m, n] of a host tensor as pairs, [m, n / 2]: column c is silu(v[2c]) times v[2c + 1], rounded to f32 (Lazy.gluIn on the host). Effects: fail. ### `Tensor.add` ```gizlang function Tensor.add(self: Tensor, y: Tensor): Tensor ``` SELF + Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.sub` ```gizlang function Tensor.sub(self: Tensor, y: Tensor): Tensor ``` SELF - Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.mul` ```gizlang function Tensor.mul(self: Tensor, y: Tensor): Tensor ``` SELF * Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.div` ```gizlang function Tensor.div(self: Tensor, y: Tensor): Tensor ``` SELF / Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.maximum` ```gizlang function Tensor.maximum(self: Tensor, y: Tensor): Tensor ``` The larger of SELF and Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.minimum` ```gizlang function Tensor.minimum(self: Tensor, y: Tensor): Tensor ``` The smaller of SELF and Y, element by element, with broadcasting. Effects: fail, read, write, console, io. ### `Tensor.neg` ```gizlang function Tensor.neg(self: Tensor): Tensor ``` -SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.abs` ```gizlang function Tensor.abs(self: Tensor): Tensor ``` The absolute value of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.sqrt` ```gizlang function Tensor.sqrt(self: Tensor): Tensor ``` The square root of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.exp` ```gizlang function Tensor.exp(self: Tensor): Tensor ``` e to the power SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.log` ```gizlang function Tensor.log(self: Tensor): Tensor ``` The natural logarithm of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.tanh` ```gizlang function Tensor.tanh(self: Tensor): Tensor ``` The hyperbolic tangent of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.sigmoid` ```gizlang function Tensor.sigmoid(self: Tensor): Tensor ``` The sigmoid 1 / (1 + e^-x) of SELF, element by element. Effects: fail, read, write, console, io. ### `Tensor.relu` ```gizlang function Tensor.relu(self: Tensor): Tensor ``` max(SELF, 0), element by element. Effects: fail, read, write, console, io. ### `Tensor.square` ```gizlang function Tensor.square(self: Tensor): Tensor ``` SELF squared, element by element. Effects: fail, read, write, console, io. ### `Tensor.gelu` ```gizlang function Tensor.gelu(self: Tensor): Tensor ``` GELU of SELF (the tanh approximation), element by element. Effects: fail, read, write, console, io. ### `Tensor.geluErf` ```gizlang function Tensor.geluErf(self: Tensor): Tensor ``` GELU of SELF (the exact erf form), element by element. Effects: fail, read, write, console, io. ### `Tensor.expand` ```gizlang function Tensor.expand(self: Tensor, shape: Vec): Tensor ``` The tensor broadcast to `shape`, where it is. Effects: fail, read, write, console, io. ### `Tensor.filled` ```gizlang function Tensor.filled(self: Tensor, shape: Vec, value: f64): Tensor ``` `value` in every element of `shape`, where this tensor is (host memory or its GPU). Effects: fail, read, write, console, io. ### `Tensor.item` ```gizlang function Tensor.item(self: Tensor): f64 ``` The value of a one-element tensor. Effects: fail, read, write, console, io. ### `Tensor.sum` ```gizlang function Tensor.sum(self: Tensor, policy: Reduction): f64 ``` The sum of all values, in the order POLICY sets (strict: the same on any core count and device). Effects: fail, read, write, console, io. ### `Tensor.mean` ```gizlang function Tensor.mean(self: Tensor, policy: Reduction): f64 ``` The mean of all values, summed in the order POLICY sets. Effects: fail, read, write, console, io. ### `Tensor.max` ```gizlang function Tensor.max(self: Tensor): f64 ``` The largest value. Effects: fail, read, write, console, io. ### `Tensor.transpose` ```gizlang function Tensor.transpose(self: Tensor): Tensor ``` The 2-d transpose, in bands of 32 output rows on every core (Task.fill); columns past the last whole band one by one. Effects: fail, read, write, console, io. ### `Tensor.softmax` ```gizlang function Tensor.softmax(self: Tensor): Tensor ``` softmax over the last axis: max-subtracted so no exp overflows. Effects: fail, read, write, console, io. ### `Tensor.packed` ```gizlang function Tensor.packed(self: Tensor, transposed: bool): Tensor ``` This 2-d host tensor as the B of products (read transposed when `transposed`, as matmulT's transposeB), its panels packed once and kept: a product with it on the CPU packs nothing. A weight is packed once, at load; a reshape drops the panels. Effects: fail. ### `Tensor.matmul` ```gizlang function Tensor.matmul(self: Tensor, y: Tensor): Tensor ``` self (..., m, k) times y (k, n): leading axes are rows too. Effects: fail, read, write, console, io. ### `Tensor.total` ```gizlang function Tensor.total(self: Tensor, policy: Reduction): Tensor ``` The sum (Tensor.sum) as a tensor of shape [] where this one is: on a GPU it stays there, with no wait. Effects: fail, read, write, console, io. ### `Tensor.average` ```gizlang function Tensor.average(self: Tensor, policy: Reduction): Tensor ``` The mean (Tensor.mean) as a tensor of shape [] where this one is, as Tensor.total. Effects: fail, read, write, console, io. ### `Tensor.sumTo` ```gizlang function Tensor.sumTo(self: Tensor, shape: Vec): Tensor ``` Summed down to `shape` over the axes broadcasting stretched (the adjoint of broadcasting; a reduction over any axes that keeps them as 1). Over the last axis only, each row adds as `sum(Reduction.strict())` adds a part (f64 lanes); otherwise each element adds its sources in index order from 0 in f32. Effects: fail, read, write, console, io. ### `Tensor.maxTo` ```gizlang function Tensor.maxTo(self: Tensor, shape: Vec): Tensor ``` The largest element over the axes that `shape` keeps as 1 (NaN is ignored, as by `max`; -Infinity where every element is NaN or none is left). Effects: fail, read, write, console, io. ### `Tensor.take` ```gizlang function Tensor.take(self: Tensor, ids: Vec): Tensor ``` Rows `ids` of the tensor (its first axis): an embedding lookup. Effects: fail, read, write, console, io. ### `Tensor.addRows` ```gizlang function Tensor.addRows(self: Tensor, ids: Vec, rows: i64): Tensor ``` The adjoint of take: a (rows, ...) tensor whose row r adds the rows i of this one with ids[i] = r, in increasing i (on a GPU each output element adds its own list, so no two threads write one place and the order holds). Effects: fail, read, write, console, io. ### `Tensor.slice` ```gizlang function Tensor.slice(self: Tensor, axis: i64, start: i64, end: i64): Tensor ``` Elements start..end of `axis`. Effects: fail, read, write, console, io. ### `Tensor.permute` ```gizlang function Tensor.permute(self: Tensor, axes: Vec): Tensor ``` The axes reordered: output axis i is input axis axes[i]. Effects: fail, read, write, console, io. ### `Tensor.rows` ```gizlang function Tensor.rows(self: Tensor, r0: i64, r1: i64): Tensor ``` Rows [r0, r1) of the tensor ([rows, ...] on a GPU) on the same memory, with no copy (Tensor.slice copies). Effects: fail. ### `Tensor.address` ```gizlang function Tensor.address(self: Tensor): u64 ``` The device address of the tensor's first element (a kernel's buffer argument). Effects: fail. ### `Stored.upload` ```gizlang function Stored.upload(self: Stored, sys: System, gpu: Gpu): Buffer throws string ``` The stored tensor's raw bytes in a new buffer on `gpu`, read from its file straight to the device (Gpu.uploadFile). Effects: fail, read, write, io.