GizStack
View .mdOpen in ChatGPTOpen in Claude

gizml:ml

one interface over the estimators of gizlang:{linalg,trees,svm,cluster}, with the algorithms and default hyperparameters of scikit-learn, xgboost, LightGBM and CatBoost: fit, predict, predict_proba, the metrics, and the model bytes. The linear models, the optimizers (L-BFGS, SGD) and the metrics are here. Matrices are row-major f64.

gizlang
import * as Ml from "gizml:ml";

Types

Data

gizlang
type Data = { rows: i64; features: i64; x: Vec<f64>; targets: i64; y: Vec<f64>; }

A training set: rows × features in x, rows × targets in y (labels for classification, none for clustering).

  • rows: i64
  • features: i64
  • x: Vec<f64>
  • targets: i64
  • y: Vec<f64>

Estimator

gizlang
type Estimator = | { kind: "linear"; w: Vec<f64>; outputs: i64; } | { kind: "trees"; ensemble: Ensemble; } | { kind: "svm"; svm: Svm; } | { kind: "centers"; centers: Centers; } | { kind: "project"; mean: Vec<f64>; components: Vec<f64>; outputs: i64; }

linear: outputs × (features + 1) weights, the intercept last. project: (x - mean) · each component.

  • linear
  • trees
  • svm
  • centers
  • project

Model

gizlang
type Model = { task: string; algorithm: string; features: i64; targets: i64; classes: Vec<f64>; link: string; parts: Vec<Estimator>; explained: Vec<f64>; }

The outputs of a row are those of each part in order. link (classification): logistic (one raw score), softmax (a raw score per class), proba (probabilities), samme (AdaBoost votes) or scores (no probabilities: the sign of one score, else the largest). classes: the label of each class index. explained: PCA's explained variance ratios.

  • task: string
  • algorithm: string
  • features: i64
  • targets: i64
  • classes: Vec<f64>
  • link: string
  • parts: Vec<M25.Estimator>
  • explained: Vec<f64>

Sgd

gizlang
type Sgd = { loss: string; rate: string; alpha: f64; l2: bool; eta0: f64; power: f64; cost: f64; epsilon: f64; maxIter: i64; tol: f64; patience: i64; }

Stochastic gradient descent as scikit-learn's _plain_sgd. loss: squared_error, hinge, perceptron, epsilon_insensitive; rate: optimal, invscaling, constant, pa1 (passive-aggressive). y is ±1 for classification.

  • loss: string
  • rate: string
  • alpha: f64
  • l2: bool
  • eta0: f64
  • power: f64
  • cost: f64
  • epsilon: f64
  • maxIter: i64
  • tol: f64
  • patience: i64

SgdState

gizlang
type SgdState = { w: Vec<f64>; b: f64; scale: f64; t: f64; init: f64; }

One SGD model as it learns: the weights (times scale), the intercept, the step count and the start of the optimal rate.

  • w: Vec<f64>
  • b: f64
  • scale: f64
  • t: f64
  • init: f64

Partial

gizlang
type Partial = { model: Model; sgd: Sgd; states: Vec<SgdState>; batch: MiniBatch; rng: Random; }

A model that learns one batch of rows at a time, as scikit-learn's partial_fit: one pass of SGD over each batch, or one step of mini-batch k-means. A classifier takes its classes before the first batch.

  • model: M25.Model
  • sgd: M25.Sgd
  • states: Vec<M25.SgdState>
  • batch: M19.MiniBatch
  • rng: gizlang_random.Random

ClassMetrics

gizlang
type ClassMetrics = { f1: f64; f1Micro: f64; precision: f64; recall: f64; accuracy: f64; mcc: f64; }

The confusion metrics of PostgresML (src/metrics.rs): f1 macro, precision, recall and accuracy, f32; mcc as linfa's multiclass formula. y and yhat are labels rounded to integers.

  • f1: f64
  • f1Micro: f64
  • precision: f64
  • recall: f64
  • accuracy: f64
  • mcc: f64

Fused

gizlang
type Fused = { id: string; score: f64; }

An item of a fused ranking and its score.

  • id: string
  • score: f64

Functions

number

gizlang
function number(h: Json, key: string, d: f64): f64

The number under key in the hyperparameters h, else d.

Effects: fail.

integer

gizlang
function integer(h: Json, key: string, d: i64): i64

The integer under key in the hyperparameters h, else d.

Effects: fail.

text

gizlang
function text(h: Json, key: string, d: string): string

The text under key in the hyperparameters h, else d.

Effects: fail.

flag

gizlang
function flag(h: Json, key: string, d: bool): bool

The boolean hyperparameter KEY of H, or D when H does not have it.

Effects: fail.

ridge

gizlang
function ridge(x: Vec<f64>, n: i64, d: i64, y: Vec<f64>, t: i64, alpha: f64, center: bool): Vec<f64>

Least squares (ridge when alpha > 0) of t targets: the minimum-norm solution, as LAPACK's lstsq gives it.

Effects: fail.

fit

gizlang
function fit(task: string, algorithm: string, h: Json, data: Data, seed0: u64): Model throws string

Fits algorithm for task with the hyperparameters h (scikit-learn's keyword arguments).

Effects: fail.

streams

gizlang
function streams(task: string, algorithm: string): bool

Whether algorithm of task learns one batch at a time (Partial): the SGD models and mini-batch k-means. The others need all their rows at once.

partial

gizlang
function partial(task: string, algorithm: string, h: Json, features: i64, targets: i64, classes: Vec<f64>, seed0: u64): Partial throws string

A model of TASK and ALGORITHM that learns batch by batch (partial_fit), with hyperparameters H.

Effects: fail.

resume

gizlang
function resume(m: Model, h: Json, seed: u64): Partial throws string

A Partial that goes on from m, an SGD model (scikit-learn's warm_start: its weights, the step count again 1).

Effects: fail.

sum32

gizlang
function sum32(v: Vec<f64>): f64

The f32 sum of ndarray (linfa's): eight running sums over blocks of eight, joined pairwise, then the rest in order.

Effects: fail.

r2

gizlang
function r2(y: Vec<f64>, yhat: Vec<f64>): f64

R², mean absolute and mean squared error in f32, as linfa computes them for PostgresML over the flattened targets.

Effects: fail.

meanAbsoluteError

gizlang
function meanAbsoluteError(y: Vec<f64>, yhat: Vec<f64>): f64

The mean of the absolute differences between y and yhat.

Effects: fail.

meanSquaredError

gizlang
function meanSquaredError(y: Vec<f64>, yhat: Vec<f64>): f64

The mean of the squared differences between y and yhat.

Effects: fail.

classMetrics

gizlang
function classMetrics(y: Vec<f64>, yhat: Vec<f64>, classes: i64): ClassMetrics throws string

The confusion metrics of predictions YHAT for labels Y of CLASSES classes. It fails when the lengths differ.

Effects: fail.

sklearnScores

gizlang
function sklearnScores(y: Vec<f64>, yhat: Vec<f64>): ClassMetrics throws string

scikit-learn's classification scores of PostgresML's sklearn_classification_metrics, in f32: f1, precision and recall of class 1 (average "binary") for at most two labels, else their mean over the labels (average "macro").

Effects: fail.

sklearnF1

gizlang
function sklearnF1(y: Vec<f64>, yhat: Vec<f64>): f64 throws string

scikit-learn's f1_score with average "binary": an error for more than two labels.

Effects: fail.

sklearnRocAuc

gizlang
function sklearnRocAuc(y: Vec<f64>, score: Vec<f64>): f64 throws string

scikit-learn's roc_auc_score of two classes, the larger one positive: the Mann-Whitney statistic, ties as halves.

Effects: fail.

sklearnRegression

gizlang
function sklearnRegression(y: Vec<f64>, yhat: Vec<f64>): Vec<f64>

scikit-learn's r2_score, mean_squared_error and mean_absolute_error, in f32: [r2, mse, mae]. A constant truth gives an r2 of 1 for an exact prediction, else 0.

Effects: fail.

rocAuc

gizlang
function rocAuc(y: Vec<f64>, score: Vec<f64>): f64

linfa's area under the ROC curve (PostgresML's roc_auc): the scores of 0 or more in ascending order, a point at each new score, the (tp, fp) rates of the points integrated by the trapezoid rule over tp, in f32.

Effects: fail.

logLoss

gizlang
function logLoss(y: Vec<f64>, p: Vec<f64>): f64

linfa's log loss: each probability clamped to [ε, 1 − ε] of f32, in f32.

Effects: fail.

silhouette

gizlang
function silhouette(x: Vec<f64>, n: i64, d: i64, labels: Vec<f64>): f64 throws string

The mean silhouette coefficient (Euclidean); 2 to n - 1 distinct labels.

Effects: fail.

rrf

gizlang
function rrf(ranks: Vec<f64>, k: f64): f64

The reciprocal rank fusion score of one item: the sum of 1 / (k + rank) over the lists that ranked it, each rank from 1; a rank under 1 is a list that did not rank it. It divides first, so a large k or rank does not overflow.

Effects: fail.

fuse

gizlang
function fuse(lists: Vec<Vec<string>>, weights: Vec<f64>, k: f64): Vec<Fused>

The fusion of ranked lists of ids (text; the best first), as hybrid search joins a text ranking and a vector ranking: each id's weighted reciprocal rank sum, weights[i] times 1 / (k + rank) in list i (an empty weights: all 1). The result is by score, the higher first; equal scores keep the order in which the ids first come. A repeated id in a list counts at its first rank.

Effects: fail.

recallAt

gizlang
function recallAt(ranked: Vec<string>, relevant: Vec<string>, k: i64): f64

Recall at k: the share of the relevant ids in the first k ranked (k 0: all); 0 with no relevant id.

Effects: fail.

precisionAt

gizlang
function precisionAt(ranked: Vec<string>, relevant: Vec<string>, k: i64): f64

Precision at k: the share of the first k places (k 0: all ranked) that hold a relevant id.

Effects: fail.

reciprocalRank

gizlang
function reciprocalRank(ranked: Vec<string>, relevant: Vec<string>, k: i64): f64

The reciprocal rank: 1 / the place of the first relevant id in the first k (k 0: all); 0 when none is there. Its mean over queries is MRR.

Effects: fail.

averagePrecision

gizlang
function averagePrecision(ranked: Vec<string>, relevant: Vec<string>, k: i64): f64

The average precision in the first k (k 0: all), as trec_eval's map: the precision at each place of a relevant id, summed and divided by the count of relevant ids.

Effects: fail.

ndcgAt

gizlang
function ndcgAt(ranked: Vec<string>, relevant: Vec<string>, gains: Vec<f64>, k: i64): f64

The normalized discounted cumulative gain at k (k 0: all), as trec_eval's ndcg_cut: the sum of gain / log2(place

    1. over the first k ranked, divided by that of the ideal order. gains[i] is the graded relevance of relevant[i]; with no gains each relevant id has gain 1. 0 when the ideal gain is 0.

Effects: fail.

encode

gizlang
function encode(m: Model): bytes

The model as bytes: "GZML", the format version, then the fields in order.

Effects: fail.

decode

gizlang
function decode(data: bytes): Model throws string

The model of encode; each later format version keeps reading the earlier ones.

Effects: fail.

Methods

Partial.feed

gizlang
function Partial.feed(self: mut Partial, x: Vec<f64>, y: Vec<f64>, n: i64): bool throws string

Learns from the n rows of x (row-major) and y (targets a row; a class value for a classifier).

Effects: fail.

Partial.result

gizlang
function Partial.result(self: Partial): Model

The model learned so far.

Effects: fail.

Model.outputs

gizlang
function Model.outputs(self: Model, row: Vec<f64>): Vec<f64>

The raw outputs of a row: the concatenation of the outputs of each part.

Effects: fail.

Model.predict

gizlang
function Model.predict(self: Model, row: Vec<f64>): Vec<f64>

The prediction of a row: each target (regression), the label (classification), the cluster (clustering), the components (decomposition).

Effects: fail.

Model.predictRows

gizlang
function Model.predictRows(self: Model, x: Vec<f64>, n: i64): Vec<f64>

The predictions of n rows of x (row-major, features wide), one row after another, on all threads.

Effects: fail.

Model.probaRows

gizlang
function Model.probaRows(self: Model, x: Vec<f64>, n: i64): Vec<f64> throws string

The class probabilities of n rows of x, one row after another, on all threads.

Effects: fail.

Model.proba

gizlang
function Model.proba(self: Model, row: Vec<f64>): Vec<f64> throws string

The probability of each class, in the order of the classes.

Effects: fail.

Constants

ML_FORMAT

gizlang
const ML_FORMAT = 1

The version of the model file format.