gizml:ml
one interface over the estimators of gizlang:{linalg,trees,svm,cluster}, with the algorithms and default hyperparameters of scikit-learn, xgboost, LightGBM and CatBoost: fit, predict, predict_proba, the metrics, and the model bytes. The linear models, the optimizers (L-BFGS, SGD) and the metrics are here. Matrices are row-major f64.
import * as Ml from "gizml:ml";Types
Data
type Data = { rows: i64; features: i64; x: Vec<f64>; targets: i64; y: Vec<f64>; }A training set: rows × features in x, rows × targets in y (labels for classification, none for clustering).
rows:i64features:i64x:Vec<f64>targets:i64y:Vec<f64>
Estimator
type Estimator = | { kind: "linear"; w: Vec<f64>; outputs: i64; } | { kind: "trees"; ensemble: Ensemble; } | { kind: "svm"; svm: Svm; } | { kind: "centers"; centers: Centers; } | { kind: "project"; mean: Vec<f64>; components: Vec<f64>; outputs: i64; }linear: outputs × (features + 1) weights, the intercept last. project: (x - mean) · each component.
lineartreessvmcentersproject
Model
type Model = { task: string; algorithm: string; features: i64; targets: i64; classes: Vec<f64>; link: string; parts: Vec<Estimator>; explained: Vec<f64>; }The outputs of a row are those of each part in order. link (classification): logistic (one raw score), softmax (a raw score per class), proba (probabilities), samme (AdaBoost votes) or scores (no probabilities: the sign of one score, else the largest). classes: the label of each class index. explained: PCA's explained variance ratios.
task:stringalgorithm:stringfeatures:i64targets:i64classes:Vec<f64>link:stringparts:Vec<M25.Estimator>explained:Vec<f64>
Sgd
type Sgd = { loss: string; rate: string; alpha: f64; l2: bool; eta0: f64; power: f64; cost: f64; epsilon: f64; maxIter: i64; tol: f64; patience: i64; }Stochastic gradient descent as scikit-learn's _plain_sgd. loss: squared_error, hinge, perceptron, epsilon_insensitive; rate: optimal, invscaling, constant, pa1 (passive-aggressive). y is ±1 for classification.
loss:stringrate:stringalpha:f64l2:booleta0:f64power:f64cost:f64epsilon:f64maxIter:i64tol:f64patience:i64
SgdState
type SgdState = { w: Vec<f64>; b: f64; scale: f64; t: f64; init: f64; }One SGD model as it learns: the weights (times scale), the intercept, the step count and the start of the optimal
rate.
w:Vec<f64>b:f64scale:f64t:f64init:f64
Partial
type Partial = { model: Model; sgd: Sgd; states: Vec<SgdState>; batch: MiniBatch; rng: Random; }A model that learns one batch of rows at a time, as scikit-learn's partial_fit: one pass of SGD over each batch, or one step of mini-batch k-means. A classifier takes its classes before the first batch.
model:M25.Modelsgd:M25.Sgdstates:Vec<M25.SgdState>batch:M19.MiniBatchrng:gizlang_random.Random
ClassMetrics
type ClassMetrics = { f1: f64; f1Micro: f64; precision: f64; recall: f64; accuracy: f64; mcc: f64; }The confusion metrics of PostgresML (src/metrics.rs): f1 macro, precision, recall and accuracy, f32; mcc as linfa's multiclass formula. y and yhat are labels rounded to integers.
f1:f64f1Micro:f64precision:f64recall:f64accuracy:f64mcc:f64
Fused
type Fused = { id: string; score: f64; }An item of a fused ranking and its score.
id:stringscore:f64
Functions
number
function number(h: Json, key: string, d: f64): f64The number under key in the hyperparameters h, else d.
Effects: fail.
integer
function integer(h: Json, key: string, d: i64): i64The integer under key in the hyperparameters h, else d.
Effects: fail.
text
function text(h: Json, key: string, d: string): stringThe text under key in the hyperparameters h, else d.
Effects: fail.
flag
function flag(h: Json, key: string, d: bool): boolThe boolean hyperparameter KEY of H, or D when H does not have it.
Effects: fail.
ridge
function ridge(x: Vec<f64>, n: i64, d: i64, y: Vec<f64>, t: i64, alpha: f64, center: bool): Vec<f64>Least squares (ridge when alpha > 0) of t targets: the minimum-norm solution, as LAPACK's lstsq gives it.
Effects: fail.
fit
function fit(task: string, algorithm: string, h: Json, data: Data, seed0: u64): Model throws stringFits algorithm for task with the hyperparameters h (scikit-learn's keyword arguments).
Effects: fail.
streams
function streams(task: string, algorithm: string): boolWhether algorithm of task learns one batch at a time (Partial): the SGD models and mini-batch k-means. The
others need all their rows at once.
partial
function partial(task: string, algorithm: string, h: Json, features: i64, targets: i64, classes: Vec<f64>, seed0: u64): Partial throws stringA model of TASK and ALGORITHM that learns batch by batch (partial_fit), with hyperparameters H.
Effects: fail.
resume
function resume(m: Model, h: Json, seed: u64): Partial throws stringA Partial that goes on from m, an SGD model (scikit-learn's warm_start: its weights, the step count again 1).
Effects: fail.
sum32
function sum32(v: Vec<f64>): f64The f32 sum of ndarray (linfa's): eight running sums over blocks of eight, joined pairwise, then the rest in order.
Effects: fail.
r2
function r2(y: Vec<f64>, yhat: Vec<f64>): f64R², mean absolute and mean squared error in f32, as linfa computes them for PostgresML over the flattened targets.
Effects: fail.
meanAbsoluteError
function meanAbsoluteError(y: Vec<f64>, yhat: Vec<f64>): f64The mean of the absolute differences between y and yhat.
Effects: fail.
meanSquaredError
function meanSquaredError(y: Vec<f64>, yhat: Vec<f64>): f64The mean of the squared differences between y and yhat.
Effects: fail.
classMetrics
function classMetrics(y: Vec<f64>, yhat: Vec<f64>, classes: i64): ClassMetrics throws stringThe confusion metrics of predictions YHAT for labels Y of CLASSES classes. It fails when the lengths differ.
Effects: fail.
sklearnScores
function sklearnScores(y: Vec<f64>, yhat: Vec<f64>): ClassMetrics throws stringscikit-learn's classification scores of PostgresML's sklearn_classification_metrics, in f32: f1, precision and recall of class 1 (average "binary") for at most two labels, else their mean over the labels (average "macro").
Effects: fail.
sklearnF1
function sklearnF1(y: Vec<f64>, yhat: Vec<f64>): f64 throws stringscikit-learn's f1_score with average "binary": an error for more than two labels.
Effects: fail.
sklearnRocAuc
function sklearnRocAuc(y: Vec<f64>, score: Vec<f64>): f64 throws stringscikit-learn's roc_auc_score of two classes, the larger one positive: the Mann-Whitney statistic, ties as halves.
Effects: fail.
sklearnRegression
function sklearnRegression(y: Vec<f64>, yhat: Vec<f64>): Vec<f64>scikit-learn's r2_score, mean_squared_error and mean_absolute_error, in f32: [r2, mse, mae]. A constant truth gives an r2 of 1 for an exact prediction, else 0.
Effects: fail.
rocAuc
function rocAuc(y: Vec<f64>, score: Vec<f64>): f64linfa's area under the ROC curve (PostgresML's roc_auc): the scores of 0 or more in ascending order, a point at each new score, the (tp, fp) rates of the points integrated by the trapezoid rule over tp, in f32.
Effects: fail.
logLoss
function logLoss(y: Vec<f64>, p: Vec<f64>): f64linfa's log loss: each probability clamped to [ε, 1 − ε] of f32, in f32.
Effects: fail.
silhouette
function silhouette(x: Vec<f64>, n: i64, d: i64, labels: Vec<f64>): f64 throws stringThe mean silhouette coefficient (Euclidean); 2 to n - 1 distinct labels.
Effects: fail.
rrf
function rrf(ranks: Vec<f64>, k: f64): f64The reciprocal rank fusion score of one item: the sum of 1 / (k + rank) over the lists that ranked it, each rank from 1; a rank under 1 is a list that did not rank it. It divides first, so a large k or rank does not overflow.
Effects: fail.
fuse
function fuse(lists: Vec<Vec<string>>, weights: Vec<f64>, k: f64): Vec<Fused>The fusion of ranked lists of ids (text; the best first), as hybrid search joins a text ranking and a vector ranking:
each id's weighted reciprocal rank sum, weights[i] times 1 / (k + rank) in list i (an empty weights: all 1). The
result is by score, the higher first; equal scores keep the order in which the ids first come. A repeated id in a
list counts at its first rank.
Effects: fail.
recallAt
function recallAt(ranked: Vec<string>, relevant: Vec<string>, k: i64): f64Recall at k: the share of the relevant ids in the first k ranked (k 0: all); 0 with no relevant id.
Effects: fail.
precisionAt
function precisionAt(ranked: Vec<string>, relevant: Vec<string>, k: i64): f64Precision at k: the share of the first k places (k 0: all ranked) that hold a relevant id.
Effects: fail.
reciprocalRank
function reciprocalRank(ranked: Vec<string>, relevant: Vec<string>, k: i64): f64The reciprocal rank: 1 / the place of the first relevant id in the first k (k 0: all); 0 when none is there. Its mean over queries is MRR.
Effects: fail.
averagePrecision
function averagePrecision(ranked: Vec<string>, relevant: Vec<string>, k: i64): f64The average precision in the first k (k 0: all), as trec_eval's map: the precision at each place of a relevant id, summed and divided by the count of relevant ids.
Effects: fail.
ndcgAt
function ndcgAt(ranked: Vec<string>, relevant: Vec<string>, gains: Vec<f64>, k: i64): f64The normalized discounted cumulative gain at k (k 0: all), as trec_eval's ndcg_cut: the sum of gain / log2(place
- over the first k ranked, divided by that of the ideal order.
gains[i]is the graded relevance ofrelevant[i]; with no gains each relevant id has gain 1. 0 when the ideal gain is 0.
- over the first k ranked, divided by that of the ideal order.
Effects: fail.
encode
function encode(m: Model): bytesThe model as bytes: "GZML", the format version, then the fields in order.
Effects: fail.
decode
function decode(data: bytes): Model throws stringThe model of encode; each later format version keeps reading the earlier ones.
Effects: fail.
Methods
Partial.feed
function Partial.feed(self: mut Partial, x: Vec<f64>, y: Vec<f64>, n: i64): bool throws stringLearns from the n rows of x (row-major) and y (targets a row; a class value for a classifier).
Effects: fail.
Partial.result
function Partial.result(self: Partial): ModelThe model learned so far.
Effects: fail.
Model.outputs
function Model.outputs(self: Model, row: Vec<f64>): Vec<f64>The raw outputs of a row: the concatenation of the outputs of each part.
Effects: fail.
Model.predict
function Model.predict(self: Model, row: Vec<f64>): Vec<f64>The prediction of a row: each target (regression), the label (classification), the cluster (clustering), the components (decomposition).
Effects: fail.
Model.predictRows
function Model.predictRows(self: Model, x: Vec<f64>, n: i64): Vec<f64>The predictions of n rows of x (row-major, features wide), one row after another, on all threads.
Effects: fail.
Model.probaRows
function Model.probaRows(self: Model, x: Vec<f64>, n: i64): Vec<f64> throws stringThe class probabilities of n rows of x, one row after another, on all threads.
Effects: fail.
Model.proba
function Model.proba(self: Model, row: Vec<f64>): Vec<f64> throws stringThe probability of each class, in the order of the classes.
Effects: fail.
Constants
ML_FORMAT
const ML_FORMAT = 1The version of the model file format.