# gizml:ml one interface over the estimators of gizlang:{linalg,trees,svm,cluster}, with the algorithms and default hyperparameters of scikit-learn, xgboost, LightGBM and CatBoost: fit, predict, predict_proba, the metrics, and the model bytes. The linear models, the optimizers (L-BFGS, SGD) and the metrics are here. Matrices are row-major f64. ```gizlang import * as Ml from "gizml:ml"; ``` ## Types ### `Data` ```gizlang type Data = { rows: i64; features: i64; x: Vec; targets: i64; y: Vec; } ``` A training set: rows × features in x, rows × targets in y (labels for classification, none for clustering). - `rows`: `i64` - `features`: `i64` - `x`: `Vec` - `targets`: `i64` - `y`: `Vec` ### `Estimator` ```gizlang type Estimator = | { kind: "linear"; w: Vec; outputs: i64; } | { kind: "trees"; ensemble: Ensemble; } | { kind: "svm"; svm: Svm; } | { kind: "centers"; centers: Centers; } | { kind: "project"; mean: Vec; components: Vec; outputs: i64; } ``` linear: outputs × (features + 1) weights, the intercept last. project: (x - mean) · each component. - `linear` - `trees` - `svm` - `centers` - `project` ### `Model` ```gizlang type Model = { task: string; algorithm: string; features: i64; targets: i64; classes: Vec; link: string; parts: Vec; explained: Vec; } ``` The outputs of a row are those of each part in order. link (classification): logistic (one raw score), softmax (a raw score per class), proba (probabilities), samme (AdaBoost votes) or scores (no probabilities: the sign of one score, else the largest). classes: the label of each class index. explained: PCA's explained variance ratios. - `task`: `string` - `algorithm`: `string` - `features`: `i64` - `targets`: `i64` - `classes`: `Vec` - `link`: `string` - `parts`: `Vec` - `explained`: `Vec` ### `Sgd` ```gizlang type Sgd = { loss: string; rate: string; alpha: f64; l2: bool; eta0: f64; power: f64; cost: f64; epsilon: f64; maxIter: i64; tol: f64; patience: i64; } ``` Stochastic gradient descent as scikit-learn's _plain_sgd. loss: squared_error, hinge, perceptron, epsilon_insensitive; rate: optimal, invscaling, constant, pa1 (passive-aggressive). y is ±1 for classification. - `loss`: `string` - `rate`: `string` - `alpha`: `f64` - `l2`: `bool` - `eta0`: `f64` - `power`: `f64` - `cost`: `f64` - `epsilon`: `f64` - `maxIter`: `i64` - `tol`: `f64` - `patience`: `i64` ### `SgdState` ```gizlang type SgdState = { w: Vec; b: f64; scale: f64; t: f64; init: f64; } ``` One SGD model as it learns: the weights (times `scale`), the intercept, the step count and the start of the optimal rate. - `w`: `Vec` - `b`: `f64` - `scale`: `f64` - `t`: `f64` - `init`: `f64` ### `Partial` ```gizlang type Partial = { model: Model; sgd: Sgd; states: Vec; batch: MiniBatch; rng: Random; } ``` A model that learns one batch of rows at a time, as scikit-learn's partial_fit: one pass of SGD over each batch, or one step of mini-batch k-means. A classifier takes its classes before the first batch. - `model`: `M25.Model` - `sgd`: `M25.Sgd` - `states`: `Vec` - `batch`: `M19.MiniBatch` - `rng`: `gizlang_random.Random` ### `ClassMetrics` ```gizlang type ClassMetrics = { f1: f64; f1Micro: f64; precision: f64; recall: f64; accuracy: f64; mcc: f64; } ``` The confusion metrics of PostgresML (src/metrics.rs): f1 macro, precision, recall and accuracy, f32; mcc as linfa's multiclass formula. y and yhat are labels rounded to integers. - `f1`: `f64` - `f1Micro`: `f64` - `precision`: `f64` - `recall`: `f64` - `accuracy`: `f64` - `mcc`: `f64` ### `Fused` ```gizlang type Fused = { id: string; score: f64; } ``` An item of a fused ranking and its score. - `id`: `string` - `score`: `f64` ## Functions ### `number` ```gizlang function number(h: Json, key: string, d: f64): f64 ``` The number under `key` in the hyperparameters `h`, else `d`. Effects: fail. ### `integer` ```gizlang function integer(h: Json, key: string, d: i64): i64 ``` The integer under `key` in the hyperparameters `h`, else `d`. Effects: fail. ### `text` ```gizlang function text(h: Json, key: string, d: string): string ``` The text under `key` in the hyperparameters `h`, else `d`. Effects: fail. ### `flag` ```gizlang function flag(h: Json, key: string, d: bool): bool ``` The boolean hyperparameter KEY of H, or D when H does not have it. Effects: fail. ### `ridge` ```gizlang function ridge(x: Vec, n: i64, d: i64, y: Vec, t: i64, alpha: f64, center: bool): Vec ``` Least squares (ridge when alpha > 0) of t targets: the minimum-norm solution, as LAPACK's lstsq gives it. Effects: fail. ### `fit` ```gizlang function fit(task: string, algorithm: string, h: Json, data: Data, seed0: u64): Model throws string ``` Fits `algorithm` for `task` with the hyperparameters `h` (scikit-learn's keyword arguments). Effects: fail. ### `streams` ```gizlang function streams(task: string, algorithm: string): bool ``` Whether `algorithm` of `task` learns one batch at a time (Partial): the SGD models and mini-batch k-means. The others need all their rows at once. ### `partial` ```gizlang function partial(task: string, algorithm: string, h: Json, features: i64, targets: i64, classes: Vec, seed0: u64): Partial throws string ``` A model of TASK and ALGORITHM that learns batch by batch (partial_fit), with hyperparameters H. Effects: fail. ### `resume` ```gizlang function resume(m: Model, h: Json, seed: u64): Partial throws string ``` A Partial that goes on from `m`, an SGD model (scikit-learn's warm_start: its weights, the step count again 1). Effects: fail. ### `sum32` ```gizlang function sum32(v: Vec): f64 ``` The f32 sum of ndarray (linfa's): eight running sums over blocks of eight, joined pairwise, then the rest in order. Effects: fail. ### `r2` ```gizlang function r2(y: Vec, yhat: Vec): f64 ``` R², mean absolute and mean squared error in f32, as linfa computes them for PostgresML over the flattened targets. Effects: fail. ### `meanAbsoluteError` ```gizlang function meanAbsoluteError(y: Vec, yhat: Vec): f64 ``` The mean of the absolute differences between `y` and `yhat`. Effects: fail. ### `meanSquaredError` ```gizlang function meanSquaredError(y: Vec, yhat: Vec): f64 ``` The mean of the squared differences between `y` and `yhat`. Effects: fail. ### `classMetrics` ```gizlang function classMetrics(y: Vec, yhat: Vec, classes: i64): ClassMetrics throws string ``` The confusion metrics of predictions YHAT for labels Y of CLASSES classes. It fails when the lengths differ. Effects: fail. ### `sklearnScores` ```gizlang function sklearnScores(y: Vec, yhat: Vec): ClassMetrics throws string ``` scikit-learn's classification scores of PostgresML's sklearn_classification_metrics, in f32: f1, precision and recall of class 1 (average "binary") for at most two labels, else their mean over the labels (average "macro"). Effects: fail. ### `sklearnF1` ```gizlang function sklearnF1(y: Vec, yhat: Vec): f64 throws string ``` scikit-learn's f1_score with average "binary": an error for more than two labels. Effects: fail. ### `sklearnRocAuc` ```gizlang function sklearnRocAuc(y: Vec, score: Vec): f64 throws string ``` scikit-learn's roc_auc_score of two classes, the larger one positive: the Mann-Whitney statistic, ties as halves. Effects: fail. ### `sklearnRegression` ```gizlang function sklearnRegression(y: Vec, yhat: Vec): Vec ``` scikit-learn's r2_score, mean_squared_error and mean_absolute_error, in f32: [r2, mse, mae]. A constant truth gives an r2 of 1 for an exact prediction, else 0. Effects: fail. ### `rocAuc` ```gizlang function rocAuc(y: Vec, score: Vec): f64 ``` linfa's area under the ROC curve (PostgresML's roc_auc): the scores of 0 or more in ascending order, a point at each new score, the (tp, fp) rates of the points integrated by the trapezoid rule over tp, in f32. Effects: fail. ### `logLoss` ```gizlang function logLoss(y: Vec, p: Vec): f64 ``` linfa's log loss: each probability clamped to [ε, 1 − ε] of f32, in f32. Effects: fail. ### `silhouette` ```gizlang function silhouette(x: Vec, n: i64, d: i64, labels: Vec): f64 throws string ``` The mean silhouette coefficient (Euclidean); 2 to n - 1 distinct labels. Effects: fail. ### `rrf` ```gizlang function rrf(ranks: Vec, k: f64): f64 ``` The reciprocal rank fusion score of one item: the sum of 1 / (k + rank) over the lists that ranked it, each rank from 1; a rank under 1 is a list that did not rank it. It divides first, so a large k or rank does not overflow. Effects: fail. ### `fuse` ```gizlang function fuse(lists: Vec>, weights: Vec, k: f64): Vec ``` The fusion of ranked lists of ids (text; the best first), as hybrid search joins a text ranking and a vector ranking: each id's weighted reciprocal rank sum, weights[i] times 1 / (k + rank) in list i (an empty `weights`: all 1). The result is by score, the higher first; equal scores keep the order in which the ids first come. A repeated id in a list counts at its first rank. Effects: fail. ### `recallAt` ```gizlang function recallAt(ranked: Vec, relevant: Vec, k: i64): f64 ``` Recall at k: the share of the relevant ids in the first k ranked (k 0: all); 0 with no relevant id. Effects: fail. ### `precisionAt` ```gizlang function precisionAt(ranked: Vec, relevant: Vec, k: i64): f64 ``` Precision at k: the share of the first k places (k 0: all ranked) that hold a relevant id. Effects: fail. ### `reciprocalRank` ```gizlang function reciprocalRank(ranked: Vec, relevant: Vec, k: i64): f64 ``` The reciprocal rank: 1 / the place of the first relevant id in the first k (k 0: all); 0 when none is there. Its mean over queries is MRR. Effects: fail. ### `averagePrecision` ```gizlang function averagePrecision(ranked: Vec, relevant: Vec, k: i64): f64 ``` The average precision in the first k (k 0: all), as trec_eval's map: the precision at each place of a relevant id, summed and divided by the count of relevant ids. Effects: fail. ### `ndcgAt` ```gizlang function ndcgAt(ranked: Vec, relevant: Vec, gains: Vec, k: i64): f64 ``` The normalized discounted cumulative gain at k (k 0: all), as trec_eval's ndcg_cut: the sum of gain / log2(place + 1) over the first k ranked, divided by that of the ideal order. `gains[i]` is the graded relevance of `relevant[i]`; with no gains each relevant id has gain 1. 0 when the ideal gain is 0. Effects: fail. ### `encode` ```gizlang function encode(m: Model): bytes ``` The model as bytes: "GZML", the format version, then the fields in order. Effects: fail. ### `decode` ```gizlang function decode(data: bytes): Model throws string ``` The model of `encode`; each later format version keeps reading the earlier ones. Effects: fail. ## Methods ### `Partial.feed` ```gizlang function Partial.feed(self: mut Partial, x: Vec, y: Vec, n: i64): bool throws string ``` Learns from the `n` rows of x (row-major) and y (`targets` a row; a class value for a classifier). Effects: fail. ### `Partial.result` ```gizlang function Partial.result(self: Partial): Model ``` The model learned so far. Effects: fail. ### `Model.outputs` ```gizlang function Model.outputs(self: Model, row: Vec): Vec ``` The raw outputs of a row: the concatenation of the outputs of each part. Effects: fail. ### `Model.predict` ```gizlang function Model.predict(self: Model, row: Vec): Vec ``` The prediction of a row: each target (regression), the label (classification), the cluster (clustering), the components (decomposition). Effects: fail. ### `Model.predictRows` ```gizlang function Model.predictRows(self: Model, x: Vec, n: i64): Vec ``` The predictions of `n` rows of `x` (row-major, `features` wide), one row after another, on all threads. Effects: fail. ### `Model.probaRows` ```gizlang function Model.probaRows(self: Model, x: Vec, n: i64): Vec throws string ``` The class probabilities of `n` rows of `x`, one row after another, on all threads. Effects: fail. ### `Model.proba` ```gizlang function Model.proba(self: Model, row: Vec): Vec throws string ``` The probability of each class, in the order of the classes. Effects: fail. ## Constants ### `ML_FORMAT` ```gizlang const ML_FORMAT = 1 ``` The version of the model file format.