gizai:llm/encoders

BERT-family encoders (BERT, RoBERTa, XLM-RoBERTa) for embeddings and ranking.

import * as Encoders from "gizai:llm/encoders";

Types

BertConfig

type BertConfig = { kind: string; layers: i64; heads: i64; dim: i64; eps: f64; act: string; positions: i64; offset: i64; }

A BERT-family encoder as a Hugging Face config.json gives it: BERT, RoBERTa and XLM-RoBERTa have one layout and differ in where positions start (offset: 0 for BERT, pad_token_id + 1 for RoBERTa). act is hidden_act: "gelu" (the erf form), "relu" or a tanh form ("gelu_new", "gelu_pytorch_tanh").

• kind: string
• layers: i64
• heads: i64
• dim: i64
• eps: f64
• act: string
• positions: i64
• offset: i64

ModelFiles

type ModelFiles = (path: string) => Result<Option<bytes>, string>

Reads the file at a path of a model's repository: none when the model has no such file.

Encoder

type Encoder = { config: BertConfig; weights: Map<string, Tensor>; tokenizer: Tokenizer; pooling: string; normalize: bool; limit: i64; }

A sentence encoder as sentence-transformers 3 builds it from a model's files: an encoder, then the pooling of its token states ("mean", "cls", "max", "mean_sqrt_len" or "last"), then L2 normalization when the model has that module.

• config: M52.BertConfig
• weights: _gizlang_sdk_collections.Map<string, _gizlang_lib_ai__tensor__storage.Tensor>
• tokenizer: M27.Tokenizer
• pooling: string
• normalize: bool
• limit: i64

CrossEncoder

type CrossEncoder = { config: BertConfig; weights: Map<string, Tensor>; tokenizer: Tokenizer; head: string; labels: i64; activation: string; limit: i64; }

A cross-encoder as sentence-transformers' CrossEncoder builds it: a sequence classification model that reads a pair of texts (a query and a document) and gives the scores of its labels. head is "bert" (BertForSequence- Classification: the pooler's tanh on the first token, then the classifier) or "roberta" (RobertaFor- and XLMRobertaForSequenceClassification: classifier.dense, tanh, classifier.out_proj). activation is the function on the scores: "sigmoid" (the default with one label), "identity" (the default with more) or "tanh".

• config: M52.BertConfig
• weights: _gizlang_sdk_collections.Map<string, _gizlang_lib_ai__tensor__storage.Tensor>
• tokenizer: M27.Tokenizer
• head: string
• labels: i64
• activation: string
• limit: i64

Ranked

type Ranked = { corpus: i64; score: f64; }

A document of a ranking: its index in the documents and its score.

• corpus: i64
• score: f64

Functions

bertWeights

function bertWeights(tensors: Vec<Named>): Map<string, Tensor>

The tensors of a BertModel or RobertaModel checkpoint by their names, without the "bert." or "roberta." prefix of a task model, and with LayerNorm.gamma and beta (older checkpoints) as weight and bias.

Effects: fail.

weight

function weight(w: Map<string, Tensor>, name: string): Tensor throws string

The tensor name of the weights w, or an error naming it.

Effects: fail.

bert

function bert(c: BertConfig, w: Map<string, Tensor>, ids: Vec<i64>, types: Vec<i64>, sequences: i64): Tensor throws string

The last hidden states (sequences tokens, dim) of the encoder for sequences sequences of one length: their token ids and token types one sequence after another, every token attended (no padding). The weights may be on a GPU (Encoder.on); the states are where the weights are.

Effects: fail, read, write, console, io.

heads

function heads(x: Tensor, n: i64, t: i64, h: i64): Tensor

The rows (n t, h hd) of n sequences as (n h, t, hd): each head of each sequence attends on its own.

Effects: fail, read, write, console, io.

groups

function groups(encs: Vec<Tok.Encoding>, batch: i64): Vec<Vec<i64>>

Groups of the encodings with one token count, at most batch in a group, so a group goes through the encoder together with no padding: each group is the indexes of its encodings.

Effects: fail.

encoderOf

function encoderOf(files: ModelFiles): Encoder throws string

The encoder of a model by the files it reads from its repository: config.json, tokenizer.json, model.safetensors and, for a sentence-transformers model, modules.json, sentence_bert_config.json and the pooling module's config.json. Without modules.json the tokens are mean-pooled and not normalized, as sentence-transformers does.

Effects: fail, read, write, console, io.

crossEncoderOf

function crossEncoderOf(files: ModelFiles): CrossEncoder throws string

The cross-encoder of a model by its repository's files: config.json (its sentence_transformers.activation_fn, when present, names the activation), tokenizer.json and model.safetensors.

Effects: fail, read, write, console, io.

Methods

Encoder.on

function Encoder.on(self: Encoder, gpu: Gpu): Encoder

The encoder with its weights on gpu: it encodes there and gives the same embeddings to f32 precision.

Effects: fail, read, write, console, io.

Encoder.embed

function Encoder.embed(self: Encoder, text: string, normalize: bool): Vec<f64> throws string

The embedding of a text (Encoder.embedBatch of one).

Effects: fail, read, write, console, io.

Encoder.embedBatch

function Encoder.embedBatch(self: Encoder, texts: Vec<string>, normalize: bool, batch: i64): Vec<Vec<f64>> throws string

The embeddings of texts: their token states pooled and, when the model says so or normalize, of length 1. The texts of one token count go through the encoder together, at most batch of them at a time, so no token is padding.

Effects: fail, read, write, console, io.

CrossEncoder.on

function CrossEncoder.on(self: CrossEncoder, gpu: Gpu): CrossEncoder

The cross-encoder with its weights on gpu.

Effects: fail, read, write, console, io.

CrossEncoder.scores

function CrossEncoder.scores(self: CrossEncoder, query: string, documents: Vec<string>, batch: i64): Vec<Vec<f64>> throws string

The scores of the labels of each pair (query, documents[i]), after the activation: CrossEncoder.predict. The pairs of one token count go through the encoder together, at most batch at a time.

Effects: fail, read, write, console, io.

CrossEncoder.rank

function CrossEncoder.rank(self: CrossEncoder, query: string, documents: Vec<string>, top: i64, batch: i64): Vec<Ranked> throws string

The documents by the score of (query, document), the highest first, at most top of them (0: all): CrossEncoder. rank. Equal scores keep the documents' order. The model must have one label.

Effects: fail, read, write, console, io.