GizStack
View .mdOpen in ChatGPTOpen in Claude

gizdb/ml/embed

pgml.embed, pgml.rank and the local models of pgml.transform: the embeddings of sentence-transformers models, the scores of cross-encoders and the language models of chat (gizai/llm). A model is its files in the models folder (gizlang/hub); each session keeps the models (gizai/nn, gizai/llm) it loaded, as each PostgresML backend keeps its models.

gizlang
import * as Embed from "gizdb/ml/embed";

Types

Models

gizlang
type Models = { hub: Hub.Hub; encoders: Map<string, Encoder>; crossEncoders: Map<string, CrossEncoder>; decoders: Map<string, LlmState.Engine>; gpu: Option<Gpu>; probed: bool; }

Where a session finds models, the models it loaded by repository (with "@gpu" for those on the GPU), and the GPU (none: not opened yet, or there is none; probed: it was tried).

  • hub: gizlang_hub.Hub
  • encoders: _gizlang_sdk_collections.Map<string, M56.Encoder>
  • crossEncoders: _gizlang_sdk_collections.Map<string, M56.CrossEncoder>
  • decoders: _gizlang_sdk_collections.Map<string, M55.Engine>
  • gpu: _gizlang_sdk_core.Option<M7.Gpu>
  • probed: bool

Functions

models

gizlang
function models(h: Hub.Hub): Models

A session's empty model cache over the model hub H.

Effects: fail, alloc.

modelsFolder

gizlang
function modelsFolder(): string

The models folder of a server: $GIZDB_MODELS ("none": no models, ""), else the shared folder of gizlang/hub.

Effects: fail, io, alloc.

serverModels

gizlang
function serverModels(folder: string): Option<Models>

The models of a server session in FOLDER (modelsFolder; "": none). Downloads are on: $GIZDB_MODELS_ALLOW (a comma list of "owner/name" and "owner/") limits them to those repositories, and $GIZDB_MODELS_LIMIT to files of at most that many bytes (default 4 GiB; 0: any size).

Effects: fail, io, alloc.

localModel

gizlang
function localModel(m: mut Models, repo: string): Option<LlmState.Engine> throws DbError

The engine of REPO's language model when the models folder holds it (no download: a model API may serve other names), on the server's GPU when it has one; none for a model the folder does not hold. LlmModel.converted converts the model once, with codes of 8 bits.

Effects: fail, read, write, console, io, alloc.

embeddings

gizlang
function embeddings(m: mut Models, repo: string, texts: Vec<string>, kwargs: string, download: bool): Vec<Value> throws DbError

The embeddings of TEXTS by the model REPO, as real[] values; download: whether the role may download a model.

Effects: fail, read, write, console, io, alloc.

ranked

gizlang
function ranked(m: mut Models, repo: string, query: string, documents: Vec<string>, kwargs: string, download: bool): Vec<Vec<Value>> throws DbError

pgml.rank: the rows (corpus_id from 0, score, text) of the DOCUMENTS by the cross-encoder REPO's score for (QUERY, document), the highest first; text is the document only with return_documents.

Effects: fail, read, write, console, io, alloc.