gizdb/ml/embed
pgml.embed, pgml.rank and the local models of pgml.transform: the embeddings of sentence-transformers models, the scores of cross-encoders and the language models of chat (gizai/llm). A model is its files in the models folder (gizlang/hub); each session keeps the models (gizai/nn, gizai/llm) it loaded, as each PostgresML backend keeps its models.
import * as Embed from "gizdb/ml/embed";Types
Models
type Models = { hub: Hub.Hub; encoders: Map<string, Encoder>; crossEncoders: Map<string, CrossEncoder>; decoders: Map<string, LlmState.Engine>; gpu: Option<Gpu>; probed: bool; }Where a session finds models, the models it loaded by repository (with "@gpu" for those on the GPU), and the GPU
(none: not opened yet, or there is none; probed: it was tried).
hub:gizlang_hub.Hubencoders:_gizlang_sdk_collections.Map<string, M56.Encoder>crossEncoders:_gizlang_sdk_collections.Map<string, M56.CrossEncoder>decoders:_gizlang_sdk_collections.Map<string, M55.Engine>gpu:_gizlang_sdk_core.Option<M7.Gpu>probed:bool
Functions
models
function models(h: Hub.Hub): ModelsA session's empty model cache over the model hub H.
Effects: fail, alloc.
modelsFolder
function modelsFolder(): stringThe models folder of a server: $GIZDB_MODELS ("none": no models, ""), else the shared folder of gizlang/hub.
Effects: fail, io, alloc.
serverModels
function serverModels(folder: string): Option<Models>The models of a server session in FOLDER (modelsFolder; "": none). Downloads are on: $GIZDB_MODELS_ALLOW (a comma list of "owner/name" and "owner/") limits them to those repositories, and $GIZDB_MODELS_LIMIT to files of at most that many bytes (default 4 GiB; 0: any size).
Effects: fail, io, alloc.
localModel
function localModel(m: mut Models, repo: string): Option<LlmState.Engine> throws DbErrorThe engine of REPO's language model when the models folder holds it (no download: a model API may serve other names), on the server's GPU when it has one; none for a model the folder does not hold. LlmModel.converted converts the model once, with codes of 8 bits.
Effects: fail, read, write, console, io, alloc.
embeddings
function embeddings(m: mut Models, repo: string, texts: Vec<string>, kwargs: string, download: bool): Vec<Value> throws DbErrorThe embeddings of TEXTS by the model REPO, as real[] values; download: whether the role may download a model.
Effects: fail, read, write, console, io, alloc.
ranked
function ranked(m: mut Models, repo: string, query: string, documents: Vec<string>, kwargs: string, download: bool): Vec<Vec<Value>> throws DbErrorpgml.rank: the rows (corpus_id from 0, score, text) of the DOCUMENTS by the cross-encoder REPO's score for (QUERY, document), the highest first; text is the document only with return_documents.
Effects: fail, read, write, console, io, alloc.