GizStackGizStack
View .mdOpen in ChatGPTOpen in Claude

gizai/models/zonos/model

Zyphra's ZONOS3 Mini, text to speech in 43 languages in a voice of a 192-value speaker embedding (its default voices: gguf/voices/LANGUAGE/NAME.embedding.npy). The prompt is the speaker row, three condition rows (loudness, bandwidth, speaking rate; "unknown" by default), the text (SuperBPE, between BOS and EOS) and three frames of codec silence; the model then writes a frame a step, codebook c delayed by c frames, until the end of audio; the frames go through the codec (gizai/models/zonos/codec), whole or as they come (speaking: the first samples after the first chunk of frames). Sampling is the native runtime's (servers/ggml/src/zonos3-sampler.cpp) with the release's defaults.

gizlang
import * as Model from "gizai/models/zonos/model";

Types

Zonos

gizlang
type Zonos = { lm: Lm.Lm; codec: Codec.Codec; tokenizer: Tokenizer; }

The model, its codec and its tokenizer.

  • lm: _gizlang_lib_ai__models__zonos__lm.Lm
  • codec: _gizlang_lib_ai__models__zonos__codec.Codec
  • tokenizer: M19.Tokenizer

Functions

sampling

gizlang
function sampling(): Lm.Sampling

The release's defaults (zonos3/registries/generation.json), at most 2,000 frames (133 s).

load

gizlang
function load(folder: string, gpu: Gpu, codec: string = ""): Zonos throws string

ZONOS3 Mini of its repository folder folder (lm/zonos3_mini and the release's default decoder gguf/zac-keep2.gguf, or codec: gguf/zac.gguf is the full one, 3 times slower).

Effects: fail, read, write, console, io, alloc.

Methods

Zonos.textIds

gizlang
function Zonos.textIds(self: Zonos, text: string): Vec<i64>

The text's ids: BOS, the tokenizer's ids moved past the four specials, EOS.

Effects: fail, read, write, alloc.

Zonos.speak

gizlang
function Zonos.speak(self: Zonos, text: string, speaker: Vec<f32>, s: Lm.Sampling): Vec<f32>

Speech of text (44.1 kHz mono f32) in the voice of speaker (192 values).

Effects: fail, read, write, console, io, alloc.

Zonos.frames

gizlang
function Zonos.frames(self: Zonos, text: string, speaker: Vec<f32>, s: Lm.Sampling): Vec<Vec<i64>>

The frames ([frames][10], undelayed) of text in the voice of speaker.

Effects: fail, read, write, console, io, alloc.

Zonos.speaking

gizlang
function Zonos.speaking(self: Zonos, text: string, speaker: Vec<f32>, s: Lm.Sampling, each: (samples: Vec<f32>, last: bool) => bool): void

Speech of text in the voice of speaker as it is made: each gets the samples (44.1 kHz mono f32) each piece of frames finishes, the same as speak gives at once, with last on the end; false from it stops the speech.

Effects: fail, read, write, console, io, alloc.

Constants

RATE

gizlang
const RATE: i64 = Codec.RATE

The speech's samples a second.