# gizai:higgs

Boson AI's Higgs audio tokenizer v2, the codec of OmniVoice: 24 kHz mono audio as 8 codes of 1024 values every 960 samples (25 a second), and back.

Encoding joins two views of the audio at 25 Hz: DAC's acoustic encoder of the 24 kHz samples (256 values a frame), and the mean of HuBERT base's 13 hidden states of the audio at 16 kHz, every second frame, through a semantic encoder (768 values). Their [acoustic; semantic] row goes through a linear to 1024 values, and 8 quantizers code it in turn: each projects the residual to 64 values, takes its codebook's nearest row, and takes that row's projection back out of the residual. Decoding adds the 8 projections, takes them to 256 values, and DAC's decoder makes the samples (its transposed convolutions with output padding stride % 2, and no tanh). The checkpoint holds its convolutions with their weight norms folded, but HuBERT's positional convolution's (its g and v, the norm over all but the taps).

```gizlang
import * as Higgs from "gizai:higgs";
```

## Types

### `Higgs`

```gizlang
type Higgs = { gpu: Option<Gpu>; named: Map<string, Tensor>; alphas: Map<string, Vec<f32>>; books: Vec<Tensor>; pos: Vec<Tensor>; }
```

The codec where it runs: its tensors by name (F32 on the GPU, or on the host), the Snake alphas and the codebooks
on the host, and HuBERT's positional convolution's weight with its norm applied (one per group).

- `gpu`: `_gizlang_sdk_core.Option<M10.Gpu>`
- `named`: `_gizlang_sdk_collections.Map<string, M38.Tensor>`
- `alphas`: `_gizlang_sdk_collections.Map<string, Vec<f32>>`
- `books`: `Vec<M38.Tensor>`
- `pos`: `Vec<M38.Tensor>`

## Functions

### `load`

```gizlang
function load(sys: System, path: string, gpu: Option<Gpu>): Higgs throws string
```

The codec in `path` (audio_tokenizer/model.safetensors), on `gpu` when there is one.

Effects: fail, read, write, console, io.

## Methods

### `Higgs.decode`

```gizlang
function Higgs.decode(self: Higgs, codes: Vec<Vec<i64>>): Vec<f32>
```

The samples (24 kHz) of `codes`: CODEBOOKS rows of a code a frame each. HOP samples a frame.

Effects: fail, read, write, console, io.

### `Higgs.encode`

```gizlang
function Higgs.encode(self: Higgs, samples: Vec<f32>): Vec<Vec<i64>>
```

The codes of `samples` (24 kHz, a multiple of HOP of them): CODEBOOKS rows of samples / HOP codes.

Effects: fail, read, write, console, io.

## Constants

### `RATE`

```gizlang
const RATE = 24000
```

The samples a second of the audio the codec takes and makes.

### `HOP`

```gizlang
const HOP = 960
```

The samples of one frame of codes.

### `CODEBOOKS`

```gizlang
const CODEBOOKS = 8
```

The codes of a frame (quantizers).

### `CODES`

```gizlang
const CODES = 1024
```

The values of a code.

gizai:higgs

Boson AI's Higgs audio tokenizer v2, the codec of OmniVoice: 24 kHz mono audio as 8 codes of 1024 values every 960 samples (25 a second), and back.

Encoding joins two views of the audio at 25 Hz: DAC's acoustic encoder of the 24 kHz samples (256 values a frame), and the mean of HuBERT base's 13 hidden states of the audio at 16 kHz, every second frame, through a semantic encoder (768 values). Their [acoustic; semantic] row goes through a linear to 1024 values, and 8 quantizers code it in turn: each projects the residual to 64 values, takes its codebook's nearest row, and takes that row's projection back out of the residual. Decoding adds the 8 projections, takes them to 256 values, and DAC's decoder makes the samples (its transposed convolutions with output padding stride % 2, and no tanh). The checkpoint holds its convolutions with their weight norms folded, but HuBERT's positional convolution's (its g and v, the norm over all but the taps).

gizlang

import * as Higgs from "gizai:higgs";

Types

`Higgs`

gizlang

type Higgs = { gpu: Option<Gpu>; named: Map<string, Tensor>; alphas: Map<string, Vec<f32>>; books: Vec<Tensor>; pos: Vec<Tensor>; }

The codec where it runs: its tensors by name (F32 on the GPU, or on the host), the Snake alphas and the codebooks on the host, and HuBERT's positional convolution's weight with its norm applied (one per group).

gpu: _gizlang_sdk_core.Option<M10.Gpu>
named: _gizlang_sdk_collections.Map<string, M38.Tensor>
alphas: _gizlang_sdk_collections.Map<string, Vec<f32>>
books: Vec<M38.Tensor>
pos: Vec<M38.Tensor>

Functions

`load`

gizlang

function load(sys: System, path: string, gpu: Option<Gpu>): Higgs throws string

The codec in path (audio_tokenizer/model.safetensors), on gpu when there is one.

Effects: fail, read, write, console, io.

Methods

`Higgs.decode`

gizlang

function Higgs.decode(self: Higgs, codes: Vec<Vec<i64>>): Vec<f32>

The samples (24 kHz) of codes: CODEBOOKS rows of a code a frame each. HOP samples a frame.

Effects: fail, read, write, console, io.

`Higgs.encode`

gizlang

function Higgs.encode(self: Higgs, samples: Vec<f32>): Vec<Vec<i64>>

The codes of samples (24 kHz, a multiple of HOP of them): CODEBOOKS rows of samples / HOP codes.

Effects: fail, read, write, console, io.

Constants

`RATE`

gizlang

const RATE = 24000

The samples a second of the audio the codec takes and makes.

`HOP`

gizlang

const HOP = 960

The samples of one frame of codes.

`CODEBOOKS`

gizlang

const CODEBOOKS = 8

The codes of a frame (quantizers).

`CODES`

gizlang

const CODES = 1024

The values of a code.