# gizai:wav2vec

the speech encoder of wav2vec 2.0 base and HuBERT base (one architecture, with transformers' names).

16 kHz samples go through seven convolutions (no bias; the first's GroupNorm has a group a channel; GELU after each) to 512 values every 320 samples (50 a second): the features. A LayerNorm and a linear take each to 768, a grouped positional convolution (16 groups, 128 taps, its weight norm applied when it loads; the extra last row of an even kernel dropped) through GELU is added, then a LayerNorm and 12 post-norm transformer layers (12 heads, GELU). The hidden states are the rows after that LayerNorm and after each layer: 13 of [frames, 768].

```gizlang
import * as Wav2vec from "gizai:wav2vec";
```

## Types

### `Encoder`

```gizlang
type Encoder = { gpu: Option<Gpu>; named: Map<string, Tensor>; pos: Vec<Tensor>; }
```

The encoder where it runs: its tensors by name without the prefix (F32), and the positional convolution's weight
with its norm applied, one part a group.

- `gpu`: `_gizlang_sdk_core.Option<M10.Gpu>`
- `named`: `_gizlang_sdk_collections.Map<string, M40.Tensor>`
- `pos`: `Vec<M40.Tensor>`

## Functions

### `load`

```gizlang
function load(sys: System, named: Map<string, Stored>, prefix: string, gpu: Option<Gpu>): Encoder throws string
```

The encoder whose tensors are `prefix` and a transformers name in `named` ("wav2vec2.", "semantic_model."), on
`gpu` when there is one. The positional weight is its g and v (weight_g, weight_v, or their parametrizations).

Effects: fail, read, write, console, io.

## Methods

### `Encoder.features`

```gizlang
function Encoder.features(self: Encoder, samples: Vec<f32>): Tensor
```

The features of the samples (16 kHz): [frames, 512] F32, where the encoder runs.

Effects: fail, read, write, console, io.

### `Encoder.states`

```gizlang
function Encoder.states(self: Encoder, features: Tensor): Vec<Tensor>
```

The 13 hidden states ([frames, 768] F32 each, where the encoder runs) of `features` ([frames, 512]).

Effects: fail, read, write, console, io.

## Constants

### `RATE`

```gizlang
const RATE = 16000
```

The rate of the samples the encoder takes.

### `FEATURES`

```gizlang
const FEATURES = 512
```

The values of a feature, and of a hidden state.

### `WIDTH`

```gizlang
const WIDTH = 768
```

### `LAYERS`

```gizlang
const LAYERS = 12
```

The transformer's layers: the hidden states are one more.