# gizai/models/ardy/model

NVIDIA's ARDY (ARDY-SOMA-RP-30FPS-Horizon60), a person's motion from a text, made two seconds at a time after the motion so far, as nv-tlabs/ardy computes it. A token is 4 frames: the global root of each (its place x, y, z and heading cos, sin) and 128 latent values of the body. A window denoises 15 new tokens after at most 60 tokens of history in 10 DDIM steps, each step two transformers of 8 layers: the root stage gives the roots, the body stage the latents, reading the roots as local velocities. Classifier-free guidance weighs the text against no text. A causal transformer (the FSQ tokenizer's decoder) gives each frame's body: the joints' places, global rotations (6D), velocities and foot contacts of SOMA's 30 joints. Many people step in one batch.

```gizlang
import * as Model from "gizai/models/ardy/model";
```

## Types

### `Ardy`

```gizlang
type Ardy = { gpu: Gpu; root: Stage; body: Stage; rootHistory: Affine; rootNew: Affine; bodyHistory: Affine; bodyNew: Affine; rootGoal: Affine; bodyGoal: Affine; decIn: Affine; decCond: Affine; decLayers: Vec<Layer>; decOut: Affine; decPlaces: Tensor; mean: Vec<f64>; std: Vec<f64>; latentMean: Vec<f64>; latentStd: Vec<f64>; }
```

ARDY on a GPU: its two stages, the projections of a token into each (history and generation: the generation's
first columns; its constraint columns, the observed body and the mask of TOKEN frames, apart with no bias), the
decoder, and the statistics: of the features (global root, local root, body: 370) and of the latents after
quantization.

- `gpu`: `M7.Gpu`
- `root`: `M79.Stage`
- `body`: `M79.Stage`
- `rootHistory`: `M79.Affine`
- `rootNew`: `M79.Affine`
- `bodyHistory`: `M79.Affine`
- `bodyNew`: `M79.Affine`
- `rootGoal`: `M79.Affine`
- `bodyGoal`: `M79.Affine`
- `decIn`: `M79.Affine`
- `decCond`: `M79.Affine`
- `decLayers`: `Vec<M79.Layer>`
- `decOut`: `M79.Affine`
- `decPlaces`: `_gizlang_lib_ai__tensor__storage.Tensor<f32>` — The decoder's places of its tokens ([HISTORY / TOKEN + HORIZON / TOKEN, DEC_WIDTH]).
- `mean`: `Vec<f64>`
- `std`: `Vec<f64>`
- `latentMean`: `Vec<f64>`
- `latentStd`: `Vec<f64>`

### `Goal`

```gizlang
type Goal = { frame: i64; x: f64; z: f64; heading?: f64; }
```

Where a person's root is to be at frame `frame` of its next window (0 to HORIZON - 1): its place x and z in the
world, and its heading (radians from +z toward +x) when given.

- `frame`: `i64`
- `x`: `f64`
- `z`: `f64`
- `heading`: `_gizlang_sdk_core.Option<f64>`

### `People`

```gizlang
type People = { tokens: Vec<Vec<f64>>; center: Vec<Vec<f64>>; heading: Vec<f64>; text: Vec<Vec<f32>>; goals: Vec<Vec<Goal>>; }
```

People in motion, one sequence each: their history of tokens (HYBRID values each, roots about the place `center`
last recentred on: x and z each), their heading at the first token they read, their text vectors ([TEXT]), and the
goals of their next window (Ardy.step reads and clears them).

- `tokens`: `Vec<Vec<f64>>`
- `center`: `Vec<Vec<f64>>`
- `heading`: `Vec<f64>`
- `text`: `Vec<Vec<f32>>`
- `goals`: `Vec<Vec<M79.Goal>>`

## Functions

### `load`

```gizlang
function load(folder: string, gpu: Gpu): Ardy throws string
```

ARDY of `folder` (denoiser.safetensors, tokenizer.safetensors, stats/) on `gpu`.

Effects: fail, read, write, console, io, alloc.

### `people`

```gizlang
function people(texts: Vec<Vec<f32>>): People
```

People with no motion yet, facing +z at the origin, each with its text vector.

Effects: fail, alloc.

### `frames`

```gizlang
function frames(features: Vec<f64>): Frames
```

FEATURES (Ardy.features) as frames of SOMA30 (gizlang/retarget): each joint's turn from its global 6D rotation
(the first two columns of its matrix) made local to its parent's, and the root's place.

Effects: fail, read, write, console, io, alloc.

## Methods

### `Ardy.step`

```gizlang
function Ardy.step(self: Ardy, p: mut People, seed: u64, noise: Option<(person: i64) => Vec<f32>> = undefined): void
```

The next window of every person: HORIZON new frames as 15 tokens added to its history, its roots recentred on its
last frame. `noise` gives a person's starting noise (15 x HYBRID); none: drawn from `seed`. A person's goals guide
its window as a third pass: no text, the goals observed (classifier-free guidance of the goals, as of the text).

Effects: fail, read, write, console, io, alloc.

### `Ardy.features`

```gizlang
function Ardy.features(self: Ardy, p: People, since: i64 = 0): Vec<Vec<f64>>
```

Every person's frames from frame `since` (a window's first; 0: all) as features ([frames x REP], not normalized),
its roots in the world: each window of 15 tokens decoded after at most HISTORY frames before it, all people in one
batch. A game takes each window's new frames with `since` at its first.

Effects: fail, read, write, console, io, alloc.

## Constants

### `FPS`

```gizlang
const FPS: f64 = 30.0
```

The frames a second.

### `HORIZON`

```gizlang
const HORIZON: i64 = 60
```

The new frames of a window (Ardy.step).

### `JOINTS`

```gizlang
const JOINTS: Vec<string> = ["Hips", "Spine1", "Spine2", "Chest", "Neck1", "Neck2", "Head", "Jaw", "LeftEye", "RightEye", "LeftShoulder", "LeftArm", "LeftForeArm", "LeftHand", "LeftHandThumbEnd", "LeftHandMiddleEnd", "RightShoulder", "RightArm", "RightForeArm", "RightHand", "RightHandThumbEnd", "RightHandMiddleEnd", "LeftUpperLeg", "LeftLowerLeg", "LeftFoot", "LeftToeBase", "RightUpperLeg", "RightLowerLeg", "RightFoot", "RightToeBase"]
```

SOMA's 30 joints (ARDY's somaskel30), parents first, and each one's parent. Its thighs and shins are named as VRM
names them (SOMA's own names are Leg and Shin).

### `REP`

```gizlang
const REP: i64 = 366
```

A frame's features: root place (3), heading (2), the other joints' places (29 x 3), global rotations (30 x 6),
velocities (30 x 3), foot contacts (4); the root's part; the body's part; the local root (rotation velocity,
x and z velocity, height).

### `TEXT`

```gizlang
const TEXT: i64 = 4096
```

A text vector's width (LLM2Vec's).

