GizStackGizStack
View .mdOpen in ChatGPTOpen in Claude

gizai/tasks/face

speech to the motion of the face that says it. A model family that moves a face from speech gives an Animator (gizai/family's face): motion gives each frame's MHR expression weights. The motion goes into the voice's WAV file as its "face" chunk (gizlang/wav Face), so a game plays the face with the voice (gizgame's speech). giz ai face, a worker (faceService: operation audio.face) and a program use every such model so.

gizlang
import * as Face from "gizai/tasks/face";

Types

Animator

gizlang
type Animator = { gpu: Gpu; rate: i64; fps: f64; motion: (samples: Vec<f32>) => Vec<f64> throws string; }

What a family gives: the card it loaded on, the rate of the mono speech it reads, and motion: each frame's weights of the MHR expressions (EXPRESSION_NAMES' order), fps frames a second from the speech's start.

  • gpu: M7.Gpu
  • rate: i64
  • fps: f64
  • motion: (arg0: Vec<f32>) => Vec<f64> throws string

Functions

faced

gizlang
function faced(a: Animator, ffmpeg: string, audio: bytes): bytes throws string

audio (a sound file's bytes, in any format the program ffmpeg reads) as a WAV file with the face that says it: a WAV file keeps its sound, another becomes 16-bit PCM at the model's rate.

Effects: fail, read, write, console, io, alloc.

faceService

gizlang
function faceService(name: string, load: (gpu: Gpu) => Animator throws string, capacity: i64 = 4): Hosting.Service

The service of face model name as giz's inference worker: operation audio.face, capacity jobs held; load makes the model on the host's GPU. A job's input: audio (an uploaded recording); its result is face.wav (faced).

Effects: fail, read, write, console, io, alloc.