gizai:video/decode

any audio or video file decoded with ffmpeg.

import * as Decode from "gizai:video/decode";

Functions

ffmpegOut

function ffmpegOut(sys: System, ffmpeg: string, args: Vec<string>, input: bytes = b""): bytes throws string

What the program ffmpeg (its name on PATH, or a path) writes to its standard output when it runs with args, its standard input input; its error output when it fails.

Effects: fail, read, write, io.

ffprobe

function ffprobe(ffmpeg: string): string

The ffprobe beside the program ffmpeg: in its folder, else on PATH.

Effects: fail.

audio

function audio(sys: System, ffmpeg: string, path: string, most: f64 = 0.0, rate: i64 = 0, channels: i64 = 0, bits: i64 = 32): Wave throws string

The samples of the audio in the file path (any audio or video file ffmpeg reads: MP3, AAC, M4A, FLAC, Ogg, MP4, WebM, ...) through the program ffmpeg: at most most seconds (when above 0), so a long file stays small; resampled by ffmpeg to rate and mixed to channels (0: the file's own); as bits-bit samples (32: floats; 16: PCM, as a reference decoder that reads 16-bit WAV gets them).

Effects: fail, read, write, io.

mono

function mono(wave: Wave, rate: i64): Vec<f32>

The mean of the channels of wave, at rate (resampled as torchaudio does): a speech model's mono input.

Effects: fail.

probe

function probe(sys: System, ffmpeg: string, path: string): Result<Vec<i64>, string>

The width, height, (whole) frame rate and frames (by its duration; 0: not known) of the first video stream of the file path, by ffprobe next to ffmpeg.

Effects: fail, read, write, io.

frameIndices

function frameIndices(rate: i64, target: i64, count: i64): Vec<i64>

The source frames that WanGP picks for count frames at target fps from a video of rate fps (its _resample_frame_indices from frame 0, which H3 and Qwen's video readers share).

Effects: fail.

frames

function frames(sys: System, ffmpeg: string, path: string, wanted: Vec<i64>, width: i64, height: i64, mask: bool): Result<Vec<bytes>, string>

The frames wanted (ascending indices, as frameIndices gives them) of the video path, as width x height RGB bytes each (a mask: one byte each, 1 where the gray is over 127.5), resized by Lanczos as WanGP's preprocess_video_with_mask does; those past its end are left out.

Effects: fail, read, write, io.

filtered

function filtered(sys: System, ffmpeg: string, path: string, n: i64, filter: string, format: string, size: i64): Vec<bytes> throws string

At most the first n frames of the video path through ffmpeg's filter, as format (rgb24, gray) frames of size bytes.

Effects: fail, read, write, io.