GizStack
View .mdOpen in ChatGPTOpen in Claude

gizai/models/pi3/model

Pi3X (π³, Wang et al., 2025), the camera of each of many pictures of one scene in one pass, with no reference view: DINOv2 ViT-L/14 with registers reads each picture; 36 blocks with RoPE2D and q/k norms attend in turn within a picture and across all; the last two blocks' tokens, joined, go through a camera decoder and head (a rotation by SVD and a translation per picture) and through a point decoder and MoGe's convolution head (each pixel's ray, x / z and y / z), whose median gives the focal length; a metric token reads all tokens across and scales the translations to meters. The confidence head and the optional depth, ray and pose inputs are not read.

gizlang
import * as Model from "gizai/models/pi3/model";

Types

Model

gizlang
type Model = { w: Scope; vit: Vit; gpu: Gpu; }

The network on a GPU.

  • w: _gizlang_lib_ai__netops__linear.Scope
  • vit: _gizlang_lib_ai__parts__dinov2.Vit
  • gpu: M7.Gpu

Cameras

gizlang
type Cameras = { c2w: Vec<Vec<f64>>; fx: f64; fy: f64; }

The cameras of the pictures of one scene: each camera to world matrix (OpenCV axes, row by row, 16 values; the first picture's near the identity, in meters) and the focal lengths over the picture's width and height.

  • c2w: Vec<Vec<f64>>
  • fx: f64
  • fy: f64

Functions

load

gizlang
function load(file: string, gpu: Gpu): Model throws string

The model of file (Pi3X's safetensors) on gpu, less the parts that read optional inputs.

Effects: fail, read, write, console, io, alloc.

size

gizlang
function size(width: i64, height: i64, n: i64, each: i64 = 250000): Vec<i64>

The size Pi3X reads n pictures of width x height at: about 12 million pixels over all (each at most a picture), the picture's shape kept, each side a multiple of 14. [width, height].

Effects: fail, alloc.

Methods

Model.cameras

gizlang
function Model.cameras(self: Model, given: Tensor, focal: i64, each: i64 = 250000, eight: bool = true): Cameras

The cameras of pictures ([n, 3, H, W], colors 0 to 1; resized by bilinear to size(..., each) when they are not at it), the focal lengths read from focal of them (spread over the list). eight: the decoder's attention across all pictures on the 8-bit matrix cores where the GPU has them.

Effects: fail, read, write, console, io, alloc.