GizStackGizStack
View .mdOpen in ChatGPTOpen in Claude

gizai/models/depthanything/cameras

the cameras of many pictures of one scene in one pass, by Depth Anything 3 (ByteDance, 2025; DA3-BASE, DA3-LARGE or DA3-GIANT). DINOv2 (ViT-B, L or g, /14) reads each picture. Its first blocks attend within each picture; before the last of them, with three pictures or more, the reference picture is chosen (the one whose class token is nearest the middle of the pictures' spread in similarity, norm and variance) and put first. From then on, the class token becomes a camera token (one for the reference, one for the others), the blocks take q/k norms and RoPE2D, and they attend across all pictures (odd blocks, every patch at one position) and within each (even blocks). The camera token after the last block, joined to the one after the last block within each picture, goes through the camera decoder: a translation, a quaternion and two fields of view a picture. The depth, ray and Gaussian heads are not read. The choice of the reference is close (its scores differ in the fourth digit), so rounding can change it; the cameras then differ as PyTorch's do with another reference.

gizlang
import * as Cameras from "gizai/models/depthanything/cameras";

Types

Model

gizlang
type Model = { w: Scope; vit: Vit; across: i64; metric: Option<Vit>; gpu: Gpu; }

The network on a GPU; across, the first block that attends across pictures (and takes the camera tokens, q/k norms and RoPE2D): the first with q/k norms. With metric (DA3-Metric's ViT-L, a nested checkpoint's), the cameras are in meters.

  • w: _gizlang_lib_ai__netops__linear.Scope
  • vit: _gizlang_lib_ai__parts__dinov2.Vit
  • across: i64
  • metric: _gizlang_sdk_core.Option<_gizlang_lib_ai__parts__dinov2.Vit>
  • gpu: M7.Gpu

Functions

load

gizlang
function load(file: string, gpu: Gpu): Model throws string

The model of file (a DA3 safetensors: ViT-B, L or g; or a nested one, DA3-GIANT and DA3-Metric) on gpu: the encoder and camera decoder, and with DA3-Metric the depth heads of both (the ray, Gaussian and camera encoder parts are not read).

Effects: fail, read, write, console, io, alloc.

size

gizlang
function size(width: i64, height: i64): Vec<i64>

The size DA3 reads a picture of width x height at: the longest side SIDE, the shape kept, each side rounded to a multiple of 14. [width, height].

Effects: fail, alloc.

Methods

Model.cameras

gizlang
function Model.cameras(self: Model, given: Tensor, eight: bool = true): Dinov2.Cameras

The cameras of given (the reference picture's the identity, in meters with DA3-Metric, else in DA3's own scale; the focal lengths the median over the pictures; [n, 3, H, W], colors 0 to 1; resized by bilinear to size(...) when they are not at it). eight: attention across all pictures on the 8-bit matrix cores where the GPU has them.

Effects: fail, read, write, console, io, alloc.

Constants

FILE

gizlang
const FILE = "model.safetensors"

DA3's checkpoint in each repository (DA3-BASE, Apache-2.0; DA3-LARGE and DA3-GIANT, CC BY-NC 4.0).