# gizai/world/scene

a 3D scene of one photo (Mira-Scene's task) on gizlang's own parts. A vision chat model names the things, their boxes and what holds each; Pi3X gives each pixel's point in meters; the cutter cuts each thing out; a mesh model that sees the photo's camera (Pixal3D: its mesh is aligned to the picture) makes each thing, rigged when it is a character. So a thing's place needs no model of canonical coordinates (Mira-Scene's CCM): its size and place follow in closed form from its crop, its depth and the focal length. A plane fitted to the floor's points gives the up direction; things stand on the floor or on the surface of what holds them that is nearest under their measured place (a shelf, a top); the floor is a plane colored from the photo, and the rest of the photo is the backdrop, a mesh of its points that goes on behind the things.

The cutter cuts out what stands out in a crop, which is not always the thing named: a stool before a desk is the desk's cut too. So each pixel is one thing's (claimed): a cut under two names is the thing of the smaller box, a thing inside a larger thing's cut at its depth is a part of that thing (books on a shelf: made with the shelf), and a thing before another takes its pixels from it.

```gizlang
import * as Scene from "gizai/world/scene";
```

## Types

### `Found`

```gizlang
type Found = { name: string; box: Vec<f64>; on: i64; character: bool; }
```

A thing the vision model found: its name, its box (x0, y0, x1, y1 in the photo's pixels), what holds it, and
whether it is a character (rigged).

- `name`: `string`
- `box`: `Vec<f64>`
- `on`: `i64`
- `character`: `bool`

### `Parts`

```gizlang
type Parts = { look: (photo: Img.Image) => (() => string throws string) throws string; points: (photo: Img.Image) => Points throws string; cuts: (crops: Vec<Img.Image>) => (() => Vec<bytes> throws string) throws string; sculptor: () => Sculptor throws string; }
```

The parts a scene is made with: the vision model's answer to LOOK_PROMPT about a photo, the photo's points
(Pi3X), the cutter (a picture's alpha) and the mesh model (a Sculptor with `mesh`). Each may load its model when called
and free it after, so that one card of 16 GB holds them in turn.

- `look`: `(arg0: gizlang_image.Image) => (() => string throws string) throws string` — Starts the look at the photo; what it gives waits for the answer. A look on another machine runs beside the
work here.
- `points`: `(arg0: gizlang_image.Image) => M79.Points throws string`
- `cuts`: `(arg0: Vec<gizlang_image.Image>) => (() => Vec<bytes> throws string) throws string` — Starts cutting the things out of their crops; what it gives waits for each crop's alpha. Cuts on another
machine run beside the work here.
- `sculptor`: `() => M86.Sculptor throws string`

## Functions

### `foundOf`

```gizlang
function foundOf(text: string, width: i64, height: i64): Vec<Found> throws string
```

The things of the vision model's answer `text` (LOOK_PROMPT's list) in a photo of `width` x `height`. A model
may wrap the list in an object (`{"items": [...]}`), in a code fence, leave the ids out (they count from 1), and
name a holder by its id (a number, or its digits) or by its name (of two things of the name, the one whose box
lies the most under the thing's).

Effects: fail, alloc.

### `scene`

```gizlang
function scene(parts: Parts, photo: bytes, seed: i64, progress: (stage: string, step: i64, total: i64) => bool): bytes throws string
```

The scene of `photo` (a JPEG, PNG or WebP file's bytes) as one GLB: the backdrop, the floor, and each thing (named
by what it is; a character rigged), in meters, y up, the floor at y = 0, the photo's camera at x = z = 0 looking
toward -z. `progress` hears the stage, the step and the steps; false from it stops the work.

Effects: fail, read, write, console, io, alloc.

## Constants

### `LOOK_PROMPT`

```gizlang
const LOOK_PROMPT = "List every separate movable thing in the picture: furniture, objects, people and animals. " + "Do not list the floor, walls, ceiling, windows, doors or the sky. Answer with JSON only, no other words: " + "[{\"id\":1,\"name\":\"a short name\",\"box\":[x0,y0,x1,y1],\"on\":\"floor\" or \"wall\" or the id of the " + "thing it stands on or in,\"character\":true for a person or an animal}]. Ids count from 1. Boxes are on a 0 to " + "1000 scale of the picture's width and height."
```

What the vision model is asked about the photo.

### `ON_FLOOR`

```gizlang
const ON_FLOOR: i64 = -1
```

What holds a thing: the floor, nothing (a wall, the ceiling, or a part of the room that is no thing), or another
thing (its index among the found, 0 or more).

### `ON_NOTHING`

```gizlang
const ON_NOTHING: i64 = -2
```

A thing held by nothing: on a wall or the ceiling.

