# gizhw:npu the NPUs of the machine, whatever their driver, and the runs of kernels on them. An NPU is an accel-class device (/dev/accel/accelN) whose driver a backend serves (amdxdna.giz: AMD XDNA). A program uses one `Session` for device buffers, loaded kernels and their runs; a kernel's firmware image and control code are in its backend's formats (XDNA: an xclbin's PDI and an AIE instruction list, which Amdxdna.Txn writes). The DRM driver's name and version are read the same way for every NPU (drm.giz), and each backend adds its own firmware reads. Adding an NPU vendor is one backend and one case in each function here. ```gizlang import * as Npu from "gizhw:npu"; ``` ## Types ### `Npu` ```gizlang type Npu = { node: string; driver: string; } ``` An NPU: its node `node` (under /dev) and the driver bound to it. - `node`: `string` - `driver`: `string` ### `Info` ```gizlang type Info = { driver: string; version: string; firmware: string; } ``` What an NPU reports: its driver's name and version, and its firmware's version. - `driver`: `string` - `version`: `string` - `firmware`: `string` ### `Session` ```gizlang type Session = { npu: Npu; xdna: Amdxdna.Context; loaded: i64; } ``` An open NPU: the device and its backend's state (XDNA: the hardware context of the first kernel loaded; each other kernel gets its own, as a context holds one firmware image). - `npu`: `M14.Npu` - `xdna`: `M1.Context` - `loaded`: `i64` ### `Buffer` ```gizlang type Buffer = Amdxdna.Bo ``` A device buffer: kernels reach it by its device address `at`. ### `Kernel` ```gizlang type Kernel = Amdxdna.Kernel ``` A kernel to load: its firmware image and its control code. ### `Program` ```gizlang type Program = Amdxdna.Program ``` A loaded kernel, ready to run. ### `Command` ```gizlang type Command = Amdxdna.Command ``` A command buffer: one run's packet, from its submit until the run ends. ## Functions ### `supports` ```gizlang function supports(driver: string): bool ``` Whether a backend serves the driver `driver`. ### `devices` ```gizlang function devices(sys: System): Vec throws IoError ``` The NPUs of the machine: the accel devices a backend serves. Effects: fail, io. ### `info` ```gizlang function info(sys: System, npu: Npu): Info throws string ``` What NPU `npu` reports, read through its node (opened and closed here). Effects: fail, io. ### `open` ```gizlang function open(sys: System, which: i64 = 0): Session throws string ``` Opens NPU number `which` of the machine (devices). Effects: fail, io. ### `recoded` ```gizlang function recoded(p: Program, code: bytes): Program throws string ``` Program `p` with its control code replaced by `code`. Effects: fail, io. ### `variant` ```gizlang function variant(p: Program, code: bytes): Program throws string ``` Another program of the firmware of `p`, with control code `code` (a kernel whose runs differ only in their control code, such as each kind of a model's layers). Effects: fail, io. ### `submit` ```gizlang function submit(p: Program, cmd: Command, args: Vec, buffers: Vec = []): u64 throws string ``` Queues a run of `p` in `cmd` with the arguments `args` (the buffers' device addresses among them) and returns its point on the timeline of p's firmware: runs of one firmware go in order. The run's `buffers` stay held until it ends. Effects: fail, io. ### `finish` ```gizlang function finish(p: Program, point: u64, commands: Vec): void throws string ``` Waits until the firmware of `p` reaches `point` (submit), then checks that each of `commands` completed. Effects: fail, io. ### `run` ```gizlang function run(p: Program, args: Vec, buffers: Vec = []): void throws string ``` Runs `p` once (submit and finish in its own command buffer). Effects: fail, io. ## Methods ### `Session.close` ```gizlang function Session.close(self: Session): void ``` Closes the session: its buffers and programs go with it. Effects: fail, io. ### `Session.buffer` ```gizlang function Session.buffer(self: Session, size: i64): Buffer throws string ``` A device buffer of `size` zero bytes. Effects: fail, io. ### `Session.upload` ```gizlang function Session.upload(self: Session, b: Buffer, data: bytes, at: i64 = 0): void ``` Copies `data` into buffer `b` at byte `at`, before the runs that read it. Effects: fail, io. ### `Session.download` ```gizlang function Session.download(self: Session, b: Buffer, size: i64, at: i64 = 0): bytes ``` The `size` bytes of buffer `b` from byte `at`, as the runs before left them. Effects: fail, io. ### `Session.downloadInto` ```gizlang function Session.downloadInto(self: Session, b: Buffer, into: mut bytes, at: i64 = 0): void ``` As download, the `into.length` bytes from byte `at` copied into `into`, with no allocation. Effects: fail, io. ### `Session.load` ```gizlang function Session.load(self: mut Session, kernel: Kernel): Program throws string ``` Loads `kernel`, with a command buffer for its `run`. Effects: fail, io. ### `Session.command` ```gizlang function Session.command(self: Session): Command throws string ``` A command buffer for a run submitted while others are in flight. Effects: fail, io.