Use GizStack storage from the command line

giz storage manages a storage range: buckets, objects, keys, usage, a FUSE mount and block volumes. The same commands work against a local range and against gizstack.com. Only two environment variables change.

Preview Runs today. The interface can still change. Roadmap

This guide is task-oriented. giz storage help lists every operation.

Configure the client

The client never receives the master key of the range. On GizStack, the Storage area of your project gives both values. On your own node, giz storage node issue gives the first administrator credential, and giz storage key create gives all later ones.

Start a local node

1. Set a master key of a minimum of 32 bytes, and start one node:

   shCopy

   export GIZ_STORAGE_KEY=local-master-key-0123456789abcdefghij
   NODES="1,127.0.0.1,7400,local,local,1"
   giz storage node start /tmp/node1 --id 1 --listen 127.0.0.1:7400 --tenant my-tenant --nodes "$NODES" &

2. Issue the administrator credential (UID 0, GID 0), and configure the client:

   shCopy

   export GIZ_STORAGE_CREDENTIAL=$(giz storage node issue my-tenant 0 0)
   export GIZ_STORAGE_NODES="$NODES"

For gizstack.com, set GIZ_STORAGE_NODES and GIZ_STORAGE_CREDENTIAL to the values that the Storage area of your project shows. All the other steps are the same.

Buckets

giz storage bucket create photos
giz storage bucket list
giz storage bucket delete photos

bucket list shows only the buckets that the credential may list. bucket delete deletes only an empty bucket.

Objects

An object location is giz://BUCKET/KEY.

giz storage cp photo.jpg giz://photos/2026/photo.jpg
giz storage ls giz://photos/
giz storage ls giz://photos/2026/
giz storage stat giz://photos/2026/photo.jpg
giz storage cp giz://photos/2026/photo.jpg copy.jpg
giz storage rm giz://photos/2026/photo.jpg

• cp writes its progress to stderr.
• An upload of more than 8 MiB sends 8 MiB parts. If the upload stops, run the same cp again. It continues the unfinished upload of the same key and does not send a part again when the range already has that part.
• A download writes to TARGET.VERSION.part. If it stops, run the same cp again. It keeps the whole chunks of the partial file. The target appears only when every byte is written, and each chunk is checked against its identity.
• ls shows the key part up to the next / as one entry, with its /.
• rm writes a delete marker. The earlier versions stay.

Presigned URLs

presign makes an S3 GET URL that anyone can use until it expires. Set the S3 gateway and its signing key:

giz storage presign giz://photos/2026/photo.jpg 600

SECONDS is 1 to 604800 (7 days). The default is 3600.

Keys

Key management needs the administrator credential (UID 0).

giz storage key create 1001 42
giz storage key list
giz storage key revoke 1001 42

• key create prints the credential of the next key generation of the principal. The earlier key of the principal stops working at its next request.
• key revoke stops the key of the principal. Its open connections stop at their next request.
• A new principal has no grants. An administrator gives grants (STORAGE, authorization).

Usage

giz storage usage

The result gives the retained immutable identities, their bytes (each identity once), the copy count and the stored bytes (bytes × copies).

Mount the file system

giz storage mount /mnt/storage

The command serves the mount until it is unmounted (fusermount3 -u /mnt/storage). The UID and GID of the credential must be those of the mount user. FUSE needs no other privilege.

The administrator (UID 0) owns the root of a new file system. To let a user write in it, the administrator gives the user the root, or a directory, first:

giz storage chown / 1000 1000

Block volumes

giz storage volume create disk 1073741824
giz storage volume snapshot disk monday
giz storage volume clone disk@monday disk2
giz storage volume list

The size is in bytes, a multiple of 4096. A snapshot is read-only. giz storage volume serve disk --nbd serves a volume to the Linux NBD client.

Embed storage in a program

gizfs:node runs a store inside your program. It has the same buckets, objects and versions, with no server and no network. embedded makes the folder when it is missing, and a later run opens the same store:

import {System} from "gizlang:system";
import {embedded} from "gizfs:node";
import * as Object from "gizfs:object";

export function main(sys: System): void {
    const store = embedded(sys, "store-data", "my-tenant")!;
    store.changeObject(Object.Command.bucket("my-tenant", "photos", false))!;
    store.put("photos", "2026/hello.txt", utf8("Hello"), "text/plain")!;
    print(string(store.get("photos", "2026/hello.txt", 0)!));
    store.close();
}

• put takes an object of any size. Above 8 MiB it sends 8 MiB parts. putFrom(bucket, key, size, contentType, piece, done) reads the bytes from piece(offset, count) one part at a time, so a large file is never all in memory.
• get(bucket, key, version) reads a version (0: the latest) and checks each chunk.

Keep cold files in a bucket

objectDisk(store, bucket, prefix) from gizfs:tier shows the objects of a bucket as files. A program keeps a file on its own disk while it is hot. When the file is cold, the program moves it with replaceFile or replaceRange, and it reads it back by range:

import * as Volume from "gizfs:volume";
import {objectDisk} from "gizfs:tier";

/** Moves a cold segment into the bucket "photos" of STORE, and reads it back. */
export function keepCold(store: Volume.Client): string {
    const cold = objectDisk(store, "photos", "cold/");
    cold.replaceFile("segment-1", utf8("old rows"))!;
    return string(cold.readFile("segment-1")!);
}

• A read fetches only the chunks it touches. An open file keeps the last 4 chunks, so the pages of one chunk cost one fetch.
• An open file keeps the version it opened, even when another program writes a new one.
• A file is written whole. A write to part of a file gives READ_ONLY.

Clients in more than one region

A range can have nodes in more than one region. The site in each node descriptor (ID,HOST,PORT,PROVIDER,SITE,INCARNATION) names its region. Set GIZ_STORAGE_SITE to the site of the client:

export GIZ_STORAGE_SITE=us-east
giz storage mount /mnt/photos

A program sets it with client.inSite("us-east"). A mount of node start --mount uses the site of its node.

• Reads. A read goes to a node of the client's site first. That node answers when it has the client's own writes and heard the leader in the last second. Otherwise the leader answers. A client always reads its own writes. A write of another client can show up to about one second late.
• Bytes. A node reads a chunk from a copy in its own site first. With --domains 2 and nodes in two sites, each site keeps a copy of each chunk.
• Writes. A write commits at the leader. The leader counts the writes of each site. When one site makes two thirds of the writes in a minute, after five minutes with the same leader, the range moves toward that site:
  • A voter of that site that has the whole log becomes the leader.
  • When the site has no voter, or the leader's site has no majority of the voters, a node of the site becomes a voter in place of a voter of another site. The bytes move as for topology.

Output and exit status

• The result is one line for each record, with tab-separated fields. With --json, the result is one JSON array of objects.
• Exit status 0: success. 1: the operation failed (the reason is on stderr). 2: the command line is wrong (the usage is on stderr).