# gizfs:bucket An S3-compatible bucket of another provider (AWS S3, R2, MinIO, another GizFS gateway) as a Disk: the store keeps its chunk files there as it keeps them in a folder (diskOf), and nothing above the Disk knows which one it has. Each file is one object under PREFIX. A read open pins the object's ETag, so a later put fails its reads (STALE_RESOURCE) rather than mixing two versions; a write open holds the file in memory and puts it whole at sync or close, so a file written here is at most WRITE_LIMIT bytes (the store's chunk files are far smaller) and a bigger one goes through replaceRange, a multipart upload. A put is durable when it returns, so Disk.sync has nothing to do. Requests are signed with SigV4 (the canonical form is fs/s3.giz's, the one owner of it) and carry the SHA-256 of their body, so plain HTTP to a local endpoint is accepted too. ```gizlang import * as Bucket from "gizfs:bucket"; ``` ## Types ### `Bucket` ```gizlang type Bucket = { prefix: string; virtual: bool; bucket: string; host: string; calls: Task.Channel; } ``` The objects of BUCKET whose keys begin with PREFIX, as files named by the rest of the key, at HOST. VIRTUAL addresses the bucket as a host name (BUCKET.host, as AWS asks of new buckets), else as the first path segment. - `prefix`: `string` - `virtual`: `bool` - `bucket`: `string` - `host`: `string` - `calls`: `gizlang_task.Channel` ### `Object` ```gizlang type Object = { name: string; size: i64; etag: string; } ``` An object of a bucket: its name under the bucket's prefix, its size and its ETag, which changes when its bytes do. - `name`: `string` - `size`: `i64` - `etag`: `string` ## Functions ### `bucketDisk` ```gizlang function bucketDisk(sys: System, endpoint: string, region: string, bucket: string, prefix: string, access: string, secret: bytes, virtual: bool): Bucket throws string ``` The bucket BUCKET of the endpoint ENDPOINT (an http or https URL with no path) in REGION, signed with ACCESS and SECRET. Its requests go to a thread of its own (bucketThread), which signs them and keeps its connections: a Bucket holds no Ref, so a Disk that is one crosses Task.thread (Table.analyzeAside sends a table image there, its Disks with it) and works there. A Bucket that held its pool made every program that reaches both refuse to build (E_EFFECT, 10/27). Close ends the thread. Effects: fail, read, write, console, io. ### `isBucket` ```gizlang function isBucket(disk: string): bool ``` True when DISK (an entry of node start --disks) names a bucket: an http(s) URL. Effects: fail. ### `bucketAt` ```gizlang function bucketAt(sys: System, url: string): Bucket throws string ``` The bucket of URL, ENDPOINT/BUCKET[/PREFIX], with the credentials of the environment: GIZ_STORAGE_BUCKET_ACCESS and GIZ_STORAGE_BUCKET_SECRET, GIZ_STORAGE_BUCKET_REGION (us-east-1 when unset) and GIZ_STORAGE_BUCKET_VIRTUAL=1 for virtual-host addressing. The secret never goes on a command line. Effects: fail, read, write, console, io. ## Methods ### `Bucket.close` ```gizlang function Bucket.close(self: Bucket): void ``` Ends the Bucket's thread once its requests in flight are answered; later requests fail. Effects: fail, io. ### `Bucket.objects` ```gizlang function Bucket.objects(self: Bucket, path: string, most: i64): Vec throws IoError ``` The objects under PATH ("" for all) at any depth, at most MOST, by name. Effects: fail, io. ### `Bucket.entries` ```gizlang function Bucket.entries(self: Bucket, path: string): Entries throws IoError ``` The names under PATH ("" for all): one level, as a folder lists them. Effects: fail, io. ### `Bucket.list` ```gizlang function Bucket.list(self: Bucket, path: string): Vec throws IoError ``` The names under PATH, at most a million (more is an invalid argument). Effects: fail, io. ### `Bucket.open` ```gizlang function Bucket.open(self: Bucket, path: string, mode: i64): File throws IoError ``` A read open pins the object's ETag; a write open (WRITE) holds the file in memory until its sync or close. Effects: fail, io. ### `Bucket.keep` ```gizlang function Bucket.keep(self: Bucket, path: string, data: bytes): bool throws IoError ``` The put of a write open's held bytes: durable when it returns. Effects: fail, io. ### `Bucket.readFile` ```gizlang function Bucket.readFile(self: Bucket, path: string): bytes throws IoError ``` The bytes of the object at PATH. Effects: fail, io. ### `Bucket.replaceFile` ```gizlang function Bucket.replaceFile(self: Bucket, path: string, data: bytes): i64 throws IoError ``` Puts DATA as the object at PATH: durable when it returns. Effects: fail, io. ### `Bucket.replaceRange` ```gizlang function Bucket.replaceRange(self: Bucket, path: string, source: File, start: i64, length: i64, windowBytes: i64): i64 throws IoError ``` LENGTH bytes of SOURCE from START as PATH, one part at a time: WINDOWBYTES does not apply. Effects: fail, io. ### `Bucket.remove` ```gizlang function Bucket.remove(self: Bucket, path: string): bool throws IoError ``` False when PATH was not there (S3's DELETE answers 204 either way, so a HEAD asks first). Effects: fail, io. ### `Bucket.link` ```gizlang function Bucket.link(self: Bucket, from: string, to: string): bool throws IoError ``` A copy: an object store names no object twice. Effects: fail, read, write, console, io. ### `Bucket.sync` ```gizlang function Bucket.sync(self: Bucket): bool throws IoError ``` True: each put is durable when it returns, so nothing waits. ## Constants ### `PART_BYTES` ```gizlang const PART_BYTES = 8388608 ``` The bytes of one request body or ranged read: one multipart part, under gizlang:http's MOST_BODY.