From 2c41acab4c97f584e8e199b31dc456194b8d7e6c Mon Sep 17 00:00:00 2001 From: tsne Date: Mon, 6 Jul 2026 16:52:20 +0200 Subject: initial --- README.md | 168 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 README.md (limited to 'README.md') diff --git a/README.md b/README.md new file mode 100644 index 0000000..714d523 --- /dev/null +++ b/README.md @@ -0,0 +1,168 @@ +# hopper + +`hopper` is a small command-line tool that deploys content-hashed web assets +from a gzipped tarball to [Bunny CDN Storage](https://bunny.net/storage/), and +prunes obsolete files from a storage zone. + + + +## How hopper Thinks About Your Assets + +Modern site builders produce content-hashed asset filenames. The name embeds +a hash of the file's contents, e.g. `assets/app.a1b2c3.js`. Because the hash +changes whenever the bytes change, a given filename always maps to exactly the +same bytes forever. `hopper` calls these files *immutable assets*. + +The exception is *HTML files* (e.g. `index.html`, `404.html`). They are the +*entry points* that reference the immutable assets, they keep the same name +across releases, and their contents change on every deploy. `hopper` calls +these *mutable assets*. + +`hopper` categorizes every `*.html` file (case-insensitive) as a *mutable +asset* and every other file as an *immutable asset*. To save bandwidth, +immutable assets are uploaded only if the zone does not contain this asset +already or the sizes differ. Mutable assets, on the other hand, are always +uploaded, and uploaded only after all immutable assets succeeded. This ensures +that an application stays in a consistent state when an immutable asset upload +fails. Re-running the upload simply resumes. + +Because uploading accumulates files on the storage zone, `hopper` can prune old +files. Immutable assets that are not touched for a very long time still keep +their old modification date, so pruning files only on their age would be +dangerous. Instead pruning deletes only old files which are not in the current +release. + + + +## Installation + +`hopper` is distributed as source and built with the Go toolchain, which +compiles into a single static binary. To install `hopper` clone the repository +and run `make install` in its root. This installs the binary to +`$PREFIX/bin/hopper` (`$PREFIX` defaults to `/usr/local`). + + + +## Quickstart + +Deploy a release archive to a Bunny storage zone: + +```sh +export BUNNY_ZONE=my-zone +export BUNNY_ACCESS_KEY=your-storage-access-key + +hopper push dist.tar.gz +``` + +Later, clean up files from old releases that are no longer referenced and have +aged past the grace window (default: 30 days): + +```sh +hopper prune dist.tar.gz +# or define a custom grace window +hopper prune --older-than=90d dist.tar.gz +``` + +Always preview a destructive run first: + +```sh +hopper prune --dryrun dist.tar.gz +``` + + + +## Usage + +``` +hopper [global flags] ⟨subcommand⟩ [flags] ⟨args⟩ +``` + +`hopper` has exactly two subcommands: `push` and `prune`. The position of the +global flags are important. They must be before any subcommand. + + +### Global Flags + +Global flags must be defined before any subcommand. Some flags also have a +fallback to an environment variable, so command line arguments always overwrite +the environment variables. An empty environment variable is treated as unset. + +| Flag | Env Fallback | Default | Meaning | +|---------------------|-------------------------|--------------------------|---------------------------------------------------| +| `--zone` | `BUNNY_ZONE` | *(none)* | The Bunny storage zone name. | +| `--endpoint` | `BUNNY_ENDPOINT` | `storage.bunnycdn.com` | The Bunny storage endpoint hostname. | +| `--access-key-file` | `BUNNY_ACCESS_KEY_FILE` | *(none)* | Path to a file containing the storage access key. | +| `--verbose` | *(none)* | `false` | Extra detail (listing walk, retries, checksums). | +| `--quiet` | *(none)* | `false` | Print only the final summary and errors. | + +*Note:* The access key can also be provided directly using the `BUNNY_ACCESS_KEY` +environment variable. Both, `BUNNY_ACCESS_KEY_FILE` and `--access-key-file` can +overwrite this access key. + + +### Subcommand `push` + +``` +hopper [global flags] push [flags] ⟨source⟩ +``` + +Uploads a new release to the storage zone. It iterates over all files of the +`⟨source⟩`, uploads all new immutable assets, and then uploads all mutable +assets. After successfully uploading the assets, `hopper` checks the ratio of +the number of files contained in the storage zone to the number of files in +the given source. If this ratio becomes too large, a hint will be printed +suggesting to run `hopper prune`. + +*Note:* An empty source uploads nothing and prints a warning. It is not +considered an error (i.e. it exits with code 0), because the operation is +non-destructive. + +#### Flags +| Flag | Meaning | +|-----------------|---------------------------------------------------------------------------------------------------| +| `--concurrency` | The number of concurrent uploads. By default it auto-detects a value based on the number of CPUs. | +| `--prefix` | A path prefix in the storage zone. Every asset is uploaded into this directory. Default: none. | +| `--root` | The subtree of the source that should be uploaded. Default: All source files. | +| `--dryrun` | Run the command without any mutating operations. This flag can be used to verify the upload. | + + +### Subcommand `prune` + +``` +hopper [global flags] prune [flags] ⟨args⟩ +``` + +Deletes obsolete files. It builds the *live set* from the given source and a +view of the storage zone. It then deletes every remote file that is not in the +live set and whose last modification is older than a certain grace window. +Finally, it removes all empty directories it can find (failures here do not +affect the exit code). All deletes are independent. If one fails, `hopper` +keeps going deleting all other obsolete files. + +*Note:* If the live set is empty, the prune operation refuses to run, because +it could delete the entire zone. In this case, please check your `--root` flag +and the source. + +#### Flags +| Flag | Meaning | +|-----------------|--------------------------------------------------------------------------------------------------------------------------| +| `--concurrency` | The number of concurrent deletes. By default it auto-detects a value based on the number of CPUs. | +| `--prefix` | A path prefix in the storage zone. Only assets in this directory are deleted. Default: none. | +| `--root` | The subtree of the source that defines the live set. Default: All source files. | +| `--older-than` | The grace window that is used to determine obsolete files. Only files older than this will be deleted. Default: 30 days. | +| `--dryrun` | Run the command without any mutating operations. This flag can be used to verify the deletes. | + + + +## Retry Policy + +If Bunny rate limits your requests, `hopper` receives a `429` and reports it. +In case the response contains the `Retry-After` header, `hopper` honors this +header and automatically retries up to three times. Every other HTTP failure +or a `429` without `Retry-After` fails immediately and reports an error message. + + + +## License + +MIT. -- cgit v1.3