1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
|
# hopper
`hopper` is a small command-line tool that deploys content-hashed web assets
from a gzipped tarball to [Bunny CDN Storage](https://bunny.net/storage/), and
prunes obsolete files from a storage zone.
## How hopper Thinks About Your Assets
Modern site builders produce content-hashed asset filenames. The name embeds
a hash of the file's contents, e.g. `assets/app.a1b2c3.js`. Because the hash
changes whenever the bytes change, a given filename always maps to exactly the
same bytes forever. `hopper` calls these files *immutable assets*.
The exception is *HTML files* (e.g. `index.html`, `404.html`). They are the
*entry points* that reference the immutable assets, they keep the same name
across releases, and their contents change on every deploy. `hopper` calls
these *mutable assets*.
`hopper` categorizes every `*.html` file (case-insensitive) as a *mutable
asset* and every other file as an *immutable asset*. To save bandwidth,
immutable assets are uploaded only if the zone does not contain this asset
already or the sizes differ. Mutable assets, on the other hand, are always
uploaded, and uploaded only after all immutable assets succeeded. This ensures
that an application stays in a consistent state when an immutable asset upload
fails. Re-running the upload simply resumes.
Because uploading accumulates files on the storage zone, `hopper` can prune old
files. Immutable assets that are not touched for a very long time still keep
their old modification date, so pruning files only on their age would be
dangerous. Instead pruning deletes only old files which are not in the current
release.
## Installation
`hopper` is distributed as source and built with the Go toolchain, which
compiles into a single static binary. To install `hopper` clone the repository
and run `make install` in its root. This installs the binary to
`$PREFIX/bin/hopper` (`$PREFIX` defaults to `/usr/local`).
## Quickstart
Deploy a release archive to a Bunny storage zone:
```sh
export BUNNY_ZONE=my-zone
export BUNNY_ACCESS_KEY=your-storage-access-key
hopper push dist.tar.gz
```
Later, clean up files from old releases that are no longer referenced and have
aged past the grace window (default: 30 days):
```sh
hopper prune dist.tar.gz
# or define a custom grace window
hopper prune --older-than=90d dist.tar.gz
```
Always preview a destructive run first:
```sh
hopper prune --dryrun dist.tar.gz
```
## Usage
```
hopper [global flags] ⟨subcommand⟩ [flags] ⟨args⟩
```
`hopper` has exactly two subcommands: `push` and `prune`. The position of the
global flags are important. They must be before any subcommand.
### Global Flags
Global flags must be defined before any subcommand. Some flags also have a
fallback to an environment variable, so command line arguments always overwrite
the environment variables. An empty environment variable is treated as unset.
| Flag | Env Fallback | Default | Meaning |
|---------------------|-------------------------|--------------------------|---------------------------------------------------|
| `--zone` | `BUNNY_ZONE` | *(none)* | The Bunny storage zone name. |
| `--endpoint` | `BUNNY_ENDPOINT` | `storage.bunnycdn.com` | The Bunny storage endpoint hostname. |
| `--access-key-file` | `BUNNY_ACCESS_KEY_FILE` | *(none)* | Path to a file containing the storage access key. |
| `--verbose` | *(none)* | `false` | Extra detail (listing walk, retries, checksums). |
| `--quiet` | *(none)* | `false` | Print only the final summary and errors. |
*Note:* The access key can also be provided directly using the `BUNNY_ACCESS_KEY`
environment variable. Both, `BUNNY_ACCESS_KEY_FILE` and `--access-key-file` can
overwrite this access key.
### Subcommand `push`
```
hopper [global flags] push [flags] ⟨source⟩
```
Uploads a new release to the storage zone. It iterates over all files of the
`⟨source⟩`, uploads all new immutable assets, and then uploads all mutable
assets. After successfully uploading the assets, `hopper` checks the ratio of
the number of files contained in the storage zone to the number of files in
the given source. If this ratio becomes too large, a hint will be printed
suggesting to run `hopper prune`.
*Note:* An empty source uploads nothing and prints a warning. It is not
considered an error (i.e. it exits with code 0), because the operation is
non-destructive.
#### Flags
| Flag | Meaning |
|-----------------|---------------------------------------------------------------------------------------------------|
| `--concurrency` | The number of concurrent uploads. By default it auto-detects a value based on the number of CPUs. |
| `--prefix` | A path prefix in the storage zone. Every asset is uploaded into this directory. Default: none. |
| `--root` | The subtree of the source that should be uploaded. Default: All source files. |
| `--dryrun` | Run the command without any mutating operations. This flag can be used to verify the upload. |
### Subcommand `prune`
```
hopper [global flags] prune [flags] ⟨args⟩
```
Deletes obsolete files. It builds the *live set* from the given source and a
view of the storage zone. It then deletes every remote file that is not in the
live set and whose last modification is older than a certain grace window.
Finally, it removes all empty directories it can find (failures here do not
affect the exit code). All deletes are independent. If one fails, `hopper`
keeps going deleting all other obsolete files.
*Note:* If the live set is empty, the prune operation refuses to run, because
it could delete the entire zone. In this case, please check your `--root` flag
and the source.
#### Flags
| Flag | Meaning |
|-----------------|--------------------------------------------------------------------------------------------------------------------------|
| `--concurrency` | The number of concurrent deletes. By default it auto-detects a value based on the number of CPUs. |
| `--prefix` | A path prefix in the storage zone. Only assets in this directory are deleted. Default: none. |
| `--root` | The subtree of the source that defines the live set. Default: All source files. |
| `--older-than` | The grace window that is used to determine obsolete files. Only files older than this will be deleted. Default: 30 days. |
| `--dryrun` | Run the command without any mutating operations. This flag can be used to verify the deletes. |
## Retry Policy
If Bunny rate limits your requests, `hopper` receives a `429` and reports it.
In case the response contains the `Retry-After` header, `hopper` honors this
header and automatically retries up to three times. Every other HTTP failure
or a `429` without `Retry-After` fails immediately and reports an error message.
## License
MIT.
|
