Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
9adcfed28e
|
|||
|
71b64baf45
|
|||
|
68c1f03503
|
2
Cargo.lock
generated
2
Cargo.lock
generated
@@ -4,7 +4,7 @@ version = 4
|
||||
|
||||
[[package]]
|
||||
name = "ahab"
|
||||
version = "0.4.2"
|
||||
version = "0.5.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"clap",
|
||||
|
||||
368
README.md
368
README.md
@@ -1,20 +1,140 @@
|
||||
# ahab
|
||||
|
||||
A wrapper around `docker compose` for our dockerized django projects, so the
|
||||
A wrapper around `docker compose` for our dockerized Django projects, so the
|
||||
same commands work in every repository.
|
||||
|
||||
## installing
|
||||
It reads `docker compose config` to work out which service runs Django and
|
||||
which one is the database, so nothing here takes a service name or a `-f`.
|
||||
Everything it does, it does by running `docker`, `git` and the Postgres tools.
|
||||
|
||||
## Installing
|
||||
|
||||
You will need Rust installed. Clone the repo and run:
|
||||
|
||||
You will need rust installed. Clone repo and run:
|
||||
```bash
|
||||
cargo install --path .
|
||||
```
|
||||
|
||||
## what it makes of a project
|
||||
## Django
|
||||
|
||||
`ahab status` prints the services it picked, the containers behind them, the
|
||||
credentials it would use and what the store holds, reporting whatever it cannot
|
||||
work out rather than stopping at the first thing:
|
||||
Each of these runs `docker compose run --rm` against the Django service, so it
|
||||
goes through the image's own entrypoint:
|
||||
|
||||
```bash
|
||||
ahab django run <cmd> # anything, in a fresh container
|
||||
ahab django bash # a shell in a fresh container
|
||||
ahab django manage <args> # manage.py
|
||||
ahab django makemigrations
|
||||
ahab django migrate <args>
|
||||
ahab django shell
|
||||
ahab django make-command <app> <name>
|
||||
```
|
||||
|
||||
## Postgres
|
||||
|
||||
```bash
|
||||
ahab postgres dump <path> # pg_dump, custom format
|
||||
ahab postgres dump -F plain <path> # pg_dump, plain SQL
|
||||
ahab postgres dump -F tar <path> # pg_dump, tar
|
||||
ahab postgres dump -F directory <d> # pg_dump, a directory of files
|
||||
ahab postgres dump -F cluster <path> # pg_dumpall, roles and all databases
|
||||
ahab postgres dump -F plain -z <path> # any of them, gzipped
|
||||
ahab postgres import <path> # drop, create, then restore
|
||||
ahab postgres psql <args> # psql in the database container
|
||||
```
|
||||
|
||||
Dumps are written `0600`, under a name of their own until they are complete and
|
||||
then renamed over the target.
|
||||
|
||||
Arguments reach `psql` untouched, and Docker is only asked for a terminal when
|
||||
there is one to hand over, so both of these work:
|
||||
|
||||
```bash
|
||||
ahab postgres psql # interactive
|
||||
ahab postgres psql -tAc 'select count(*) from auth_user' | wc -l
|
||||
```
|
||||
|
||||
### What an import makes of a dump
|
||||
|
||||
The shape is read from the file rather than from its name:
|
||||
|
||||
- a `PGDMP` or `toc.dat` header — an archive, restored with `pg_restore`, as is
|
||||
a directory produced by `pg_dump -Fd`
|
||||
- a `pg_dumpall` header — a whole cluster, which creates its own databases and
|
||||
carries role statements, so the database is dropped but not recreated and the
|
||||
dump goes to `psql` connected to `postgres`
|
||||
- gzip magic — decompressed on the way in, whichever of the three is inside
|
||||
- anything else — SQL, fed to `psql` with `ON_ERROR_STOP` and
|
||||
`--single-transaction`, so a bad file rolls back instead of half applying
|
||||
|
||||
A cluster restore runs without `ON_ERROR_STOP`, so the expected complaints
|
||||
about roles that already exist do not stop it. Any other error fails the
|
||||
import. Importing into a cluster that has never held the database works either
|
||||
way.
|
||||
|
||||
Containers are stopped before an import starts, so one that fails leaves the
|
||||
project stopped and says so.
|
||||
|
||||
## Link
|
||||
|
||||
A sandbox that mounts the repository can read anything untracked sitting in it
|
||||
— a `.env`, a dump, a key. Those paths can be moved out to a store of ahab's
|
||||
own and symlinked back: the host resolves the link, the sandbox does not.
|
||||
|
||||
```bash
|
||||
ahab link add .env secrets/ # move out, leave symlinks behind
|
||||
ahab link restore .env # move it back into the repository
|
||||
ahab link restore --all # everything this repo has linked
|
||||
ahab link list # what the store holds for this repository
|
||||
ahab link check # what a sandbox can still read
|
||||
ahab link check --porcelain # `<code> <path>`, for scripts
|
||||
ahab link check --exit-code # exit 4 when anything is outside the store
|
||||
```
|
||||
|
||||
Restoring is the inverse of adding: the file comes back to where it was and the
|
||||
store keeps nothing. A second checkout linking the same path is left with a
|
||||
dangling symlink, there being only ever one stored copy.
|
||||
|
||||
Every stored path is in one of three states:
|
||||
|
||||
```
|
||||
$ ahab link list
|
||||
store: /home/you/.local/share/ahab/git.aflabs.org/urnik/afurnik
|
||||
linked: .env
|
||||
shadowed: config.local.py
|
||||
missing: secrets/token
|
||||
```
|
||||
|
||||
- `linked` — the symlink is there and points at the store
|
||||
- `shadowed` — a real file took the path back, so nothing reads the stored copy
|
||||
- `missing` — the symlink is gone, and nothing reads the stored copy either
|
||||
|
||||
Only a linked path can be moved back, so `restore --all` says how many it left
|
||||
alone.
|
||||
|
||||
Adding refuses two kinds of path:
|
||||
|
||||
- **a tracked path**, which git would restore anyway. Where one is a symlink
|
||||
leading out of the repository, `check` reports it under the code `T>`
|
||||
- **a symlink already leading out of the repository**, since moving it would
|
||||
move the pointer and leave the contents where they are
|
||||
|
||||
For scripts there is `--porcelain`, and `-z` for filenames that need it:
|
||||
entries end with a NUL, and the two paths of a symlink leading elsewhere are
|
||||
separated by one as well, the way `git status -z` reports a rename.
|
||||
|
||||
The store lives under
|
||||
`${XDG_DATA_HOME:-$HOME/.local/share}/ahab/<host>/<owner>/<repo>/`, derived
|
||||
from the Git `origin` remote, or `_local/<checkout>` when there is no remote to
|
||||
name it after. Its directories are created `0700`. A remote or checkout name
|
||||
that needed characters replacing carries a short fingerprint of the original,
|
||||
so two of them cannot share a directory.
|
||||
|
||||
## What ahab makes of your project
|
||||
|
||||
Running `ahab status` prints the services it picked, the containers behind
|
||||
them, the credentials it would use and what the store holds. It reports
|
||||
whatever it cannot work out rather than stopping at the first problem:
|
||||
|
||||
```
|
||||
$ ahab status
|
||||
@@ -30,12 +150,14 @@ store: /home/you/.local/share/ahab/git.aflabs.org/urnik/afurnik
|
||||
`ahab link check` lists what a sandbox can still read
|
||||
```
|
||||
|
||||
## seeing what it runs
|
||||
## Seeing what it runs
|
||||
|
||||
`-v` prints each docker command as it runs, `--dry-run` prints the ones it would
|
||||
run without running them, and `-q` prints only what was asked for, dropping the
|
||||
progress along the way (compose's own progress included). All work before or
|
||||
after the subcommand.
|
||||
- `-v` — print each Docker command as it runs
|
||||
- `--dry-run` — print the commands it would run, without running them
|
||||
- `-q` — print only what was asked for, dropping the progress along the way,
|
||||
Compose's own included
|
||||
|
||||
All three work before or after the subcommand:
|
||||
|
||||
```bash
|
||||
ahab -v django migrate
|
||||
@@ -43,30 +165,50 @@ ahab --dry-run postgres import ./dump
|
||||
ahab -q postgres dump ./dump
|
||||
```
|
||||
|
||||
Nothing is written under `--dry-run`, filesystem included: `link add` and
|
||||
Nothing is written under `--dry-run`, the filesystem included: `link add` and
|
||||
`django make-command` say what they would do and leave the tree alone.
|
||||
|
||||
## exit codes
|
||||
## Service detection
|
||||
|
||||
- `0` — it ran and had nothing to report
|
||||
- `1` — it could not finish
|
||||
- `2` — the arguments were wrong
|
||||
- `4` — it ran fine and found something worth reporting, i.e.
|
||||
`link check --exit-code` with anything outside the store
|
||||
Services can be named anything; ahab matches on what they are:
|
||||
|
||||
## shell completion
|
||||
- **postgres** — the service whose image is a Postgres flavour, matching
|
||||
`postg`, `timescale`, `pgvector` or `citus`
|
||||
- **django** — the service that both builds an image and sets
|
||||
`DJANGO_SETTINGS_MODULE`. Where more than one does, a worker beside the web
|
||||
service say, the one publishing ports wins
|
||||
|
||||
`ahab completions <shell>` writes a completion script to stdout, for bash,
|
||||
elvish, fish, powershell or zsh:
|
||||
If nothing matches, or two candidates cannot be told apart, ahab says so and
|
||||
lists the services it looked at rather than guessing.
|
||||
|
||||
The role and database come from `POSTGRES_USER` and `POSTGRES_DB` on the
|
||||
detected service, falling back to `db` when neither is set. Both have to be
|
||||
names: a value holding an `=`, a URL or a leading dash is refused, libpq
|
||||
reading such a name as a whole connection string.
|
||||
|
||||
## The compose file
|
||||
|
||||
Compose finds its own file and ahab does not pass `-f`, so set Docker's own
|
||||
`COMPOSE_FILE` when the file is not in the working directory, including its
|
||||
`base.yaml:override.yaml` form. A project's `.env` is a good place for it,
|
||||
since Docker reads that too:
|
||||
|
||||
```
|
||||
COMPOSE_FILE=docker/docker-compose.yaml
|
||||
```
|
||||
|
||||
## Shell completion
|
||||
|
||||
Completion scripts are written to stdout, for Bash, Elvish, Fish, PowerShell
|
||||
or Zsh:
|
||||
|
||||
```bash
|
||||
ahab completions zsh > ~/.local/share/zsh/completions/_ahab
|
||||
eval "$(ahab completions zsh)" # or one line in .zshrc, never goes stale
|
||||
```
|
||||
|
||||
Completions are also generated during the build. They land in
|
||||
`target/*/build/*/out/` by default, and `SHELL_COMPLETIONS_DIR_<SHELL>`
|
||||
installs a single shell's file straight into place:
|
||||
They are generated during the build as well, landing in
|
||||
`target/*/build/*/out/`. To install a single shell's file straight into place:
|
||||
|
||||
```bash
|
||||
SHELL_COMPLETIONS_DIR_ZSH=~/.local/share/zsh/completions \
|
||||
@@ -74,170 +216,20 @@ SHELL_COMPLETIONS_DIR_FISH=~/.config/fish/completions \
|
||||
cargo install --path .
|
||||
```
|
||||
|
||||
`SHELL_COMPLETIONS_DIR` writes every shell into one directory instead.
|
||||
Set `SHELL_COMPLETIONS_DIR` to write every shell into one directory instead.
|
||||
Either way the path has to be absolute.
|
||||
|
||||
## the compose file
|
||||
## Exit codes
|
||||
|
||||
`ahab` does not pass `-f`. docker compose finds the file itself, so set
|
||||
docker's own `COMPOSE_FILE` when it is not in the working directory, including
|
||||
its `base.yaml:override.yaml` form. A project's `.env` is a good place for it,
|
||||
since docker reads that too:
|
||||
- `0` — it ran and had nothing to report
|
||||
- `1` — it could not finish
|
||||
- `2` — the arguments were wrong
|
||||
- `4` — it ran fine and found something worth reporting, which today means
|
||||
`link check --exit-code` with anything outside the store
|
||||
|
||||
```
|
||||
COMPOSE_FILE=docker/docker-compose.yaml
|
||||
```
|
||||
## Environment variables
|
||||
|
||||
## service detection
|
||||
|
||||
`ahab` finds the services it needs in `docker compose config`, so they can be
|
||||
named anything:
|
||||
|
||||
- **postgres**: the service whose image is a postgres flavour, matching
|
||||
`postg`, `timescale`, `pgvector` or `citus`
|
||||
- **django**: the service that both builds an image and sets
|
||||
`DJANGO_SETTINGS_MODULE`. Where more than one does — a worker beside the web
|
||||
service, say — the one publishing ports wins
|
||||
|
||||
If nothing matches, or two candidates cannot be told apart, `ahab` says so and
|
||||
lists the services it looked at rather than guessing. Two services that both
|
||||
build and set the settings module are ambiguous unless exactly one of them
|
||||
publishes ports.
|
||||
|
||||
`POSTGRES_USER` and `POSTGRES_DB` are read off the detected postgres service, so
|
||||
`dropdb`, `createdb`, `pg_restore` and `pg_dump` use the role and database the
|
||||
project declares, falling back to `db` when it declares neither. Both have to
|
||||
be names: `ahab` hands them to those tools as a role and a database, and libpq
|
||||
reads a database name holding an `=` or a url as a whole connection string,
|
||||
which would send a dump to whatever server it names. A value that could be read
|
||||
as something other than a name is refused rather than passed on.
|
||||
|
||||
## django
|
||||
|
||||
```bash
|
||||
ahab django run <cmd> # in a fresh container, through the entrypoint
|
||||
ahab django bash # shell in a fresh container
|
||||
ahab django manage <args> # manage.py
|
||||
ahab django makemigrations
|
||||
ahab django migrate <args>
|
||||
ahab django shell
|
||||
ahab django make-command <app> <name>
|
||||
```
|
||||
|
||||
There is no `ahab django test`: which runner a project uses is the project's
|
||||
choice, and `manage.py test` exits 0 having collected nothing when the tests are
|
||||
written for pytest, so a wrong guess reads as a pass. Name the runner instead:
|
||||
|
||||
```bash
|
||||
ahab django run pytest -x tests/
|
||||
```
|
||||
|
||||
## postgres
|
||||
|
||||
```bash
|
||||
ahab postgres dump <path> # pg_dump, custom format
|
||||
ahab postgres dump -F plain <path> # pg_dump, plain sql
|
||||
ahab postgres dump -F tar <path> # pg_dump, tar
|
||||
ahab postgres dump -F directory <d> # pg_dump, a directory of files
|
||||
ahab postgres dump -F cluster <path> # pg_dumpall, roles and all databases
|
||||
ahab postgres dump -F plain -z <path> # any of them, gzipped
|
||||
ahab postgres import <path> # drop, create, then restore
|
||||
ahab postgres psql <args> # psql in the database container
|
||||
```
|
||||
|
||||
A dump is written `0600` and under a name of its own until it is complete, then
|
||||
renamed over the target: a cluster dump carries every role's password hash, and
|
||||
the default `0644` would hand it to anyone else with an account on the machine.
|
||||
|
||||
`psql` passes its arguments through and only asks docker for a terminal when it
|
||||
has one to hand over, so both of these work:
|
||||
|
||||
```bash
|
||||
ahab postgres psql # interactive
|
||||
ahab postgres psql -tAc 'select count(*) from auth_user' | wc -l
|
||||
```
|
||||
|
||||
The format of a dump being imported is read from the file rather than its name.
|
||||
A custom format dump starts with `PGDMP` and a tar one with `toc.dat`, both of
|
||||
which go to `pg_restore`, as does a directory produced by `pg_dump -Fd`.
|
||||
Anything else is treated as sql and fed to `psql` with `ON_ERROR_STOP` and
|
||||
`--single-transaction`, so a bad file rolls back instead of half applying.
|
||||
A whole cluster dump from `pg_dumpall` is recognised by its header and handled
|
||||
differently again: it creates its own databases and carries role statements, so
|
||||
the database is dropped but not recreated, the dump goes to `psql` connected to
|
||||
`postgres`, and it runs without `ON_ERROR_STOP` because roles that already exist
|
||||
report errors that are expected. Any error it does not expect is reported and
|
||||
fails the import, since psql without `ON_ERROR_STOP` exits 0 having applied
|
||||
only part of the dump. Gzipped dumps are decompressed on the way in, whichever
|
||||
of the three they hold. A cluster that has never held the database yet is a
|
||||
valid target either way, so importing into a fresh one works.
|
||||
|
||||
`import` stops the project before it starts, so an import that fails leaves it
|
||||
stopped and says so: `docker compose up` brings it back.
|
||||
|
||||
## link
|
||||
|
||||
`ahab link` moves untracked paths out of the repository into an out-of-repo
|
||||
store and symlinks them back, so a sandbox that mounts the repository sees a
|
||||
dangling symlink instead of the contents, while the host resolves it as before.
|
||||
|
||||
```bash
|
||||
ahab link add .env secrets/ # move out, leave symlinks behind
|
||||
ahab link restore .env # move it back into the repository
|
||||
ahab link restore --all # everything this repo has linked
|
||||
ahab link list # what the store holds for this repository
|
||||
ahab link check # what a sandbox can still read
|
||||
ahab link check --porcelain # `<code> <path>`, for scripts
|
||||
ahab link check --exit-code # exit 4 when anything is outside the store
|
||||
```
|
||||
|
||||
`restore` is the inverse of `add`: the file moves out of the store and back to
|
||||
where it was, and the store keeps nothing. A second checkout linking the same
|
||||
path is left with a dangling symlink, since there is only ever one stored copy.
|
||||
|
||||
`list` says what state each stored path is in, which `check` cannot see because
|
||||
it asks git about the repository rather than reading the store:
|
||||
|
||||
```
|
||||
$ ahab link list
|
||||
store: /home/you/.local/share/ahab/git.aflabs.org/urnik/afurnik
|
||||
linked: .env
|
||||
shadowed: config.local.py
|
||||
missing: secrets/token
|
||||
```
|
||||
|
||||
`missing` is a path whose symlink is gone, `shadowed` one that a real file took
|
||||
back; either way the stored copy is the one nobody reads.
|
||||
|
||||
`restore --all` moves back the linked ones and says how many it left, since
|
||||
there is nothing to undo for a path whose symlink is gone.
|
||||
|
||||
`-z` is the porcelain format for scripts that must survive any filename: entries
|
||||
end with a NUL, and the two paths of a symlink leading elsewhere are separated
|
||||
by one as well, the way `git status -z` reports a rename.
|
||||
|
||||
`check` looks at tracked paths too, since git tracks symlinks and one can lead
|
||||
out of the repository without appearing in any untracked listing. `add` cannot
|
||||
externalize a tracked path, so all `check` can do is report it, under the code
|
||||
`T>`.
|
||||
|
||||
A symlink already leading out of the repository is not something `add` will
|
||||
take: moving the link would move the pointer and leave the contents where they
|
||||
are, so the store would hold a way back out while `check`, seeing a link into
|
||||
the store, called the repository clean.
|
||||
|
||||
The store lives under
|
||||
`${XDG_DATA_HOME:-$HOME/.local/share}/ahab/<host>/<owner>/<repo>/`, derived
|
||||
from the git `origin` remote, or `_local/<checkout>` when there is no remote to
|
||||
name it after. Its directories are created `0700`: it exists to hold what
|
||||
should not be readable from the repository, and on a shared machine the default
|
||||
`0755` would leave that to whoever else has an account. A store path derived
|
||||
from a remote, or a checkout name, that needed characters replacing carries a
|
||||
short fingerprint of the original, so two of them cannot land on one directory.
|
||||
|
||||
## configuration
|
||||
|
||||
Currently `ahab` respects the following environment variables.
|
||||
|
||||
- `AHAB_LINK_ROOT`: where `ahab link` keeps its store - the same as `--store`,
|
||||
which takes precedence, and defaults to `${XDG_DATA_HOME:-$HOME/.local/share}/ahab`.
|
||||
`ahab status` reads it too, so it reports the store the link commands use
|
||||
- `AHAB_LINK_ROOT` — where `ahab link` keeps its store, the same as `--store`,
|
||||
which takes precedence. Defaults to
|
||||
`${XDG_DATA_HOME:-$HOME/.local/share}/ahab`. Read by `ahab status` too, so it
|
||||
reports the store the link commands actually use
|
||||
|
||||
Reference in New Issue
Block a user