Compare commits
1 Commits
v0.5.0
...
51e940ef9f
| Author | SHA1 | Date | |
|---|---|---|---|
|
51e940ef9f
|
336
README.md
336
README.md
@@ -3,119 +3,26 @@
|
||||
A wrapper around `docker compose` for our dockerized Django projects, so the
|
||||
same commands work in every repository.
|
||||
|
||||
It reads `docker compose config` to work out which service runs Django and
|
||||
which one is the database, so nothing here takes a service name or a `-f`.
|
||||
Everything it does, it does by running `docker`, `git` and the Postgres tools.
|
||||
|
||||
## Installing
|
||||
|
||||
You will need Rust installed. Clone repo and run:
|
||||
You will need Rust installed. Clone the repo and run:
|
||||
|
||||
```bash
|
||||
cargo install --path .
|
||||
```
|
||||
|
||||
## What it makes of a project
|
||||
|
||||
`ahab status` prints the services it picked, the containers behind them, the
|
||||
credentials it would use and what the store holds, reporting whatever it cannot
|
||||
work out rather than stopping at the first thing:
|
||||
|
||||
```
|
||||
$ ahab status
|
||||
services: db, django
|
||||
django: django (built from the project)
|
||||
container: 0a8b19e27634
|
||||
postgres: db (postgres:18-alpine)
|
||||
container: 189d4d395d62
|
||||
user: myproject
|
||||
database: myproject_db
|
||||
store: /home/you/.local/share/ahab/git.aflabs.org/urnik/afurnik
|
||||
linked: 2
|
||||
`ahab link check` lists what a sandbox can still read
|
||||
```
|
||||
|
||||
## Seeing what it runs
|
||||
|
||||
`-v` prints each Docker command as it runs, `--dry-run` prints the ones it would
|
||||
run without running them, and `-q` prints only what was asked for, dropping the
|
||||
progress along the way (Compose's own progress included). All work before or
|
||||
after the subcommand.
|
||||
|
||||
```bash
|
||||
ahab -v django migrate
|
||||
ahab --dry-run postgres import ./dump
|
||||
ahab -q postgres dump ./dump
|
||||
```
|
||||
|
||||
Nothing is written under `--dry-run`, filesystem included: `link add` and
|
||||
`django make-command` say what they would do and leave the tree alone.
|
||||
|
||||
## Exit codes
|
||||
|
||||
- `0` — It ran and had nothing to report
|
||||
- `1` — It could not finish
|
||||
- `2` — The arguments were wrong
|
||||
- `4` — It ran fine and found something worth reporting, i.e.
|
||||
`link check --exit-code` with anything outside the store
|
||||
|
||||
## Shell completion
|
||||
|
||||
`ahab completions <shell>` writes a completion script to stdout, for Bash,
|
||||
Elvish, Fish, PowerShell or Zsh:
|
||||
|
||||
```bash
|
||||
ahab completions zsh > ~/.local/share/zsh/completions/_ahab
|
||||
eval "$(ahab completions zsh)" # or one line in .zshrc, never goes stale
|
||||
```
|
||||
|
||||
Completions are also generated during the build. They land in
|
||||
`target/*/build/*/out/` by default, and `SHELL_COMPLETIONS_DIR_<SHELL>`
|
||||
installs a single shell's file straight into place:
|
||||
|
||||
```bash
|
||||
SHELL_COMPLETIONS_DIR_ZSH=~/.local/share/zsh/completions \
|
||||
SHELL_COMPLETIONS_DIR_FISH=~/.config/fish/completions \
|
||||
cargo install --path .
|
||||
```
|
||||
|
||||
`SHELL_COMPLETIONS_DIR` writes every shell into one directory instead.
|
||||
|
||||
## The compose file
|
||||
|
||||
`ahab` does not pass `-f`. Docker Compose finds the file itself, so set
|
||||
Docker's own `COMPOSE_FILE` when it is not in the working directory, including
|
||||
its `base.yaml:override.yaml` form. A project's `.env` is a good place for it,
|
||||
since Docker reads that too:
|
||||
|
||||
```
|
||||
COMPOSE_FILE=docker/docker-compose.yaml
|
||||
```
|
||||
|
||||
## Service detection
|
||||
|
||||
`ahab` finds the services it needs in `docker compose config`, so they can be
|
||||
named anything:
|
||||
|
||||
- **postgres**: The service whose image is a Postgres flavour, matching
|
||||
`postg`, `timescale`, `pgvector` or `citus`
|
||||
- **django**: The service that both builds an image and sets
|
||||
`DJANGO_SETTINGS_MODULE`. Where more than one does — a worker beside the web
|
||||
service, say — the one publishing ports wins
|
||||
|
||||
If nothing matches, or two candidates cannot be told apart, `ahab` says so and
|
||||
lists the services it looked at rather than guessing. Two services that both
|
||||
build and set the settings module are ambiguous unless exactly one of them
|
||||
publishes ports.
|
||||
|
||||
`POSTGRES_USER` and `POSTGRES_DB` are read off the detected Postgres service, so
|
||||
`dropdb`, `createdb`, `pg_restore` and `pg_dump` use the role and database the
|
||||
project declares, falling back to `db` when it declares neither. Both have to
|
||||
be names: `ahab` hands them to those tools as a role and a database, and libpq
|
||||
reads a database name holding an `=` or a URL as a whole connection string,
|
||||
which would send a dump to whatever server it names. A value that could be read
|
||||
as something other than a name is refused rather than passed on.
|
||||
|
||||
## Django
|
||||
|
||||
Each of these runs `docker compose run --rm` against the Django service, so it
|
||||
goes through the image's own entrypoint:
|
||||
|
||||
```bash
|
||||
ahab django run <cmd> # in a fresh container, through the entrypoint
|
||||
ahab django bash # shell in a fresh container
|
||||
ahab django run <cmd> # anything, in a fresh container
|
||||
ahab django bash # a shell in a fresh container
|
||||
ahab django manage <args> # manage.py
|
||||
ahab django makemigrations
|
||||
ahab django migrate <args>
|
||||
@@ -123,9 +30,10 @@ ahab django shell
|
||||
ahab django make-command <app> <name>
|
||||
```
|
||||
|
||||
There is no `ahab django test`: which runner a project uses is the project's
|
||||
choice, and `manage.py test` exits 0 having collected nothing when the tests are
|
||||
written for pytest, so a wrong guess reads as a pass. Name the runner instead:
|
||||
There is no `ahab django test`. Which runner a project uses is the project's
|
||||
choice, and `manage.py test` exits 0 having collected nothing when the tests
|
||||
are written for pytest, so a wrong guess reads as a pass. Name the runner
|
||||
instead:
|
||||
|
||||
```bash
|
||||
ahab django run pytest -x tests/
|
||||
@@ -144,41 +52,45 @@ ahab postgres import <path> # drop, create, then restore
|
||||
ahab postgres psql <args> # psql in the database container
|
||||
```
|
||||
|
||||
A dump is written `0600` and under a name of its own until it is complete, then
|
||||
renamed over the target: a cluster dump carries every role's password hash, and
|
||||
the default `0644` would hand it to anyone else with an account on the machine.
|
||||
Dumps are written `0600`, under a name of their own until they are complete and
|
||||
then renamed over the target. A cluster dump carries every role's password
|
||||
hash, so the default `0644` would hand it to anyone else with an account.
|
||||
|
||||
`psql` passes its arguments through and only asks Docker for a terminal when it
|
||||
has one to hand over, so both of these work:
|
||||
Arguments reach `psql` untouched, and Docker is only asked for a terminal when
|
||||
there is one to hand over, so both of these work:
|
||||
|
||||
```bash
|
||||
ahab postgres psql # interactive
|
||||
ahab postgres psql -tAc 'select count(*) from auth_user' | wc -l
|
||||
```
|
||||
|
||||
The format of a dump being imported is read from the file rather than its name.
|
||||
A custom format dump starts with `PGDMP` and a tar one with `toc.dat`, both of
|
||||
which go to `pg_restore`, as does a directory produced by `pg_dump -Fd`.
|
||||
Anything else is treated as SQL and fed to `psql` with `ON_ERROR_STOP` and
|
||||
`--single-transaction`, so a bad file rolls back instead of half applying.
|
||||
A whole cluster dump from `pg_dumpall` is recognised by its header and handled
|
||||
differently again: it creates its own databases and carries role statements, so
|
||||
the database is dropped but not recreated, the dump goes to `psql` connected to
|
||||
`postgres`, and it runs without `ON_ERROR_STOP` because roles that already exist
|
||||
report errors that are expected. Any error it does not expect is reported and
|
||||
fails the import, since `psql` without `ON_ERROR_STOP` exits 0 having applied
|
||||
only part of the dump. Gzipped dumps are decompressed on the way in, whichever
|
||||
of the three they hold. A cluster that has never held the database yet is a
|
||||
valid target either way, so importing into a fresh one works.
|
||||
### What an import makes of a dump
|
||||
|
||||
`import` stops the project before it starts, so an import that fails leaves it
|
||||
stopped and says so: `docker compose up` brings it back.
|
||||
The shape is read from the file rather than from its name:
|
||||
|
||||
- a `PGDMP` or `toc.dat` header — an archive, restored with `pg_restore`, as is
|
||||
a directory produced by `pg_dump -Fd`
|
||||
- a `pg_dumpall` header — a whole cluster, which creates its own databases and
|
||||
carries role statements, so the database is dropped but not recreated and the
|
||||
dump goes to `psql` connected to `postgres`
|
||||
- gzip magic — decompressed on the way in, whichever of the three is inside
|
||||
- anything else — SQL, fed to `psql` with `ON_ERROR_STOP` and
|
||||
`--single-transaction`, so a bad file rolls back instead of half applying
|
||||
|
||||
A cluster restore is the one that runs without `ON_ERROR_STOP`, because roles
|
||||
that already exist report errors which are expected. Any other error fails the
|
||||
import, since psql would otherwise exit 0 having applied part of the dump.
|
||||
Importing into a cluster that has never held the database works either way.
|
||||
|
||||
Containers are stopped before an import starts, so one that fails leaves the
|
||||
project stopped and says so.
|
||||
|
||||
## Link
|
||||
|
||||
`ahab link` moves untracked paths out of the repository into an out-of-repo
|
||||
store and symlinks them back, so a sandbox that mounts the repository sees a
|
||||
dangling symlink instead of the contents, while the host resolves it as before.
|
||||
A sandbox that mounts the repository can read anything untracked sitting in it
|
||||
— a `.env`, a dump, a key. Those paths can be moved out to a store of ahab's
|
||||
own and symlinked back, so the host resolves the link as before while the
|
||||
sandbox is left with a symlink into somewhere it was never given.
|
||||
|
||||
```bash
|
||||
ahab link add .env secrets/ # move out, leave symlinks behind
|
||||
@@ -190,12 +102,12 @@ ahab link check --porcelain # `<code> <path>`, for scripts
|
||||
ahab link check --exit-code # exit 4 when anything is outside the store
|
||||
```
|
||||
|
||||
`restore` is the inverse of `add`: the file moves out of the store and back to
|
||||
where it was, and the store keeps nothing. A second checkout linking the same
|
||||
path is left with a dangling symlink, since there is only ever one stored copy.
|
||||
Restoring is the inverse of adding: the file comes back to where it was and the
|
||||
store keeps nothing. A second checkout linking the same path is left with a
|
||||
dangling symlink, there being only ever one stored copy.
|
||||
|
||||
`list` says what state each stored path is in, which `check` cannot see because
|
||||
it asks Git about the repository rather than reading the store:
|
||||
Every stored path is in one of three states, which `list` reads from the store
|
||||
rather than from git:
|
||||
|
||||
```
|
||||
$ ahab link list
|
||||
@@ -205,39 +117,141 @@ store: /home/you/.local/share/ahab/git.aflabs.org/urnik/afurnik
|
||||
missing: secrets/token
|
||||
```
|
||||
|
||||
`missing` is a path whose symlink is gone, `shadowed` one that a real file took
|
||||
back; either way the stored copy is the one nobody reads.
|
||||
- `linked` — the symlink is there and points at the store, which is the point
|
||||
- `shadowed` — a real file took the path back, so nothing reads the stored copy
|
||||
- `missing` — the symlink is gone, and nothing reads the stored copy either
|
||||
|
||||
`restore --all` moves back the linked ones and says how many it left, since
|
||||
there is nothing to undo for a path whose symlink is gone.
|
||||
Only a linked path can be moved back, so `restore --all` says how many it left
|
||||
alone.
|
||||
|
||||
`-z` is the porcelain format for scripts that must survive any filename: entries
|
||||
end with a NUL, and the two paths of a symlink leading elsewhere are separated
|
||||
by one as well, the way `git status -z` reports a rename.
|
||||
Two things adding will not do:
|
||||
|
||||
`check` looks at tracked paths too, since Git tracks symlinks and one can lead
|
||||
out of the repository without appearing in any untracked listing. `add` cannot
|
||||
externalize a tracked path, so all `check` can do is report it, under the code
|
||||
`T>`.
|
||||
- **a tracked path**, which git would restore into the repository anyway. Where
|
||||
one is a symlink leading out, `check` reports it under the code `T>`, and
|
||||
saying so is all it can do
|
||||
- **a symlink already leading out of the repository**, because moving the link
|
||||
would move the pointer and leave the contents where they are. The store would
|
||||
then hold a way back out, while `check` saw a link into the store and called
|
||||
the repository clean
|
||||
|
||||
A symlink already leading out of the repository is not something `add` will
|
||||
take: moving the link would move the pointer and leave the contents where they
|
||||
are, so the store would hold a way back out while `check`, seeing a link into
|
||||
the store, called the repository clean.
|
||||
For scripts there is `--porcelain`, and `-z` for filenames that need it:
|
||||
entries end with a NUL, and the two paths of a symlink leading elsewhere are
|
||||
separated by one as well, the way `git status -z` reports a rename.
|
||||
|
||||
The store lives under
|
||||
`${XDG_DATA_HOME:-$HOME/.local/share}/ahab/<host>/<owner>/<repo>/`, derived
|
||||
from the Git `origin` remote, or `_local/<checkout>` when there is no remote to
|
||||
name it after. Its directories are created `0700`: it exists to hold what
|
||||
should not be readable from the repository, and on a shared machine the default
|
||||
`0755` would leave that to whoever else has an account. A store path derived
|
||||
from a remote, or a checkout name, that needed characters replacing carries a
|
||||
short fingerprint of the original, so two of them cannot land on one directory.
|
||||
name it after. Two details of that:
|
||||
|
||||
## Configuration
|
||||
- its directories are created `0700`, since it holds what should not be
|
||||
readable from the repository and `0755` would leave that to whoever else has
|
||||
an account on the machine
|
||||
- a remote or checkout name that needed characters replacing carries a short
|
||||
fingerprint of the original, so two of them cannot land on one directory
|
||||
|
||||
Currently `ahab` respects the following environment variables.
|
||||
## What ahab makes of your project
|
||||
|
||||
- `AHAB_LINK_ROOT`: Where `ahab link` keeps its store — the same as `--store`,
|
||||
which takes precedence, and defaults to `${XDG_DATA_HOME:-$HOME/.local/share}/ahab`.
|
||||
`ahab status` reads it too, so it reports the store the link commands use
|
||||
Running `ahab status` prints the services it picked, the containers behind
|
||||
them, the credentials it would use and what the store holds. It reports
|
||||
whatever it cannot work out rather than stopping at the first thing:
|
||||
|
||||
```
|
||||
$ ahab status
|
||||
services: db, django
|
||||
django: django (built from the project)
|
||||
container: 0a8b19e27634
|
||||
postgres: db (postgres:18-alpine)
|
||||
container: 189d4d395d62
|
||||
user: myproject
|
||||
database: myproject_db
|
||||
store: /home/you/.local/share/ahab/git.aflabs.org/urnik/afurnik
|
||||
linked: 2
|
||||
`ahab link check` lists what a sandbox can still read
|
||||
```
|
||||
|
||||
## Seeing what it runs
|
||||
|
||||
- `-v` — print each Docker command as it runs
|
||||
- `--dry-run` — print the commands it would run, without running them
|
||||
- `-q` — print only what was asked for, dropping the progress along the way,
|
||||
Compose's own included
|
||||
|
||||
All three work before or after the subcommand:
|
||||
|
||||
```bash
|
||||
ahab -v django migrate
|
||||
ahab --dry-run postgres import ./dump
|
||||
ahab -q postgres dump ./dump
|
||||
```
|
||||
|
||||
Nothing is written under `--dry-run`, the filesystem included: `link add` and
|
||||
`django make-command` say what they would do and leave the tree alone.
|
||||
|
||||
## Service detection
|
||||
|
||||
Services can be named anything, since ahab looks at what they are rather than
|
||||
what they are called:
|
||||
|
||||
- **postgres** — the service whose image is a Postgres flavour, matching
|
||||
`postg`, `timescale`, `pgvector` or `citus`
|
||||
- **django** — the service that both builds an image and sets
|
||||
`DJANGO_SETTINGS_MODULE`. Where more than one does, a worker beside the web
|
||||
service say, the one publishing ports wins
|
||||
|
||||
If nothing matches, or two candidates cannot be told apart, ahab says so and
|
||||
lists the services it looked at rather than guessing.
|
||||
|
||||
The role and database come from `POSTGRES_USER` and `POSTGRES_DB` on the
|
||||
detected service, falling back to `db` when neither is set. Both have to be
|
||||
names: libpq reads a database name holding an `=` or a URL as a whole
|
||||
connection string, which would send a dump to whatever server that names, so a
|
||||
value which could be read as anything but a name is refused.
|
||||
|
||||
## The compose file
|
||||
|
||||
Compose finds its own file and ahab does not pass `-f`, so set Docker's own
|
||||
`COMPOSE_FILE` when the file is not in the working directory, including its
|
||||
`base.yaml:override.yaml` form. A project's `.env` is a good place for it,
|
||||
since Docker reads that too:
|
||||
|
||||
```
|
||||
COMPOSE_FILE=docker/docker-compose.yaml
|
||||
```
|
||||
|
||||
## Shell completion
|
||||
|
||||
Completion scripts are written to stdout, for Bash, Elvish, Fish, PowerShell
|
||||
or Zsh:
|
||||
|
||||
```bash
|
||||
ahab completions zsh > ~/.local/share/zsh/completions/_ahab
|
||||
eval "$(ahab completions zsh)" # or one line in .zshrc, never goes stale
|
||||
```
|
||||
|
||||
They are generated during the build as well, landing in
|
||||
`target/*/build/*/out/`. To install a single shell's file straight into place:
|
||||
|
||||
```bash
|
||||
SHELL_COMPLETIONS_DIR_ZSH=~/.local/share/zsh/completions \
|
||||
SHELL_COMPLETIONS_DIR_FISH=~/.config/fish/completions \
|
||||
cargo install --path .
|
||||
```
|
||||
|
||||
Set `SHELL_COMPLETIONS_DIR` to write every shell into one directory instead.
|
||||
Either way the path has to be absolute, since a build script runs from wherever
|
||||
cargo happens to be building.
|
||||
|
||||
## Exit codes
|
||||
|
||||
- `0` — it ran and had nothing to report
|
||||
- `1` — it could not finish
|
||||
- `2` — the arguments were wrong
|
||||
- `4` — it ran fine and found something worth reporting, which today means
|
||||
`link check --exit-code` with anything outside the store
|
||||
|
||||
## Environment variables
|
||||
|
||||
- `AHAB_LINK_ROOT` — where `ahab link` keeps its store, the same as `--store`,
|
||||
which takes precedence. Defaults to
|
||||
`${XDG_DATA_HOME:-$HOME/.local/share}/ahab`. Read by `ahab status` too, so it
|
||||
reports the store the link commands actually use
|
||||
|
||||
Reference in New Issue
Block a user