97 Commits

Author SHA1 Message Date
49d9df64ae chore: release v0.5.0 2026-09-09 15:43:26 +02:00
19b34cd946 merge: structural code changes 2026-09-09 15:41:44 +02:00
c55273d80c refactor: one helper each for the repeated blocks 2026-09-09 15:41:30 +02:00
5dbe99a304 refactor: hand clap's own structs to the commands 2026-09-09 15:41:30 +02:00
b40dfe6124 refactor: put the dry-run and quiet checks on Ctx 2026-09-09 15:41:30 +02:00
d9439253fd test: cover the store and the exit codes before moving any of it 2026-09-09 15:41:28 +02:00
2563d56595 merge: review fixes 2026-09-09 15:41:00 +02:00
5d121b13b5 docs: say what changed about names, modes and what check looks at 2026-09-09 15:40:43 +02:00
462d9cbec2 chore: stop sharing a cache with the binary the audit job runs 2026-09-09 15:40:42 +02:00
b6a6750499 fix: dumps and imports that quietly did the wrong thing 2026-09-09 15:40:41 +02:00
6b2d86ff5a fix: make the link store hold what it says it holds 2026-09-09 15:40:40 +02:00
e146b0460a fix: keep a path's bytes, so a filename that is not utf-8 survives 2026-09-09 15:40:39 +02:00
b4b6d5918f fix: report what happened instead of losing it 2026-09-09 15:40:33 +02:00
75f293ce36 docs: bring the readme up to date with the commands that exist 2026-09-08 17:44:52 +02:00
b224519952 merge: qol improvements 2026-09-08 17:35:32 +02:00
d7a54c70f9 feat!: report findings with exit code 4, not the code for failure 2026-09-08 17:35:27 +02:00
23b7fde0ea fix: separate the paths of a -z entry with a NUL, not an arrow 2026-09-08 17:35:27 +02:00
b372472de6 feat!: drop django test, which only ever ran one of the runners 2026-09-08 17:35:26 +02:00
99010cb641 fix: end quietly when a reader leaves the pipe early 2026-09-08 17:35:25 +02:00
1ecf7f1ee4 feat: show what ahab makes of this project 2026-09-08 17:35:25 +02:00
79ef5cca2d feat: list what the store holds for this repository 2026-09-08 17:35:24 +02:00
6389f3f1f5 feat: open psql in the database container 2026-09-08 17:35:23 +02:00
61f35f5ae4 feat: silence progress with --quiet 2026-09-08 17:35:22 +02:00
cabbbde438 fix: import into a cluster that has no such database yet 2026-09-08 17:35:19 +02:00
5239e000d7 merge: code reorg 2026-09-08 15:43:51 +02:00
bb0a2cca44 refactor: name the modules after what they hold 2026-09-08 15:43:45 +02:00
a3c08f6eaa refactor: put every filesystem write behind one module 2026-09-08 15:43:44 +02:00
bbbca755be refactor: run git through the same layer as everything else 2026-09-08 15:43:43 +02:00
d7626d9f5e refactor: give every command we run a type of its own 2026-09-08 15:43:42 +02:00
3d06f7dcb0 refactor: pass invocation options down instead of reading a global 2026-09-08 15:43:41 +02:00
056da8b9b2 refactor: build one binary instead of a library and a binary 2026-09-08 15:43:37 +02:00
27c6b261a6 merge: bug fixes 2026-09-08 13:18:24 +02:00
afa0b65c13 fix: stop --dry-run from writing to the working tree 2026-09-08 13:18:17 +02:00
e1817a6824 fix: fail a dump when its first pipeline stage dies 2026-09-08 13:18:11 +02:00
432fa12c53 fix: prune the store the caller asked for 2026-09-08 13:17:58 +02:00
01b0b5c2dd merge: cli modernization 2026-09-08 12:01:34 +02:00
876e239769 chore: declare lints in the manifest 2026-09-08 12:00:58 +02:00
061b77f619 feat: colour the help, and check the command definition 2026-09-08 12:00:55 +02:00
4b1e9d68b0 feat: take the store root as an option 2026-09-08 12:00:52 +02:00
11ecd0ac89 refactor: return an exit code from main 2026-09-08 12:00:49 +02:00
aec7afb274 feat: hand back the command's exit code 2026-09-08 12:00:45 +02:00
3fa4739451 chore: update metadata 2026-09-08 11:46:21 +02:00
7bd2de6783 merge: link restore 2026-09-08 11:34:33 +02:00
260ef24174 feat: add link restore 2026-09-08 11:34:22 +02:00
ef3ad1a410 merge: general cli improvements 2026-09-08 11:27:40 +02:00
43d55b1003 style: normalise command help 2026-09-08 11:27:32 +02:00
15bcaafa85 feat: add exit-code to link check 2026-09-08 11:27:30 +02:00
b2a0c08256 feat: add verbose and dry run 2026-09-08 11:27:26 +02:00
ab93e7ef4f fix: give the readiness wait a timeout 2026-09-08 11:27:22 +02:00
a489d77b0a fix: write the dump to a temporary file first 2026-09-08 11:27:18 +02:00
1bb205bb79 feat: build commands from arguments instead of strings 2026-09-08 11:27:14 +02:00
25f7b7df9d feat: fail when a command fails 2026-09-08 11:27:09 +02:00
0aeecdcb54 feat: remove stop_all 2026-09-08 11:27:03 +02:00
9caf9599b3 merge: postgres dump formats 2026-09-08 11:13:36 +02:00
ad269308cc feat: dump in every format import understands 2026-09-08 11:13:22 +02:00
77703485b9 feat: support custom format, plain sql and cluster dumps 2026-09-08 11:12:49 +02:00
0f16949190 merge: compose subcommand deprecation 2026-09-07 15:14:41 +02:00
6e44527a9a feat: deprecate compose subcommand 2026-09-07 13:11:36 +00:00
d11b0c68cd feat: drop dotenvy 2026-09-07 13:11:36 +00:00
9d63233d9a feat: detect the django and postgres services
ahab guessed that the django service is called django and the postgres one db,
falling back on AHAB_DJANGO_CONTAINER and AHAB_POSTGRES_CONTAINER. The standard
project layout happens to agree, but nothing enforces it, and a stack naming
them web and database could not use ahab without setting both variables.

One `docker compose config --format json` call, roughly 120ms, resolves the
stack even while it is down, and the services are identified from what they
are rather than what they are called:

- postgres is the service whose image is a postgres flavour, matching postg,
  timescale, pgvector or citus. Nothing else counts.
- django is the service that both builds an image and has
  DJANGO_SETTINGS_MODULE in its environment. A celery worker sharing the same
  build and env_file matches too, so published ports break the tie: the service
  answering requests wins.

Neither guess is allowed to be wrong quietly. No match, or two candidates that
cannot be told apart, is an error naming the services it looked at. There is no
variable to fall back on: both container variables are gone, along with the
guessed defaults they backed up, so an ambiguous stack is fixed in the compose
file rather than worked around per developer.

Every service lookup goes through these rules, so `compose exec` and the whole
django group agree on which container they mean.

POSTGRES_USER and POSTGRES_DB come off the detected service, so dropdb,
createdb, pg_restore and pg_dump stop assuming the role and database are both
literally `db`, falling back to that only when the service declares neither.

Note that env_file entries are merged into a service's environment, so
POSTGRES_* is not safe for identifying the database service: one such line in a
project's .env would make the django service match as well. That is why
identification uses the image and only credentials use the environment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 13:11:35 +00:00
836bbaac08 feat: let docker compose find its own file 2026-09-07 15:06:17 +02:00
7d71dd054c docs: bring the readme up to date 2026-09-07 15:06:12 +02:00
d6bf3efb15 merge: completions improvements 2026-09-04 16:03:46 +02:00
ef19f49c63 feat: add completions subcommand 2026-09-04 16:03:35 +02:00
627812029d fix: install completions per shell, and re-run when asked 2026-09-04 16:03:08 +02:00
471169612b merge: maintenance 2026-09-04 11:46:35 +02:00
1afff02ee9 ci: add gitlab pipeline 2026-09-04 11:24:21 +02:00
8751c39cce style: run cargo fmt 2026-09-04 11:24:14 +02:00
e10e1fb017 chore: drop allow(unused) and clear what it hid 2026-09-04 11:24:02 +02:00
ec4168769e fix: keep existing contents in create_file 2026-09-04 11:23:52 +02:00
a301097120 refactor: drop unused CommandBuilder::new 2026-09-04 11:23:46 +02:00
3ed1fad4a7 chore: move to edition 2024 2026-09-04 11:23:42 +02:00
c52683e53c chore: upgrade dependencies 2026-09-04 11:23:08 +02:00
3dc3da4592 merge: link subcommand 2026-09-03 15:58:41 +02:00
54395f83e4 feat: add link command 2026-09-03 15:58:28 +02:00
63d04a9393 chore: release v0.4.2 2026-03-02 14:51:03 +01:00
f92227a679 chore: generalize build.sh 2026-03-02 14:50:43 +01:00
27a1ec6d2c chore(release): v0.4.1 2026-02-26 17:45:58 -05:00
1b1f9a31ed feat: add -L to docker cp 2026-02-26 17:45:28 -05:00
e5eddb39f7 chore(release): v0.4.0 2026-02-24 08:53:28 -05:00
2000204250 feat: update default service names 2026-02-24 08:52:08 -05:00
7912e9a516 chore(release): v0.3.2 2025-07-01 17:37:36 +02:00
568d126fc2 test: add imports to doc snippets 2025-07-01 17:36:36 +02:00
3ce4cbfad7 chore(release): v0.3.1 2025-02-12 16:53:28 +01:00
36fcc84881 feat: don't read DJANGO_SETTINGS_MODULE 2025-02-12 16:52:50 +01:00
c8b3a817f0 chore: correctly document env variables in README 2025-02-12 16:52:09 +01:00
f7737ea00d fix: dump from database db 2024-11-27 20:00:37 +01:00
f56965abc2 chore: add completion and configuration to readme 2024-11-27 18:48:20 +01:00
e7308bd5c6 feat: control default services via env vars
Read default postgres compose service name and default django compose
service name from env variables.

AHAB_POSTGRES_CONTAINER - defaults to `appserver`
AHAB_DJANGO_CONATINER - defaults to `postgresdb`

Default values are there for 100% backwards compatibility.
2024-11-27 18:13:12 +01:00
3077420db2 feat!: respect COMPOSE_FILE env and remove docker command
Read COMPOSE_FILE from env and use it if it is set. Default is still
docker/local/docker-compose.yaml.

BREAKING CHANGE: Removed docker subcommand. Added deprecation notice to
compose subcommand.
2024-10-25 09:37:13 +02:00
73a0b87c31 feat: swap println for eprintln or debug_eprintln
Unified expected output to always eprintln so outputs can be piped to
other processes. Added debug_eprintln and switched all occurances of
debug_println.
2024-10-25 09:03:49 +02:00
6fcf48ed61 feat: redirect pg_isready output to /dev/null
Fixed bug where restore commands were called inside pg_isready loop.
Made CommandBuilder::build public. Redirected stdout for pg_isready
calls to /dev/null via Stdio::null().
2023-11-27 20:58:28 +01:00
0b0ca06398 chore: update README with install info
Added install instructions to README; both normal and debug install.
2023-11-27 20:40:37 +01:00
98f02c1c08 feat: pg_isready in postgres import
Fixed issue where arguments passed to `compose stop` weren't passed to
the underlying command. Added pg_isready call between each command call
in restore script.
2023-11-27 20:40:24 +01:00
2754a3bb30 feat: compile time completion generation
Removed `completion` command in favor of compile time completion
generation. This tool will only be build from source for the foreseeable
future so this makes sense for now.
2023-08-09 14:45:07 +02:00
e061d6dc6c feat: add compose ps command
Ps is new command scoped under compose. It is just proxy to `docker
compose ps`.
2023-08-09 13:59:58 +02:00
b45cfd3e0a feat: add compose exec and compose bash commands
Exec is new command scoped under compose. It allows sending commands
directly into appserver container. I will make this
customizable/controllable in the future.
2023-06-15 19:58:43 +02:00
47 changed files with 5532 additions and 654 deletions

32
.gitignore vendored
View File

@@ -8,3 +8,35 @@ target/
# MSVC Windows builds of rustc generate these, which store debugging information
*.pdb
# the cargo home CI keeps inside the project so it lands in the cache
.cargo/
# where the completions build writes when it is not given a directory
completions/
# os
.DS_Store
# editors
.vscode
.idea
.ignore
# ai
CLAUDE.local.md
.claude
# languages
.python-version
# nix
.nix-venv
.envrc
.direnv/
# project-local
TODO.md
# database dumps, which this tool exists to keep out of a repository
dumps
*.partial

80
.gitlab-ci.yml Normal file
View File

@@ -0,0 +1,80 @@
default:
image: rust:1.97
cache:
key:
files:
- Cargo.lock
paths:
# no .cargo/bin here: the audit job runs what is in it, and this cache is
# shared by every branch and merge request, so a pipeline could otherwise
# leave behind the binary a later scheduled audit executes
- .cargo/registry
- target
variables:
# keep the registry inside the project so it lands in the cache
CARGO_HOME: $CI_PROJECT_DIR/.cargo
CARGO_TERM_COLOR: always
# pinned, rather than whatever version exists on the night this runs
CARGO_AUDIT_VERSION: "0.21.2"
# without this a push to a branch with an open merge request runs twice
workflow:
rules:
# first, because a scheduled pipeline sets CI_COMMIT_BRANCH as well and so
# would be matched by the branch rule below, or vetoed by the one above it
- if: $CI_PIPELINE_SOURCE == "schedule"
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH
check:
script:
- rustup component add rustfmt clippy
- cargo fmt --check
# --locked on the first cargo invocation too: without it clippy resolves the
# dependencies and rewrites a Cargo.lock that has drifted from Cargo.toml,
# and the --locked commands after it then pass against what it just wrote
- cargo clippy --locked --all-targets -- -D warnings
- cargo test --locked
- cargo build --release --locked
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- when: always
# runs only from a pipeline schedule, so set one up in the project settings
audit:
script:
- cargo install cargo-audit --version $CARGO_AUDIT_VERSION --locked
- cargo audit
# reports dependencies that have drifted behind, without changing the lockfile
- cargo update --dry-run
# its own cache, so the audit tool is not rebuilt on every schedule while
# still being written only by this schedule-only job. the pinned version is
# part of the key, so a bump fetches rather than reusing the old binary
cache:
key: audit-tools-$CARGO_AUDIT_VERSION
paths:
- .cargo/bin
- .cargo/registry
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
# the msrv declared in Cargo.toml, so it fails when something needs a newer rustc
msrv:
image: rust:1.85
script:
- cargo build --locked
# its own key: artifacts built by another rustc are of no use to this job, and
# sharing one only has the two toolchains taking turns overwriting it
cache:
key: msrv-$CI_COMMIT_REF_SLUG
paths:
- .cargo/registry
- target
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- when: always

295
Cargo.lock generated
View File

@@ -1,122 +1,117 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 3
version = 4
[[package]]
name = "ahab"
version = "0.3.0"
version = "0.4.2"
dependencies = [
"anyhow",
"clap",
"clap_complete",
"dotenvy",
"fs-err",
"serde_json",
"shlex",
]
[[package]]
name = "anstream"
version = "0.3.2"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ca84f3628370c59db74ee214b3263d58f9aadd9b4fe7e711fd87dc452b7f163"
checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
dependencies = [
"anstyle",
"anstyle-parse",
"anstyle-query",
"anstyle-wincon",
"colorchoice",
"is-terminal",
"is_terminal_polyfill",
"utf8parse",
]
[[package]]
name = "anstyle"
version = "1.0.0"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41ed9a86bf92ae6580e0a31281f65a1b1d867c0cc68d5346e2ae128dddfa6a7d"
checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
[[package]]
name = "anstyle-parse"
version = "0.2.0"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e765fd216e48e067936442276d1d57399e37bce53c264d6fefbe298080cb57ee"
checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
dependencies = [
"utf8parse",
]
[[package]]
name = "anstyle-query"
version = "1.0.0"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ca11d4be1bab0c8bc8734a9aa7bf4ee8316d462a08c6ac5052f888fef5b494b"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys",
]
[[package]]
name = "anstyle-wincon"
version = "1.0.1"
version = "3.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "180abfa45703aebe0093f79badacc01b8fd4ea2e35118747e5811127f926e188"
checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys",
]
[[package]]
name = "anyhow"
version = "1.0.71"
version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c7d0618f0e0b7e8ff11427422b64564d5fb0be1940354bfe2e0529b18a9d9b8"
checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470"
[[package]]
name = "bitflags"
version = "1.3.2"
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]]
name = "cc"
version = "1.0.79"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "50d30906286121d95be3d479533b458f87493b30a4b5f79a607db8f5d11aa91f"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "clap"
version = "4.3.0"
version = "4.6.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93aae7a4192245f70fe75dd9157fc7b4a5bf53e88d30bd4396f7d8f9284d5acc"
checksum = "473c7e07f409a8d772161724aa8db6a765a2532a70f9667eeb7b49d3d02fbdca"
dependencies = [
"clap_builder",
"clap_derive",
"once_cell",
]
[[package]]
name = "clap_builder"
version = "4.3.0"
version = "4.6.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f423e341edefb78c9caba2d9c7f7687d0e72e89df3ce3394554754393ac3990"
checksum = "7b48fea5a88e9ae728a2dcbedbfc0e730f7d60da42e1cb049a83c9fb8b789889"
dependencies = [
"anstream",
"anstyle",
"bitflags",
"clap_lex",
"strsim",
]
[[package]]
name = "clap_complete"
version = "4.3.0"
version = "4.6.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a04ddfaacc3bc9e6ea67d024575fafc2a813027cf374b8f24f7bc233c6b6be12"
checksum = "3be2ad0423bdbbb0e25bc89add796f3559706d4a95e1bc98e4d9662a957b6a19"
dependencies = [
"clap",
]
[[package]]
name = "clap_derive"
version = "4.3.0"
version = "4.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "191d9573962933b4027f932c600cd252ce27a8ad5979418fe78e43c07996f27b"
checksum = "d012d2b9d65aca7f18f4d9878a045bc17899bba951561ba5ec3c2ba1eed9a061"
dependencies = [
"heck",
"proc-macro2",
@@ -126,139 +121,132 @@ dependencies = [
[[package]]
name = "clap_lex"
version = "0.5.0"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2da6da31387c7e4ef160ffab6d5e7f00c42626fe39aea70a7b0f1773f7dd6c1b"
checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
[[package]]
name = "colorchoice"
version = "1.0.0"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "acbf1af155f9b9ef647e42cdc158db4b64a1b61f743629225fde6f3e0be2a7c7"
checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "dotenvy"
version = "0.15.7"
name = "fs-err"
version = "3.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1aaf95b3e5c8f23aa320147307562d361db0ae0d51242340f558153b4eb2439b"
[[package]]
name = "errno"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4bcfec3a70f97c962c307b2d2c56e358cf1d00b558d74262b5f929ee8cc7e73a"
checksum = "b91aa448ca50d7e79433bdf3ee8d99215430d2ec02ade5aefab2a073a1822e8a"
dependencies = [
"errno-dragonfly",
"libc",
"windows-sys",
]
[[package]]
name = "errno-dragonfly"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aa68f1b12764fab894d2755d2518754e71b4fd80ecfb822714a1206c2aab39bf"
dependencies = [
"cc",
"libc",
"autocfg",
]
[[package]]
name = "heck"
version = "0.4.1"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "95505c38b4572b2d910cecb0281560f54b440a19336cbbcb27bf6ce6adc6f5a8"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hermit-abi"
version = "0.3.1"
name = "is_terminal_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fed44880c466736ef9a5c5b5facefb5ed0785676d0c02d612db14e54f0d84286"
checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
[[package]]
name = "io-lifetimes"
version = "1.0.10"
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c66c74d2ae7e79a5a8f7ac924adbe38ee42a859c6539ad869eb51f0b52dc220"
dependencies = [
"hermit-abi",
"libc",
"windows-sys",
]
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "is-terminal"
version = "0.4.7"
name = "memchr"
version = "2.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adcf93614601c8129ddf72e2d5633df827ba6551541c6d8c59520a371475be1f"
dependencies = [
"hermit-abi",
"io-lifetimes",
"rustix",
"windows-sys",
]
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
[[package]]
name = "libc"
version = "0.2.144"
name = "once_cell_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b00cc1c228a6782d0f076e7b232802e0c5689d41bb5df366f2a6b6621cfdfe1"
[[package]]
name = "linux-raw-sys"
version = "0.3.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef53942eb7bf7ff43a617b3e2c1c4a5ecf5944a7c1bc12d7ee39bbb15e5c1519"
[[package]]
name = "once_cell"
version = "1.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7e5500299e16ebb147ae15a00a942af264cf3688f47923b8fc2cd5858f23ad3"
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
[[package]]
name = "proc-macro2"
version = "1.0.58"
version = "1.0.107"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa1fb82fc0c281dd9671101b66b771ebbe1eaf967b96ac8740dcba4b70005ca8"
checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.27"
version = "1.0.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f4f29d145265ec1c483c7c654450edde0bfe043d3938d6972630663356d9500"
checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rustix"
version = "0.37.19"
name = "serde"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "acf8729d8542766f1b2cf77eb034d52f40d375bb8b615d0b147089946e16613d"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"bitflags",
"errno",
"io-lifetimes",
"libc",
"linux-raw-sys",
"windows-sys",
"serde_core",
]
[[package]]
name = "strsim"
version = "0.10.0"
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73473c0e59e6d5812c5dfe2a064a6444949f089e20eec9a2e5506596494e4623"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.151"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "strsim"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "syn"
version = "2.0.16"
version = "3.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6f671d4b5ffdb8eadec19c0ae67fe2639df8684bd7bc4b83d986b8db549cf01"
checksum = "e6275cddf4610d1775e6d1fe9469b2e77d0f39fd98fb7450901b821e0c53649f"
dependencies = [
"proc-macro2",
"quote",
@@ -267,78 +255,33 @@ dependencies = [
[[package]]
name = "unicode-ident"
version = "1.0.8"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5464a87b239f13a63a501f2701565754bae92d243d4bb7eb12f6d57d2269bf4"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "utf8parse"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "711b9620af191e0cdc7468a8d14e709c3dcdb115b36f838e601583af800a370a"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.48.0"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "677d2418bec65e3338edb076e806bc1ec15693c5d0104683f2efe857f61056a9"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-targets",
"windows-link",
]
[[package]]
name = "windows-targets"
version = "0.48.0"
name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b1eb6f0cd7c80c79759c929114ef071b87354ce476d9d94271031c0497adfd5"
dependencies = [
"windows_aarch64_gnullvm",
"windows_aarch64_msvc",
"windows_i686_gnu",
"windows_i686_msvc",
"windows_x86_64_gnu",
"windows_x86_64_gnullvm",
"windows_x86_64_msvc",
]
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91ae572e1b79dba883e0d315474df7305d12f569b400fcf90581b06062f7e1bc"
[[package]]
name = "windows_aarch64_msvc"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b2ef27e0d7bdfcfc7b868b317c1d32c641a6fe4629c171b8928c7b08d98d7cf3"
[[package]]
name = "windows_i686_gnu"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "622a1962a7db830d6fd0a69683c80a18fda201879f0f447f065a3b7467daa241"
[[package]]
name = "windows_i686_msvc"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4542c6e364ce21bf45d69fdd2a8e455fa38d316158cfd43b3ac1c5b1b19f8e00"
[[package]]
name = "windows_x86_64_gnu"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca2b8a661f7628cbd23440e50b05d705db3686f894fc9580820623656af974b1"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7896dbc1f41e08872e9d5e8f8baa8fdd2677f29468c4e156210174edc7f7b953"
[[package]]
name = "windows_x86_64_msvc"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a515f5799fe4961cb532f983ce2b23082366b898e52ffbce459c86f67c8378a"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"

View File

@@ -1,17 +1,32 @@
[package]
name = "ahab"
description = "docker cli wrapper"
description = "docker compose wrapper for django projects, with service detection and database dumps"
readme = "README.md"
version = "0.3.0"
edition = "2021"
version = "0.5.0"
edition = "2024"
rust-version = "1.85"
license = "MIT"
authors = ["Matej Janežič <janezic.mj@gmail.com>"]
repository = "https://github.com/janezicmatej/ahab.git"
authors = ["Matej Janežič <matej.janezic@aflabs.com>"]
repository = "https://git.aflabs.org/janezicmatej/ahab"
build = "build.rs"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
clap = { version = "4.3.0", features = ["derive"] }
clap_complete = "4.3.0"
anyhow = "1.0.71"
dotenvy = "0.15.7"
clap = { version = "4.6.6", features = ["derive", "env"] }
clap_complete = "4.6.9"
anyhow = "1.0.104"
serde_json = "1.0.145"
fs-err = "3.3.1"
shlex = "2.0.1"
[build-dependencies]
clap = { version = "4.6.6", features = ["derive", "env"] }
clap_complete = "4.6.9"
[lints.rust]
unsafe_code = "forbid"
[lints.clippy]
dbg_macro = "warn"
todo = "warn"

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2023 Matej Janežič
Copyright (c) 2023 AFLabs d.o.o.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

242
README.md
View File

@@ -1 +1,243 @@
# ahab
A wrapper around `docker compose` for our dockerized django projects, so the
same commands work in every repository.
## installing
You will need rust installed. Clone repo and run:
```bash
cargo install --path .
```
## what it makes of a project
`ahab status` prints the services it picked, the containers behind them, the
credentials it would use and what the store holds, reporting whatever it cannot
work out rather than stopping at the first thing:
```
$ ahab status
services: db, django
django: django (built from the project)
container: 0a8b19e27634
postgres: db (postgres:18-alpine)
container: 189d4d395d62
user: myproject
database: myproject_db
store: /home/you/.local/share/ahab/git.aflabs.org/urnik/afurnik
linked: 2
`ahab link check` lists what a sandbox can still read
```
## seeing what it runs
`-v` prints each docker command as it runs, `--dry-run` prints the ones it would
run without running them, and `-q` prints only what was asked for, dropping the
progress along the way (compose's own progress included). All work before or
after the subcommand.
```bash
ahab -v django migrate
ahab --dry-run postgres import ./dump
ahab -q postgres dump ./dump
```
Nothing is written under `--dry-run`, filesystem included: `link add` and
`django make-command` say what they would do and leave the tree alone.
## exit codes
- `0` — it ran and had nothing to report
- `1` — it could not finish
- `2` — the arguments were wrong
- `4` — it ran fine and found something worth reporting, i.e.
`link check --exit-code` with anything outside the store
## shell completion
`ahab completions <shell>` writes a completion script to stdout, for bash,
elvish, fish, powershell or zsh:
```bash
ahab completions zsh > ~/.local/share/zsh/completions/_ahab
eval "$(ahab completions zsh)" # or one line in .zshrc, never goes stale
```
Completions are also generated during the build. They land in
`target/*/build/*/out/` by default, and `SHELL_COMPLETIONS_DIR_<SHELL>`
installs a single shell's file straight into place:
```bash
SHELL_COMPLETIONS_DIR_ZSH=~/.local/share/zsh/completions \
SHELL_COMPLETIONS_DIR_FISH=~/.config/fish/completions \
cargo install --path .
```
`SHELL_COMPLETIONS_DIR` writes every shell into one directory instead.
## the compose file
`ahab` does not pass `-f`. docker compose finds the file itself, so set
docker's own `COMPOSE_FILE` when it is not in the working directory, including
its `base.yaml:override.yaml` form. A project's `.env` is a good place for it,
since docker reads that too:
```
COMPOSE_FILE=docker/docker-compose.yaml
```
## service detection
`ahab` finds the services it needs in `docker compose config`, so they can be
named anything:
- **postgres**: the service whose image is a postgres flavour, matching
`postg`, `timescale`, `pgvector` or `citus`
- **django**: the service that both builds an image and sets
`DJANGO_SETTINGS_MODULE`. Where more than one does — a worker beside the web
service, say — the one publishing ports wins
If nothing matches, or two candidates cannot be told apart, `ahab` says so and
lists the services it looked at rather than guessing. Two services that both
build and set the settings module are ambiguous unless exactly one of them
publishes ports.
`POSTGRES_USER` and `POSTGRES_DB` are read off the detected postgres service, so
`dropdb`, `createdb`, `pg_restore` and `pg_dump` use the role and database the
project declares, falling back to `db` when it declares neither. Both have to
be names: `ahab` hands them to those tools as a role and a database, and libpq
reads a database name holding an `=` or a url as a whole connection string,
which would send a dump to whatever server it names. A value that could be read
as something other than a name is refused rather than passed on.
## django
```bash
ahab django run <cmd> # in a fresh container, through the entrypoint
ahab django bash # shell in a fresh container
ahab django manage <args> # manage.py
ahab django makemigrations
ahab django migrate <args>
ahab django shell
ahab django make-command <app> <name>
```
There is no `ahab django test`: which runner a project uses is the project's
choice, and `manage.py test` exits 0 having collected nothing when the tests are
written for pytest, so a wrong guess reads as a pass. Name the runner instead:
```bash
ahab django run pytest -x tests/
```
## postgres
```bash
ahab postgres dump <path> # pg_dump, custom format
ahab postgres dump -F plain <path> # pg_dump, plain sql
ahab postgres dump -F tar <path> # pg_dump, tar
ahab postgres dump -F directory <d> # pg_dump, a directory of files
ahab postgres dump -F cluster <path> # pg_dumpall, roles and all databases
ahab postgres dump -F plain -z <path> # any of them, gzipped
ahab postgres import <path> # drop, create, then restore
ahab postgres psql <args> # psql in the database container
```
A dump is written `0600` and under a name of its own until it is complete, then
renamed over the target: a cluster dump carries every role's password hash, and
the default `0644` would hand it to anyone else with an account on the machine.
`psql` passes its arguments through and only asks docker for a terminal when it
has one to hand over, so both of these work:
```bash
ahab postgres psql # interactive
ahab postgres psql -tAc 'select count(*) from auth_user' | wc -l
```
The format of a dump being imported is read from the file rather than its name.
A custom format dump starts with `PGDMP` and a tar one with `toc.dat`, both of
which go to `pg_restore`, as does a directory produced by `pg_dump -Fd`.
Anything else is treated as sql and fed to `psql` with `ON_ERROR_STOP` and
`--single-transaction`, so a bad file rolls back instead of half applying.
A whole cluster dump from `pg_dumpall` is recognised by its header and handled
differently again: it creates its own databases and carries role statements, so
the database is dropped but not recreated, the dump goes to `psql` connected to
`postgres`, and it runs without `ON_ERROR_STOP` because roles that already exist
report errors that are expected. Any error it does not expect is reported and
fails the import, since psql without `ON_ERROR_STOP` exits 0 having applied
only part of the dump. Gzipped dumps are decompressed on the way in, whichever
of the three they hold. A cluster that has never held the database yet is a
valid target either way, so importing into a fresh one works.
`import` stops the project before it starts, so an import that fails leaves it
stopped and says so: `docker compose up` brings it back.
## link
`ahab link` moves untracked paths out of the repository into an out-of-repo
store and symlinks them back, so a sandbox that mounts the repository sees a
dangling symlink instead of the contents, while the host resolves it as before.
```bash
ahab link add .env secrets/ # move out, leave symlinks behind
ahab link restore .env # move it back into the repository
ahab link restore --all # everything this repo has linked
ahab link list # what the store holds for this repository
ahab link check # what a sandbox can still read
ahab link check --porcelain # `<code> <path>`, for scripts
ahab link check --exit-code # exit 4 when anything is outside the store
```
`restore` is the inverse of `add`: the file moves out of the store and back to
where it was, and the store keeps nothing. A second checkout linking the same
path is left with a dangling symlink, since there is only ever one stored copy.
`list` says what state each stored path is in, which `check` cannot see because
it asks git about the repository rather than reading the store:
```
$ ahab link list
store: /home/you/.local/share/ahab/git.aflabs.org/urnik/afurnik
linked: .env
shadowed: config.local.py
missing: secrets/token
```
`missing` is a path whose symlink is gone, `shadowed` one that a real file took
back; either way the stored copy is the one nobody reads.
`restore --all` moves back the linked ones and says how many it left, since
there is nothing to undo for a path whose symlink is gone.
`-z` is the porcelain format for scripts that must survive any filename: entries
end with a NUL, and the two paths of a symlink leading elsewhere are separated
by one as well, the way `git status -z` reports a rename.
`check` looks at tracked paths too, since git tracks symlinks and one can lead
out of the repository without appearing in any untracked listing. `add` cannot
externalize a tracked path, so all `check` can do is report it, under the code
`T>`.
A symlink already leading out of the repository is not something `add` will
take: moving the link would move the pointer and leave the contents where they
are, so the store would hold a way back out while `check`, seeing a link into
the store, called the repository clean.
The store lives under
`${XDG_DATA_HOME:-$HOME/.local/share}/ahab/<host>/<owner>/<repo>/`, derived
from the git `origin` remote, or `_local/<checkout>` when there is no remote to
name it after. Its directories are created `0700`: it exists to hold what
should not be readable from the repository, and on a shared machine the default
`0755` would leave that to whoever else has an account. A store path derived
from a remote, or a checkout name, that needed characters replacing carries a
short fingerprint of the original, so two of them cannot land on one directory.
## configuration
Currently `ahab` respects the following environment variables.
- `AHAB_LINK_ROOT`: where `ahab link` keeps its store - the same as `--store`,
which takes precedence, and defaults to `${XDG_DATA_HOME:-$HOME/.local/share}/ahab`.
`ahab status` reads it too, so it reports the store the link commands use

75
build.rs Normal file
View File

@@ -0,0 +1,75 @@
use clap::{CommandFactory, ValueEnum};
use clap_complete::{Shell, generate_to};
use std::{env, ffi::OsString, fs, io::Error, path::PathBuf};
include!("src/cli/mod.rs");
fn shell_name(shell: Shell) -> String {
shell
.to_possible_value()
.map(|v| v.get_name().to_owned())
.unwrap_or_default()
}
fn non_empty(name: &str) -> Option<OsString> {
env::var_os(name).filter(|v| !v.is_empty())
}
fn install_dir(shell: Shell) -> Option<(PathBuf, bool)> {
let per_shell = format!("SHELL_COMPLETIONS_DIR_{}", shell_name(shell).to_uppercase());
if let Some(dir) = non_empty(&per_shell).or_else(|| non_empty("SHELL_COMPLETIONS_DIR")) {
return Some((PathBuf::from(dir), true));
}
non_empty("OUT_DIR").map(|dir| (PathBuf::from(dir), false))
}
fn main() -> Result<(), Error> {
// naming any rerun-if condition replaces cargo's default of re-running the
// script whenever anything in the package changed, so everything that
// shapes a completion script has to be named here. without them an
// installed script goes stale against the binary it completes
println!("cargo::rerun-if-changed=build.rs");
println!("cargo::rerun-if-changed=src/cli");
println!("cargo::rerun-if-changed=Cargo.toml");
println!("cargo::rerun-if-env-changed=SHELL_COMPLETIONS_DIR");
for shell in Shell::value_variants() {
println!(
"cargo::rerun-if-env-changed=SHELL_COMPLETIONS_DIR_{}",
shell_name(*shell).to_uppercase()
);
}
let mut cmd = ahab::Ahab::command();
for shell in Shell::value_variants() {
let Some((dir, requested)) = install_dir(*shell) else {
continue;
};
// a build script's working directory is the package root, so a relative
// dir would quietly install into the source tree and one holding `..`
// somewhere else again. only a path that says where it means is taken
if requested && !dir.is_absolute() {
return Err(Error::other(format!(
"the completions directory must be an absolute path, not {}",
dir.display()
)));
}
fs::create_dir_all(&dir)
.map_err(|e| Error::other(format!("creating {}: {e}", dir.display())))?;
let path = generate_to(*shell, &mut cmd, env!("CARGO_PKG_NAME"), &dir)?;
if requested {
println!(
"cargo::warning=installed {} completion: {}",
shell_name(*shell),
path.display()
);
}
}
Ok(())
}

View File

@@ -1,35 +1,31 @@
use super::{Django, Docker, DockerCompose, Postgres};
use super::{Django, Link, Postgres, Store};
use clap::builder::styling::{AnsiColor, Effects, Styles};
use clap::{Parser, Subcommand};
use clap_complete::Shell;
/// A program for interacting with various dockerized applications.
/// A program for interacting with various dockerized applications
#[derive(Parser, Debug)]
#[command(author, version, about, long_about=None)]
#[command(version, about, long_about = None)]
#[command(styles = help_styles())]
pub struct Ahab {
#[command(subcommand)]
pub command: Commands,
}
/// Print each docker command as it runs
#[arg(short, long, global = true)]
pub verbose: bool,
/// Print only what was asked for, not the progress along the way
#[arg(short, long, global = true, conflicts_with = "verbose")]
pub quiet: bool,
/// Print the docker commands that would run, without running them
#[arg(long, global = true)]
pub dry_run: bool,
}
#[derive(Debug, Subcommand)]
pub enum Commands {
/// Generate completion files
Completion {
#[arg(value_enum)]
shell: Shell,
},
/// Docker related subcommands
Docker {
#[command(subcommand)]
command: Docker,
},
/// Docker compose related subcommands
Compose {
#[command(subcommand)]
command: DockerCompose,
},
/// Django related subcommands
Django {
#[command(subcommand)]
@@ -41,4 +37,42 @@ pub enum Commands {
#[command(subcommand)]
command: Postgres,
},
/// Out-of-repo store related subcommands
Link {
#[command(subcommand)]
command: Link,
},
/// Show what ahab makes of this project
Status {
#[command(flatten)]
store: Store,
},
/// Print a shell completion script on stdout
Completions {
/// Shell to generate the script for
shell: Shell,
},
}
// clap styles headers bold by default but adds no colour
fn help_styles() -> Styles {
Styles::styled()
.header(AnsiColor::Green.on_default() | Effects::BOLD)
.usage(AnsiColor::Green.on_default() | Effects::BOLD)
.literal(AnsiColor::Cyan.on_default() | Effects::BOLD)
.placeholder(AnsiColor::Cyan.on_default())
}
#[cfg(test)]
mod tests {
use super::Ahab;
use clap::CommandFactory;
#[test]
fn the_command_is_well_formed() {
Ahab::command().debug_assert();
}
}

View File

@@ -1,37 +1,38 @@
use std::path::PathBuf;
use clap::Parser;
use clap::Subcommand;
// TODO: (matej) dsu template command
#[derive(Parser, Debug)]
#[derive(Subcommand, Debug)]
pub enum Django {
/// Prepare empty management command 'command' in app 'app'.
MakeCommand {
#[arg(value_enum)]
app: PathBuf,
#[arg(value_enum)]
name: String,
},
/// Start a bash session in a fresh django container
Bash,
/// Run Django's manage.py makemigrations.
/// Prepare an empty management command NAME in the app at APP
MakeCommand { app: PathBuf, name: String },
/// Run Django's manage.py makemigrations
Makemigrations,
/// Pass arguments to Django's manage.py.
/// Pass arguments to Django's manage.py
Manage {
#[arg(value_enum)]
#[arg(trailing_var_arg = true, allow_hyphen_values = true)]
rest: Vec<String>,
},
/// Run Django's manage.py migrate.
/// Run Django's manage.py migrate
Migrate {
#[arg(value_enum)]
#[arg(trailing_var_arg = true, allow_hyphen_values = true)]
rest: Vec<String>,
},
/// Run Django's manage.py shell.
/// Run a command in a fresh django container, through its entrypoint
Run {
#[arg(trailing_var_arg = true, allow_hyphen_values = true)]
rest: Vec<String>,
},
/// Run Django's manage.py shell
Shell,
/// Run Django's manage.py test.
Test,
}

View File

@@ -1,7 +0,0 @@
use clap::Parser;
#[derive(Parser, Debug)]
pub enum Docker {
/// Stop all containers via `docker stop $(docker ps -q)`
StopAll,
}

View File

@@ -1,28 +0,0 @@
use clap::Subcommand;
// TODO: (matej) add Exec, Bash
/// Wraper for docker compose; autodiscover compose file and source .env file.
#[derive(Subcommand, Debug)]
pub enum DockerCompose {
/// Build containers.
Build,
/// Down containers.
Down,
/// Stop, build and start containers.
Rebuild,
/// Stop and start containers.
Restart,
/// Start containers.
Start,
/// Stop containers.
Stop,
/// Up containers.
Up,
}

94
src/cli/link.rs Normal file
View File

@@ -0,0 +1,94 @@
use std::path::PathBuf;
use clap::{Args, Subcommand};
// each variant carries its own arguments as a struct rather than as fields
// spread across the enum. the command that implements it takes that struct, so
// main does not unpack what clap has already parsed and hand it on as a row of
// booleans nothing but position tells apart
#[derive(Subcommand, Debug)]
pub enum Link {
/// Move untracked paths into the store and symlink them back
Add(Add),
/// Move paths in the store back into the repository
Restore(Restore),
/// List what this repository keeps in the store
List(List),
/// List untracked paths a sandbox would still see
Check(Check),
}
#[derive(Args, Debug)]
pub struct Add {
/// Untracked or ignored paths inside the repository
#[arg(required = true)]
pub paths: Vec<PathBuf>,
/// Link to a store path that already exists
#[arg(long)]
pub force: bool,
#[command(flatten)]
pub store: Store,
}
#[derive(Args, Debug)]
pub struct Restore {
// said here rather than checked at runtime, so a mistake in the
// arguments is reported as one, with the usage and the exit code clap
// gives every other argument error
#[arg(required_unless_present = "all", conflicts_with = "all")]
pub paths: Vec<PathBuf>,
/// Restore every path this repository has in the store
#[arg(long)]
pub all: bool,
#[command(flatten)]
pub store: Store,
}
#[derive(Args, Debug)]
pub struct List {
#[command(flatten)]
pub store: Store,
}
#[derive(Args, Debug)]
pub struct Check {
/// Limit the listing to these paths
pub paths: Vec<PathBuf>,
/// Print `<code> <path>` for scripts, as `git status --porcelain` does
#[arg(long)]
pub porcelain: bool,
/// Exit with 4 when anything is outside the store, for scripts
#[arg(long)]
pub exit_code: bool,
/// Terminate porcelain entries with NUL
#[arg(short = 'z')]
pub null: bool,
#[command(flatten)]
pub store: Store,
}
#[derive(Args, Debug)]
pub struct Store {
/// Where the out-of-repo store lives
///
/// Defaults to `${XDG_DATA_HOME:-$HOME/.local/share}/ahab`.
#[arg(long = "store", env = "AHAB_LINK_ROOT", value_name = "DIR")]
pub root: Option<PathBuf>,
}
impl Store {
pub fn root(&self) -> Option<&std::path::Path> {
self.root.as_deref()
}
}

View File

@@ -1,11 +1,11 @@
mod ahab;
mod django;
mod docker;
mod docker_compose;
mod postgres;
// build.rs reaches these definitions with include!, so nothing here may refer to
// the rest of the crate: keep this tree to clap definitions only
pub mod ahab;
pub mod django;
pub mod link;
pub mod postgres;
pub use ahab::{Ahab, Commands};
pub use django::Django;
pub use docker::Docker;
pub use docker_compose::DockerCompose;
pub use postgres::Postgres;
pub use link::{Link, Store};
pub use postgres::{Format, Postgres};

View File

@@ -1,18 +1,56 @@
use std::path::PathBuf;
use clap::Subcommand;
use clap::{Args, Subcommand, ValueEnum};
#[derive(Subcommand, Debug)]
pub enum Postgres {
/// Import dump via pg_restore
Import {
#[arg(value_enum)]
path: PathBuf,
/// Import a dump, in any format ahab can produce
Import { path: PathBuf },
/// Open psql in the database container
Psql {
/// Arguments for psql itself
#[arg(trailing_var_arg = true, allow_hyphen_values = true)]
rest: Vec<String>,
},
/// Dump via pg_dump with format=c
Dump {
#[arg(value_enum)]
path: PathBuf,
},
/// Dump via pg_dump, or pg_dumpall for a whole cluster
Dump(Dump),
}
#[derive(Args, Debug)]
pub struct Dump {
pub path: PathBuf,
/// Dump format
#[arg(short = 'F', long, value_enum, default_value_t = Format::Custom)]
pub format: Format,
/// Compress the dump with gzip
#[arg(short = 'z', long)]
pub gzip: bool,
}
#[derive(ValueEnum, Clone, Copy, Debug, Default, PartialEq)]
pub enum Format {
/// pg_dump --format=c, restored with pg_restore
#[default]
#[value(alias = "c")]
Custom,
/// pg_dump --format=p, restored with psql
#[value(alias = "p")]
Plain,
/// pg_dump --format=t, restored with pg_restore
#[value(alias = "t")]
Tar,
/// pg_dump --format=d, a directory rather than a file
#[value(alias = "d")]
Directory,
/// pg_dumpall, the whole cluster including roles
#[value(alias = "all")]
Cluster,
}

284
src/cmd/argv.rs Normal file
View File

@@ -0,0 +1,284 @@
use std::{
ffi::{OsStr, OsString},
fmt::Display,
fs::File,
os::unix::process::{CommandExt, ExitStatusExt},
path::Path,
process::{Command, ExitStatus, Output, Stdio},
};
use anyhow::{Context, Result, anyhow};
use crate::ctx::Ctx;
use crate::output::write_err;
// a program and its arguments: the only thing in ahab that knows what argv looks like
#[derive(Default, Clone)]
pub struct Argv(Vec<OsString>);
// a dry run spawned nothing, so there is no status; nothing ran and nothing
// failed, which is what every caller checking one of these wants to hear
fn planned() -> ExitStatus {
ExitStatus::from_raw(0)
}
impl Display for Argv {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.quoted())
}
}
impl Argv {
pub fn new(program: impl AsRef<OsStr>) -> Self {
Self(vec![program.as_ref().to_owned()])
}
pub fn arg(mut self, arg: impl AsRef<OsStr>) -> Self {
self.0.push(arg.as_ref().to_owned());
self
}
pub fn args<I, S>(mut self, args: I) -> Self
where
I: IntoIterator<Item = S>,
S: AsRef<OsStr>,
{
self.0
.extend(args.into_iter().map(|arg| arg.as_ref().to_owned()));
self
}
// a long option and the value it takes
pub fn flag(self, name: &str, value: impl AsRef<OsStr>) -> Self {
self.arg(name).arg(value)
}
// for a message that names the tool being run, so lossy is what is wanted
pub fn program(&self) -> String {
self.0
.first()
.map(|word| word.to_string_lossy().into_owned())
.unwrap_or_default()
}
pub fn words(&self) -> &[OsString] {
&self.0
}
// for reading: the -v echo, the dry run's plan, and the command named in an
// error. lossy, because a message about a path is still worth printing
pub fn quoted(&self) -> String {
let words: Vec<String> = self
.0
.iter()
.map(|word| word.to_string_lossy().into_owned())
.collect();
// a nul byte is the only thing shlex refuses, and no argv can hold one,
// so this only happens for a word that could never have been run. it
// still has to be readable: an empty string would name no command at
// all in the very message explaining what went wrong
shlex::try_join(words.iter().map(String::as_str)).unwrap_or_else(|_| words.join(" "))
}
// one word list for a shell to read, which is a string and so cannot carry
// a word that is not text. only ascii and compose-derived words reach a
// pipeline today; this is what keeps it that way, rather than a comment
// saying so while to_string_lossy quietly rewrites a path
pub fn for_shell(&self) -> Result<String> {
let mut words = Vec::with_capacity(self.0.len());
for word in &self.0 {
let word = word.to_str().ok_or_else(|| {
anyhow!(
"`{}` cannot go in a shell pipeline: {:?} is not text",
self.quoted(),
word
)
})?;
words.push(word);
}
shlex::try_join(words).map_err(|e| anyhow!("quoting `{}`: {e}", self.quoted()))
}
pub fn run(&self, ctx: &Ctx) -> Result<()> {
if self.skipped(ctx) {
return Ok(());
}
let status = self.command(ctx)?.spawn()?.wait()?;
self.check(status)
}
// reading changes nothing, so a dry run answers the question for real
pub fn capture(&self, ctx: &Ctx) -> Result<String> {
let stdout = self.capture_bytes(ctx)?;
String::from_utf8(stdout).with_context(|| format!("reading the output of `{self}`"))
}
// git's -z listings are raw bytes: a path on unix is bytes, not text, and
// one that is not utf-8 would otherwise fail the whole listing it appears in
pub fn capture_bytes(&self, ctx: &Ctx) -> Result<Vec<u8>> {
let out = self.command(ctx)?.output()?;
if !out.status.success() {
// output() holds stderr back, so the command's own complaint has to be
// passed on here or it is lost
let stderr = String::from_utf8_lossy(&out.stderr);
return Err(match stderr.trim() {
"" => self.failure(out.status),
reason => anyhow!("`{self}` failed: {reason}"),
});
}
Ok(out.stdout)
}
// replaces this process, so the command's exit code and signals become ours
pub fn replace(&self, ctx: &Ctx) -> Result<()> {
if self.skipped(ctx) {
return Ok(());
}
let error = self.command(ctx)?.exec();
Err(error).with_context(|| format!("running `{self}`"))
}
pub fn stream_to(&self, ctx: &Ctx, out: Stdio) -> Result<()> {
if self.skipped(ctx) {
return Ok(());
}
let status = self.command(ctx)?.stdout(out).spawn()?.wait()?;
self.check(status)
}
// the status rather than an error, for callers with something better to say
pub fn status(&self, ctx: &Ctx) -> Result<ExitStatus> {
if self.skipped(ctx) {
return Ok(planned());
}
Ok(self.command(ctx)?.spawn()?.wait()?)
}
pub fn stdin_from(&self, ctx: &Ctx, input: &Path) -> Result<ExitStatus> {
if self.skipped(ctx) {
return Ok(planned());
}
let stdin = self.opened(input)?;
Ok(self.command(ctx)?.stdin(stdin).spawn()?.wait()?)
}
// both streams held back, for callers that read the command's complaints
pub fn stdin_from_captured(&self, ctx: &Ctx, input: &Path) -> Result<Output> {
if self.skipped(ctx) {
return Ok(Output {
status: planned(),
stdout: Vec::new(),
stderr: Vec::new(),
});
}
let stdin = self.opened(input)?;
Ok(self.command(ctx)?.stdin(stdin).output()?)
}
// a probe that reads its input: like capture, it is asking rather than
// changing, so a dry run gets the real answer and plans with it
pub fn probe_with_stdin(&self, ctx: &Ctx, input: &Path) -> Result<Output> {
let stdin = self.opened(input)?;
Ok(self.command(ctx)?.stdin(stdin).output()?)
}
// a probe: it asks a question rather than changing anything, so a dry run
// answers it for real, and the status is the answer instead of an error
pub fn probe_status(&self, ctx: &Ctx) -> Result<ExitStatus> {
Ok(self.command(ctx)?.stdout(Stdio::null()).spawn()?.wait()?)
}
// whether it succeeded, without failure being an error
pub fn quietly_succeeds(&self, ctx: &Ctx) -> Result<bool> {
Ok(self.probe_status(ctx)?.success())
}
fn command(&self, ctx: &Ctx) -> Result<Command> {
if ctx.verbose {
write_err(format_args!("running `{self}`"));
}
let (program, rest) = self.0.split_first().context("empty command")?;
let mut command = Command::new(program);
command.args(rest);
Ok(command)
}
fn skipped(&self, ctx: &Ctx) -> bool {
if ctx.dry_run {
// the plan is what a dry run was asked for, so --quiet keeps it
write_err(format_args!("would run `{self}`"));
return true;
}
false
}
fn opened(&self, path: &Path) -> Result<Stdio> {
let file = File::open(path).with_context(|| format!("opening {}", path.display()))?;
Ok(Stdio::from(file))
}
fn check(&self, status: ExitStatus) -> Result<()> {
if status.success() {
return Ok(());
}
Err(self.failure(status))
}
fn failure(&self, status: ExitStatus) -> anyhow::Error {
match status.code() {
Some(code) => anyhow!("`{self}` exited with {code}"),
None => anyhow!("`{self}` was killed by a signal"),
}
}
}
#[cfg(test)]
mod tests {
use super::Argv;
#[test]
fn plain_words_are_left_as_they_are() {
let argv = Argv::new("pg_dump")
.flag("--username", "myproject")
.arg("--data-only")
.arg("myproject_db");
assert_eq!(
argv.quoted(),
"pg_dump --username myproject --data-only myproject_db"
);
}
#[test]
fn anything_a_shell_would_read_as_more_than_one_word_is_quoted() {
assert_eq!(Argv::new("psql").arg("my db").quoted(), "psql 'my db'");
assert_eq!(Argv::new("sh").arg("a | b").quoted(), "sh 'a | b'");
assert_eq!(Argv::new("echo").arg("").quoted(), "echo ''");
assert_eq!(Argv::new("echo").arg("it's").quoted(), r#"echo "it's""#);
}
}

136
src/cmd/compose.rs Normal file
View File

@@ -0,0 +1,136 @@
use super::{Argv, Cmd};
use crate::ctx::Ctx;
// compose prints its own progress, which --quiet has to ask it to stop
fn compose(quiet: bool) -> Argv {
let argv = Argv::new("docker").arg("compose");
match quiet {
true => argv.flag("--progress", "quiet"),
false => argv,
}
}
// the compose commands, each already knowing whether -q asked compose to stop
// narrating. the flag was a field every construction site set by hand, and
// three of them did not, so `ahab -q` still had compose talking over the output
pub struct Compose<'a> {
ctx: &'a Ctx,
}
impl<'a> Compose<'a> {
pub(crate) fn new(ctx: &'a Ctx) -> Self {
Self { ctx }
}
pub fn up(&self) -> Up {
Up {
quiet: self.ctx.quiet,
}
}
pub fn stop(&self) -> Stop {
Stop {
quiet: self.ctx.quiet,
}
}
pub fn start(&self, service: &'a str) -> Start<'a> {
Start {
service,
quiet: self.ctx.quiet,
}
}
// reading the project says nothing, so there is no progress to quieten
pub fn config(&self) -> Config {
Config
}
pub fn ps(&self, service: &str) -> Ps {
Ps {
service: service.to_string(),
}
}
}
// docker compose run --rm, which runs the image's entrypoint and so fixes up the
// container user before handing over
pub struct Run {
service: String,
inner: Argv,
quiet: bool,
}
impl Run {
// only reached through Cmd::in_service, which takes the context, so the
// progress flag cannot be left off by forgetting a builder call
pub(super) fn wrapping(ctx: &Ctx, service: &str, inner: Argv) -> Self {
Self {
service: service.to_string(),
inner,
quiet: ctx.quiet,
}
}
}
impl Cmd for Run {
fn argv(&self) -> Argv {
compose(self.quiet)
.arg("run")
.arg("--rm")
.arg(&self.service)
.args(self.inner.words())
}
}
// the merged project file, with every extends and env_file resolved
pub struct Config;
impl Cmd for Config {
fn argv(&self) -> Argv {
compose(false).arg("config").flag("--format", "json")
}
}
// the container id a service is running as, empty when it is not running
pub struct Ps {
service: String,
}
impl Cmd for Ps {
fn argv(&self) -> Argv {
compose(false).arg("ps").arg("--quiet").arg(&self.service)
}
}
pub struct Up {
quiet: bool,
}
impl Cmd for Up {
fn argv(&self) -> Argv {
compose(self.quiet).arg("up").arg("--detach")
}
}
pub struct Start<'a> {
service: &'a str,
quiet: bool,
}
impl Cmd for Start<'_> {
fn argv(&self) -> Argv {
compose(self.quiet).arg("start").arg(self.service)
}
}
pub struct Stop {
quiet: bool,
}
impl Cmd for Stop {
fn argv(&self) -> Argv {
compose(self.quiet).arg("stop")
}
}

44
src/cmd/django.rs Normal file
View File

@@ -0,0 +1,44 @@
use super::{Argv, Cmd};
// an interactive shell in the django container
pub struct Bash;
impl Cmd for Bash {
fn argv(&self) -> Argv {
Argv::new("bash")
}
}
// django's manage.py, with the subcommand and its arguments
pub struct Manage<'a> {
args: &'a [String],
}
impl<'a> Manage<'a> {
pub fn new(args: &'a [String]) -> Self {
Self { args }
}
}
impl Cmd for Manage<'_> {
fn argv(&self) -> Argv {
Argv::new("python").arg("manage.py").args(self.args)
}
}
// whatever the caller typed, passed through as it stands
pub struct Words<'a> {
words: &'a [String],
}
impl<'a> Words<'a> {
pub fn new(words: &'a [String]) -> Self {
Self { words }
}
}
impl Cmd for Words<'_> {
fn argv(&self) -> Argv {
Argv::default().args(self.words)
}
}

85
src/cmd/docker.rs Normal file
View File

@@ -0,0 +1,85 @@
use std::ffi::OsString;
use std::path::Path;
use super::{Argv, Cmd};
// docker exec, around a command that runs inside the container
pub struct Exec {
container: String,
interactive: bool,
tty: bool,
inner: Argv,
}
impl Exec {
pub fn wrapping(container: &str, inner: Argv) -> Self {
Self {
container: container.to_string(),
interactive: false,
tty: false,
inner,
}
}
// a terminal, for a command that is a shell rather than a pipe stage
pub fn tty(mut self, tty: bool) -> Self {
self.tty = tty;
self
}
// keep stdin open, for a command that is fed a dump
pub fn interactive(mut self) -> Self {
self.interactive = true;
self
}
}
impl Cmd for Exec {
fn argv(&self) -> Argv {
let mut argv = Argv::new("docker").arg("exec");
if self.interactive {
argv = argv.arg("--interactive");
}
if self.tty {
argv = argv.arg("--tty");
}
argv.arg(&self.container).args(self.inner.words())
}
}
// docker cp, in either direction. the local side stays an OsString: it is a
// path the user named, and a lossy one would copy something else
pub struct Cp {
from: OsString,
to: OsString,
}
impl Cp {
pub fn into_container(local: &Path, container: &str, remote: &str) -> Self {
Self {
from: local.as_os_str().to_owned(),
to: OsString::from(format!("{container}:{remote}")),
}
}
pub fn out_of_container(container: &str, remote: &str, local: &Path) -> Self {
Self {
from: OsString::from(format!("{container}:{remote}")),
to: local.as_os_str().to_owned(),
}
}
}
impl Cmd for Cp {
fn argv(&self) -> Argv {
// -L has no long form; it copies what a symlink points at
Argv::new("docker")
.arg("cp")
.arg("-L")
.arg(&self.from)
.arg(&self.to)
}
}

185
src/cmd/git.rs Normal file
View File

@@ -0,0 +1,185 @@
use std::ffi::OsString;
use std::path::Path;
use super::{Argv, Cmd};
// -C has no long form; it runs git as though from that directory
fn git(root: &Path) -> Argv {
Argv::new("git").arg("-C").arg(root)
}
// `--` stops git reading a leading dash as an option, but pathspec magic is
// read after it too, so a file actually named `:(exclude)x` or `:!x` would ask
// about a different set of paths than the one in hand. built as an OsString: a
// path is bytes, and a lossy one asks git about a name nothing on disk has
fn literal(path: &Path) -> OsString {
let mut spec = OsString::from(":(literal)");
spec.push(path);
spec
}
// the root of the repository the working directory is in
pub struct RevParse;
impl Cmd for RevParse {
fn argv(&self) -> Argv {
Argv::new("git").arg("rev-parse").arg("--show-toplevel")
}
}
// the url a remote points at, if the repository has one
pub struct ConfigGet<'a> {
pub key: &'a str,
}
impl Cmd for ConfigGet<'_> {
fn argv(&self) -> Argv {
Argv::new("git").arg("config").flag("--get", self.key)
}
}
pub struct LsFiles<'a> {
root: &'a Path,
tracked: bool,
ignored: bool,
pathspecs: Vec<OsString>,
}
impl<'a> LsFiles<'a> {
// paths git would not restore: untracked ones, and whole directories rather
// than every file inside them
pub fn untracked(root: &'a Path) -> Self {
Self {
root,
tracked: false,
ignored: false,
pathspecs: Vec::new(),
}
}
pub fn tracked(root: &'a Path) -> Self {
Self {
root,
tracked: true,
ignored: false,
pathspecs: Vec::new(),
}
}
// the ignored paths instead of the merely untracked ones
pub fn ignored(mut self) -> Self {
self.ignored = true;
self
}
pub fn limited_to<P: AsRef<Path>>(mut self, pathspecs: &[P]) -> Self {
self.pathspecs = pathspecs
.iter()
.map(|path| literal(path.as_ref()))
.collect();
self
}
}
impl Cmd for LsFiles<'_> {
fn argv(&self) -> Argv {
// -z has no long form; it separates the paths with NUL, which is the
// only separator a filename cannot contain
let mut argv = git(self.root).arg("ls-files").arg("-z");
if self.tracked {
argv = argv.arg("--cached");
} else {
argv = argv
.arg("--others")
.arg("--exclude-standard")
.arg("--directory")
.arg("--no-empty-directory");
}
if self.ignored {
argv = argv.arg("--ignored");
}
argv.arg("--").args(&self.pathspecs)
}
}
// whether a path is ignored, said through the exit code alone
pub struct CheckIgnore<'a> {
root: &'a Path,
path: &'a Path,
}
impl<'a> CheckIgnore<'a> {
pub fn new(root: &'a Path, path: &'a Path) -> Self {
Self { root, path }
}
}
impl Cmd for CheckIgnore<'_> {
fn argv(&self) -> Argv {
git(self.root)
.arg("check-ignore")
.arg("--quiet")
.arg("--")
.arg(self.path)
}
}
#[cfg(test)]
mod tests {
use std::path::{Path, PathBuf};
use super::{CheckIgnore, LsFiles};
use crate::cmd::Cmd;
#[test]
fn an_untracked_listing_asks_for_whole_directories() {
let argv = LsFiles::untracked(Path::new("/repo"))
.ignored()
.limited_to(&[PathBuf::from("a b")])
.argv();
assert_eq!(
argv.quoted(),
"git -C /repo ls-files -z --others --exclude-standard --directory \
--no-empty-directory --ignored -- ':(literal)a b'",
);
}
#[test]
fn the_tracked_listing_asks_git_only_about_the_paths_given() {
let argv = LsFiles::tracked(Path::new("/repo"))
.limited_to(&[PathBuf::from(".env")])
.argv();
assert_eq!(
argv.quoted(),
"git -C /repo ls-files -z --cached -- ':(literal).env'"
);
}
#[test]
fn a_pathspec_is_asked_for_literally_however_it_is_spelled() {
// git reads magic after `--` too, so these names must not become one
for name in [":(exclude)secret", ":!secret", ":(glob)**"] {
let argv = LsFiles::tracked(Path::new("/repo"))
.limited_to(&[PathBuf::from(name)])
.argv();
assert_eq!(
argv.words().last().unwrap(),
format!(":(literal){name}").as_str()
);
}
}
#[test]
fn check_ignore_says_nothing_and_reports_through_its_status() {
let argv = CheckIgnore::new(Path::new("/repo"), Path::new(".env")).argv();
assert_eq!(argv.quoted(), "git -C /repo check-ignore --quiet -- .env");
}
}

97
src/cmd/mod.rs Normal file
View File

@@ -0,0 +1,97 @@
mod argv;
mod compose;
mod django;
mod docker;
mod git;
mod postgres;
mod shell;
use std::{
path::Path,
process::{ExitStatus, Output, Stdio},
};
use anyhow::Result;
pub use argv::Argv;
pub use compose::{Compose, Run};
pub use django::{Bash, Manage, Words};
pub use docker::{Cp, Exec};
pub use git::{CheckIgnore, ConfigGet, LsFiles, RevParse};
pub use postgres::{CreateDb, DropDb, PgDump, PgDumpAll, PgIsReady, PgRestore, Psql};
pub use shell::{Gunzip, Gzip, Head, Pipeline, Rm};
use crate::ctx::Ctx;
// every command ahab runs, from `docker compose config` to `pg_dump`, is a type
// that renders itself here; adapters wrap one command in another and terminals
// run it, so a call site reads as one chain
pub trait Cmd {
fn argv(&self) -> Argv;
// this command as one word list, for a shell to read
fn shell(&self) -> Result<String> {
self.argv().for_shell()
}
// adapters
fn pipe(&self, next: &dyn Cmd) -> Result<Pipeline> {
Pipeline::starting(self.shell()?).pipe(next)
}
fn in_container(&self, container: &str) -> Exec {
Exec::wrapping(container, self.argv())
}
// the context comes in here because compose narrates what it is doing, and
// whether it should is the caller's -q rather than this command's business
fn in_service(&self, ctx: &Ctx, service: &str) -> Run {
Run::wrapping(ctx, service, self.argv())
}
// terminals
fn run(&self, ctx: &Ctx) -> Result<()> {
self.argv().run(ctx)
}
fn capture(&self, ctx: &Ctx) -> Result<String> {
self.argv().capture(ctx)
}
fn capture_bytes(&self, ctx: &Ctx) -> Result<Vec<u8>> {
self.argv().capture_bytes(ctx)
}
fn replace(&self, ctx: &Ctx) -> Result<()> {
self.argv().replace(ctx)
}
fn stream_to(&self, ctx: &Ctx, out: Stdio) -> Result<()> {
self.argv().stream_to(ctx, out)
}
// the status rather than an error, for callers with something better to say
fn status(&self, ctx: &Ctx) -> Result<ExitStatus> {
self.argv().status(ctx)
}
fn stdin_from(&self, ctx: &Ctx, input: &Path) -> Result<ExitStatus> {
self.argv().stdin_from(ctx, input)
}
fn stdin_from_captured(&self, ctx: &Ctx, input: &Path) -> Result<Output> {
self.argv().stdin_from_captured(ctx, input)
}
fn probe_with_stdin(&self, ctx: &Ctx, input: &Path) -> Result<Output> {
self.argv().probe_with_stdin(ctx, input)
}
fn quietly_succeeds(&self, ctx: &Ctx) -> Result<bool> {
self.argv().quietly_succeeds(ctx)
}
fn probe_status(&self, ctx: &Ctx) -> Result<ExitStatus> {
self.argv().probe_status(ctx)
}
}

279
src/cmd/postgres.rs Normal file
View File

@@ -0,0 +1,279 @@
use super::{Argv, Cmd};
use crate::cli::Format;
// whether the server is accepting connections yet
pub struct PgIsReady<'a> {
pub username: &'a str,
pub dbname: &'a str,
}
impl Cmd for PgIsReady<'_> {
fn argv(&self) -> Argv {
Argv::new("pg_isready")
.flag("--username", self.username)
.flag("--dbname", self.dbname)
}
}
pub struct DropDb<'a> {
pub username: &'a str,
pub dbname: &'a str,
}
impl Cmd for DropDb<'_> {
fn argv(&self) -> Argv {
// a cluster that never held this database is still a restore target
Argv::new("dropdb")
.flag("--username", self.username)
.arg("--if-exists")
// a database name is a positional, and getopt would read one
// beginning with a dash as an option instead
.arg("--")
.arg(self.dbname)
}
}
pub struct CreateDb<'a> {
pub username: &'a str,
pub dbname: &'a str,
}
impl Cmd for CreateDb<'_> {
fn argv(&self) -> Argv {
// template0 rather than template1, so nothing the local cluster picked up
// ends up in a restored database
Argv::new("createdb")
.flag("--username", self.username)
.flag("--encoding", "utf8")
.flag("--template", "template0")
.arg("--")
.arg(self.dbname)
}
}
// reads a custom, tar or directory format dump
pub struct PgRestore<'a> {
username: &'a str,
dbname: &'a str,
from: Option<&'a str>,
}
impl<'a> PgRestore<'a> {
pub fn new(username: &'a str, dbname: &'a str) -> Self {
Self {
username,
dbname,
from: None,
}
}
// a path in the container, for a dump that could not be streamed in
pub fn from(mut self, path: &'a str) -> Self {
self.from = Some(path);
self
}
}
impl Cmd for PgRestore<'_> {
fn argv(&self) -> Argv {
let argv = Argv::new("pg_restore")
.flag("--username", self.username)
.flag("--dbname", self.dbname);
match self.from {
Some(path) => argv.arg("--").arg(path),
None => argv,
}
}
}
// reads a plain sql dump, or runs as the interactive shell it is
pub struct Psql<'a> {
username: &'a str,
dbname: &'a str,
quiet: bool,
atomic: bool,
rest: &'a [String],
}
impl<'a> Psql<'a> {
pub fn new(username: &'a str, dbname: &'a str) -> Self {
Self {
username,
dbname,
quiet: false,
atomic: false,
rest: &[],
}
}
// say nothing and print no result rows, for a restore whose output is noise
pub fn quiet(mut self) -> Self {
self.quiet = true;
self
}
// stop at the first error and undo the rest, which a cluster dump cannot do:
// it connects to each database itself, and trips over roles already there
pub fn atomic(mut self) -> Self {
self.atomic = true;
self
}
// whatever the caller typed, for psql's own flags
pub fn args(mut self, rest: &'a [String]) -> Self {
self.rest = rest;
self
}
}
impl Cmd for Psql<'_> {
fn argv(&self) -> Argv {
let mut argv = Argv::new("psql");
if self.quiet {
argv = argv.arg("--quiet").flag("--output", "/dev/null");
}
argv = argv
.flag("--username", self.username)
.flag("--dbname", self.dbname);
if self.atomic {
argv = argv
.flag("--variable", "ON_ERROR_STOP=1")
.arg("--single-transaction");
}
argv.args(self.rest)
}
}
pub struct PgDump<'a> {
username: &'a str,
dbname: &'a str,
format: &'a str,
to: Option<&'a str>,
}
impl<'a> PgDump<'a> {
// pg_dump's -F letter, which belongs with the rest of what this module
// knows about pg_dump's argv rather than with the parser that reads the
// user's `-F custom`. a cluster has no letter: that is pg_dumpall, so
// there is no PgDump to build for it
pub fn of(username: &'a str, dbname: &'a str, format: Format) -> Option<Self> {
let format = match format {
Format::Custom => "c",
Format::Plain => "p",
Format::Tar => "t",
Format::Directory => "d",
Format::Cluster => return None,
};
Some(Self {
username,
dbname,
format,
to: None,
})
}
// a path in the container, for the directory format, which pg_dump writes
// itself rather than to stdout
pub fn to(mut self, path: &'a str) -> Self {
self.to = Some(path);
self
}
}
impl Cmd for PgDump<'_> {
fn argv(&self) -> Argv {
let argv = Argv::new("pg_dump")
.flag("--username", self.username)
.flag("--format", self.format);
match self.to {
Some(path) => argv.flag("--file", path),
None => argv,
}
.arg("--")
.arg(self.dbname)
}
}
// the whole cluster, roles and all
pub struct PgDumpAll<'a> {
pub username: &'a str,
}
impl Cmd for PgDumpAll<'_> {
fn argv(&self) -> Argv {
Argv::new("pg_dumpall").flag("--username", self.username)
}
}
#[cfg(test)]
mod tests {
use super::{DropDb, Format, PgDump, Psql};
use crate::cmd::Cmd;
#[test]
fn a_missing_database_is_not_a_failed_import() {
assert_eq!(
DropDb {
username: "u",
dbname: "db",
}
.argv()
.quoted(),
"dropdb --username u --if-exists -- db"
);
}
#[test]
fn a_directory_dump_names_the_file_it_writes() {
let directory = PgDump::of("u", "db", Format::Directory).unwrap();
assert_eq!(
directory.to("/tmp/dump").argv().quoted(),
"pg_dump --username u --format d --file /tmp/dump -- db"
);
assert_eq!(
PgDump::of("u", "db", Format::Custom)
.unwrap()
.argv()
.quoted(),
"pg_dump --username u --format c -- db"
);
}
#[test]
fn a_cluster_is_pg_dumpall_rather_than_a_pg_dump_letter() {
assert!(PgDump::of("u", "db", Format::Cluster).is_none());
for format in [
Format::Custom,
Format::Plain,
Format::Tar,
Format::Directory,
] {
assert!(PgDump::of("u", "db", format).is_some(), "{format:?}");
}
}
#[test]
fn only_a_single_database_restore_stops_at_the_first_error() {
assert!(
Psql::new("u", "db")
.quiet()
.atomic()
.argv()
.quoted()
.ends_with("--variable 'ON_ERROR_STOP=1' --single-transaction")
);
assert_eq!(
Psql::new("u", "postgres").quiet().argv().quoted(),
"psql --quiet --output /dev/null --username u --dbname postgres"
);
}
}

157
src/cmd/shell.rs Normal file
View File

@@ -0,0 +1,157 @@
use anyhow::Result;
use super::{Argv, Cmd};
// sh -c, the only way to reach a shell feature inside a container
pub struct Sh(String);
impl Sh {
pub fn new(script: impl Into<String>) -> Self {
Self(script.into())
}
}
impl Cmd for Sh {
fn argv(&self) -> Argv {
// -c has no long form
Argv::new("sh").arg("-c").arg(&self.0)
}
}
// commands joined by pipes, which only a shell can run
pub struct Pipeline {
stages: Vec<String>,
pipefail: bool,
}
impl Pipeline {
pub fn starting(first: String) -> Self {
Self {
stages: vec![first],
pipefail: true,
}
}
pub fn pipe(mut self, next: &dyn Cmd) -> Result<Self> {
self.stages.push(next.shell()?);
Ok(self)
}
// for a pipeline whose last stage closes the pipe on purpose, where the
// SIGPIPE that kills an earlier stage is the expected end and not a failure
pub fn allow_early_close(mut self) -> Self {
self.pipefail = false;
self
}
fn script(&self) -> String {
let piped = self.stages.join(" | ");
// a pipeline reports only its last stage's status, so a first stage that
// dies mid-stream reads as success without this
if self.pipefail {
format!("set -o pipefail; {piped}")
} else {
piped
}
}
}
impl Cmd for Pipeline {
fn argv(&self) -> Argv {
Sh::new(self.script()).argv()
}
}
pub struct Gzip;
impl Cmd for Gzip {
fn argv(&self) -> Argv {
Argv::new("gzip")
}
}
// busybox, which alpine based images ship, has no long options for these three
pub struct Gunzip;
impl Cmd for Gunzip {
fn argv(&self) -> Argv {
Argv::new("gunzip").arg("-c")
}
}
pub struct Head {
bytes: usize,
}
impl Head {
pub fn bytes(bytes: usize) -> Self {
Self { bytes }
}
}
impl Cmd for Head {
fn argv(&self) -> Argv {
Argv::new("head").arg("-c").arg(self.bytes.to_string())
}
}
pub struct Rm {
path: String,
}
impl Rm {
pub fn recursive(path: &str) -> Self {
Self {
path: path.to_string(),
}
}
}
impl Cmd for Rm {
fn argv(&self) -> Argv {
Argv::new("rm").arg("-rf").arg(&self.path)
}
}
#[cfg(test)]
mod tests {
use super::{Gunzip, Gzip, Head};
use crate::cmd::Cmd;
#[test]
fn a_pipeline_reports_a_stage_that_dies_mid_stream() {
let script = Gunzip.pipe(&Gzip).unwrap().shell().unwrap();
assert_eq!(script, "sh -c 'set -o pipefail; gunzip -c | gzip'");
}
#[test]
fn a_pipeline_that_closes_the_pipe_on_purpose_keeps_the_default() {
let script = Gunzip
.pipe(&Head::bytes(512))
.unwrap()
.allow_early_close()
.shell()
.unwrap();
assert_eq!(script, "sh -c 'gunzip -c | head -c 512'");
}
#[test]
fn a_pipeline_reaches_a_container_as_one_argument() {
let argv = Gunzip.pipe(&Gzip).unwrap().in_container("abc123").argv();
assert_eq!(
argv.words(),
[
"docker",
"exec",
"abc123",
"sh",
"-c",
"set -o pipefail; gunzip -c | gzip"
]
);
}
}

View File

@@ -1,82 +0,0 @@
use anyhow::{anyhow, Context, Result};
use std::{
collections::VecDeque,
ffi::OsStr,
fmt::Display,
fs::{File, OpenOptions},
path::PathBuf,
process::{Command, Stdio},
};
use crate::debug_println;
pub struct Args(Vec<String>);
impl From<&str> for Args {
fn from(value: &str) -> Self {
Self(Vec::from_iter(value.split_whitespace().map(String::from)))
}
}
impl From<&String> for Args {
fn from(value: &String) -> Self {
Self(Vec::from_iter(value.split_whitespace().map(String::from)))
}
}
#[derive(Default)]
pub struct CommandBuilder {
args: Vec<String>,
}
impl Display for CommandBuilder {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.args.join(" "))
}
}
impl CommandBuilder {
pub fn new(args: &str) -> Self {
Self::default().args("docker")
}
pub fn docker() -> Self {
Self::default().args("docker")
}
pub fn docker_compose() -> Self {
Self::default().args("docker compose -f docker/local/docker-compose.yaml")
}
pub fn args<T>(mut self, args: T) -> Self
where
Args: From<T>,
{
self.args.extend(Args::from(args).0);
self
}
fn build(self) -> Result<Command> {
debug_println!("\nran {self}\n");
let (first, rest) = self.args.split_first().context("empty args")?;
let mut command = Command::new(first);
command.args(rest);
Ok(command)
}
pub fn exec_get_stdout(mut self) -> Result<String> {
Ok(String::from_utf8(self.build()?.output()?.stdout)?)
}
pub fn exec(mut self) -> Result<()> {
self.build()?.spawn()?.wait()?;
Ok(())
}
pub fn exec_redirect_stdout(mut self, stdio: Stdio) -> Result<()> {
self.build()?.stdout(stdio).spawn()?.wait()?;
Ok(())
}
}

View File

@@ -0,0 +1,55 @@
use std::io::{self, Write};
use anyhow::{Context, Result};
use clap::CommandFactory;
use clap_complete::{Shell, generate};
use crate::cli::Ahab;
pub fn completions(shell: Shell) -> Result<()> {
let script = script(shell);
let mut stdout = io::stdout().lock();
match stdout.write_all(&script).and_then(|()| stdout.flush()) {
Err(e) if e.kind() == io::ErrorKind::BrokenPipe => Ok(()),
result => result.context("writing the completion script"),
}
}
fn script(shell: Shell) -> Vec<u8> {
let mut cmd = Ahab::command();
let name = cmd.get_name().to_string();
let mut script = Vec::new();
generate(shell, &mut cmd, name, &mut script);
script
}
#[cfg(test)]
mod tests {
use super::{Ahab, Shell, script};
use clap::{CommandFactory, ValueEnum};
#[test]
fn every_shell_gets_a_script_covering_the_subcommands() {
// asked of the parser rather than listed here, which is a list that
// silently stops covering the newest command
let subcommands: Vec<String> = Ahab::command()
.get_subcommands()
.map(|sub| sub.get_name().to_string())
.collect();
assert!(subcommands.len() > 1, "{subcommands:?}");
for shell in Shell::value_variants() {
let out = String::from_utf8(script(*shell)).expect("script is utf8");
for subcommand in &subcommands {
assert!(
out.contains(subcommand),
"{shell:?} script never mentions {subcommand}"
);
}
}
}
}

124
src/commands/django.rs Normal file
View File

@@ -0,0 +1,124 @@
use std::path::Path;
use anyhow::{Result, anyhow};
use crate::cmd::{Bash, Cmd, Manage, Words};
use crate::ctx::Ctx;
use crate::output::note;
use crate::project::Project;
const DEBUG_TEMPLATE: &str = r#"from django.core.management.base import BaseCommand
class Command(BaseCommand):
def handle(self, *args, **options):
pass
"#;
pub fn make_command(ctx: &Ctx, app: &Path, name: &str) -> Result<()> {
let app_name = app.to_string_lossy();
let app_dir = app;
// it becomes `<name>.py` under the app, and django imports it by this name,
// so anything that is not an identifier would land the file somewhere else
// or somewhere django will never look for it
if !is_module_name(name) {
return Err(anyhow!(
"{name:?} is not a usable command name; \
django imports it as a python module, so it can hold only \
letters, digits and underscores"
));
}
if !app_dir.is_dir() {
return Err(anyhow!("directory {app_name} does not exist"));
}
note!(ctx, "found app {app_name}");
let management_dir = app_dir.join("management");
if !management_dir.exists() {
ctx.fs().create_dir(&management_dir)?;
ctx.fs().touch(&management_dir.join("__init__.py"))?;
note!(ctx, "created module {app_name}.management")
};
let commands_dir = management_dir.join("commands");
if !commands_dir.exists() {
ctx.fs().create_dir(&commands_dir)?;
ctx.fs().touch(&commands_dir.join("__init__.py"))?;
note!(ctx, "created module {app_name}.management.commands")
};
ctx.fs().write_new(
&commands_dir.join(format!("{name}.py")),
DEBUG_TEMPLATE.as_bytes(),
)?;
note!(ctx, "created command {app_name}.management.commands.{name}");
Ok(())
}
fn is_module_name(name: &str) -> bool {
!name.is_empty()
&& !name.starts_with(|c: char| c.is_ascii_digit())
&& name.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')
}
pub fn bash(ctx: &Ctx) -> Result<()> {
Bash.in_service(ctx, &service(ctx)?).replace(ctx)
}
pub fn run(ctx: &Ctx, rest: &[String]) -> Result<()> {
Words::new(rest)
.in_service(ctx, &service(ctx)?)
.replace(ctx)
}
pub fn manage(ctx: &Ctx, rest: &[String]) -> Result<()> {
Manage::new(rest)
.in_service(ctx, &service(ctx)?)
.replace(ctx)
}
// shortcuts
pub fn makemigrations(ctx: &Ctx) -> Result<()> {
manage(ctx, &["makemigrations".to_string()])
}
pub fn migrate(ctx: &Ctx, rest: &[String]) -> Result<()> {
let mut args = vec!["migrate".to_string()];
args.extend_from_slice(rest);
manage(ctx, &args)
}
pub fn shell(ctx: &Ctx) -> Result<()> {
manage(ctx, &["shell".to_string()])
}
fn service(ctx: &Ctx) -> Result<String> {
Project::resolve(ctx)?.django()
}
#[cfg(test)]
mod tests {
use super::is_module_name;
#[test]
fn a_command_name_has_to_be_a_python_module_name() {
for name in ["report", "send_mail", "_private", "sync2"] {
assert!(is_module_name(name), "should be usable: {name}");
}
// a path would put the file somewhere other than the app
for name in ["../../../etc/cron.d/x", "a/b", "with space", "dash-ed", ""] {
assert!(!is_module_name(name), "should be refused: {name}");
}
}
}

475
src/commands/link.rs Normal file
View File

@@ -0,0 +1,475 @@
mod check;
mod store;
pub use check::check;
use fs_err::{read_dir, read_link};
use std::cell::Cell;
use std::path::{Path, PathBuf};
use anyhow::{Context, Result, anyhow, bail};
use self::store::{Leads, Repo, ignored, leads, resolve, symlink_metadata_opt, tracked};
use crate::cli::link as cli;
use crate::ctx::Ctx;
use crate::fsops::suffixed;
use crate::output::{line, note, plural, warning};
const BACKUP_SUFFIX: &str = ".ahab-bak";
// where the link waits while the payload comes back out of the store
const RESTORING_SUFFIX: &str = ".ahab-restoring";
// one path's failure is its own; several are counted, so a run over a list says
// what happened to each and then how the run as a whole went. a single path has
// nothing to count, so its error is simply the command's
fn each(paths: &[PathBuf], mut act: impl FnMut(&Path) -> Result<()>) -> Result<()> {
if let [only] = paths {
return act(only);
}
let mut failed = 0;
for path in paths {
if let Err(e) = act(path) {
warning!("{e:#}");
failed += 1;
}
}
match failed {
0 => Ok(()),
failed => Err(anyhow!("{failed} of {} paths failed", paths.len())),
}
}
// move untracked paths out of the repo and symlink them back
pub fn add(ctx: &Ctx, args: &cli::Add) -> Result<()> {
let repo = Repo::discover(ctx, args.store.root())?;
let report = Report::new(&repo);
each(&args.paths, |path| {
link_one(ctx, &repo, path, args.force, &report)
})
}
// move paths in the store back into the repo, the inverse of add
pub fn restore(ctx: &Ctx, args: &cli::Restore) -> Result<()> {
let repo = Repo::discover(ctx, args.store.root())?;
let report = Report::new(&repo);
// clap requires one or the other and refuses both, so only the two real
// cases are left here
let stored = match args.all {
true => stored_paths(&repo, &repo.store)?,
false => args
.paths
.iter()
.cloned()
.map(|path| (path, Stored::Linked))
.collect(),
};
// only a linked path can be moved back; the store can hold orphans too
let linked: Vec<PathBuf> = stored
.iter()
.filter(|(_, state)| *state == Stored::Linked)
.map(|(path, _)| path.clone())
.collect();
let skipped = stored.len() - linked.len();
if linked.is_empty() {
match skipped {
0 => note!(ctx, "nothing in the store for this repository"),
n => note!(
ctx,
"nothing to restore: {n} path{} in the store {} not linked, see `ahab link list`",
plural(n),
if n == 1 { "is" } else { "are" }
),
}
return Ok(());
}
if skipped > 0 {
note!(
ctx,
"leaving {skipped} path{} that the store holds but nothing links to",
plural(skipped)
);
}
each(&linked, |path| restore_one(ctx, &repo, path, &report))
}
fn restore_one(ctx: &Ctx, repo: &Repo, path: &Path, report: &Report) -> Result<()> {
let src = resolve(path)?;
let rel = repo.relative(&src)?;
let stored = repo.store.join(&rel);
let Some(meta) = symlink_metadata_opt(&src)? else {
bail!("{} does not exist", rel.display());
};
if !meta.is_symlink() {
bail!(
"{} is not a symlink, so it is not in the store",
rel.display()
);
}
let dest = read_link(&src)?;
if dest != stored {
bail!(
"{} points at {}, which is not where the store keeps it",
rel.display(),
dest.display()
);
}
if symlink_metadata_opt(&stored)?.is_none() {
bail!("{} is missing from the store", rel.display());
}
// the link is moved aside rather than removed: if the payload cannot come
// back out of the store, the repository is left pointing at where it still is
let aside = suffixed(&src, RESTORING_SUFFIX);
if symlink_metadata_opt(&aside)?.is_some() {
bail!("{} is in the way; move it aside", aside.display());
}
ctx.fs().rename(&src, &aside)?;
if let Err(e) = ctx.fs().move_path(&stored, &src) {
ctx.fs().rename(&aside, &src).with_context(|| {
format!(
"could not put the link at {} back after failing to restore it",
src.display()
)
})?;
return Err(e);
}
ctx.fs().remove_file(&aside)?;
ctx.fs().prune_empty(stored.parent(), &repo.base);
report.line("restored", &rel);
Ok(())
}
// what the repository looks like from the store's side
#[derive(PartialEq)]
enum Stored {
// the symlink is there and points at the store, which is the whole point
Linked,
// nothing is at the path the store holds it for
Missing,
// something else took the path, so the stored copy is the one nobody reads
Taken,
}
// the store mirrors the repository layout, so walking it finds every path this
// repository has put there without asking git anything
fn stored_paths(repo: &Repo, dir: &Path) -> Result<Vec<(PathBuf, Stored)>> {
let mut found = Vec::new();
let entries = match read_dir(dir) {
Ok(entries) => entries,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(found),
Err(e) => return Err(e.into()),
};
for entry in entries {
let stored = entry?.path();
let rel = stored.strip_prefix(&repo.store).with_context(|| {
format!(
"{} is not under the store {}",
stored.display(),
repo.store.display()
)
})?;
let src = repo.root.join(rel);
let state = match symlink_metadata_opt(&src)? {
None => Stored::Missing,
Some(meta) if !meta.is_symlink() => Stored::Taken,
Some(_) if read_link(&src).is_ok_and(|dest| dest == stored) => Stored::Linked,
Some(_) => Stored::Taken,
};
// a symlink the store happens to hold is a leaf, never a directory to
// walk into: is_dir would follow it and list whatever it points at
let holds_dir = symlink_metadata_opt(&stored)?.is_some_and(|meta| meta.is_dir());
// a linked directory is one entry; otherwise the paths inside it are
if state == Stored::Linked || !holds_dir {
found.push((src, state));
} else {
found.extend(stored_paths(repo, &stored)?);
}
}
found.sort_by(|(a, _), (b, _)| a.cmp(b));
Ok(found)
}
// what the store holds, without the git questions check asks
pub fn stored_summary(ctx: &Ctx, store: Option<&Path>) -> Result<(PathBuf, usize, usize)> {
let repo = Repo::discover(ctx, store)?;
let stored = stored_paths(&repo, &repo.store)?;
let linked = stored
.iter()
.filter(|(_, state)| *state == Stored::Linked)
.count();
Ok((repo.store, linked, stored.len() - linked))
}
// list what the store holds for this repository, the inverse of check
pub fn list(ctx: &Ctx, args: &cli::List) -> Result<()> {
let repo = Repo::discover(ctx, args.store.root())?;
let stored = stored_paths(&repo, &repo.store)?;
line!("store: {}", repo.store.display());
if stored.is_empty() {
line!("nothing in the store for this repository");
return Ok(());
}
for (path, state) in stored {
let rel = path.strip_prefix(&repo.root).unwrap_or(&path);
let verb = match state {
Stored::Linked => "linked",
Stored::Missing => "missing",
Stored::Taken => "shadowed",
};
entry(verb, rel);
}
Ok(())
}
// one line of what the store did with a path, in the column width `status`
// prints too: the two used to set it separately and had to be kept in step
fn entry(verb: &str, path: &Path) {
line!("\t{:<11}{}", format!("{verb}:"), path.display());
}
struct Report {
store: PathBuf,
named: Cell<bool>,
}
impl Report {
fn new(repo: &Repo) -> Self {
Self {
store: repo.store.clone(),
named: Cell::new(false),
}
}
fn line(&self, verb: &str, path: &Path) {
// worth naming once per run
if !self.named.replace(true) {
line!("store: {}", self.store.display());
}
entry(verb, path);
}
}
fn link_one(ctx: &Ctx, repo: &Repo, path: &Path, force: bool, report: &Report) -> Result<()> {
let src = resolve(path)?;
let rel = repo.relative(&src)?;
let target = repo.store.join(&rel);
// a target inside the repo would be readable from the sandbox anyway. the
// base is resolved as well as compared: one symlinked into the checkout
// passes a prefix test while landing the file straight back inside it
if target.starts_with(&repo.root) || matches!(leads(&repo.base, &repo.root), Leads::Inside) {
return Err(anyhow!(
"the store at {} is inside the repository {}; point --store or \
AHAB_LINK_ROOT somewhere else",
repo.base.display(),
repo.root.display()
));
}
stays_in_store(repo, &rel)?;
// before anything is moved in, so the tree it lands in is never briefly
// readable by anyone else
ctx.fs().ensure_private_parent(&repo.base, &target)?;
if tracked(ctx, repo, &rel)? {
return Err(anyhow!(
"{} is tracked by git; only untracked or ignored paths can be externalized",
rel.display()
));
}
match ignored(ctx, repo, &rel) {
Ok(true) => {}
Ok(false) => warning!("{} is not gitignored", rel.display()),
// saying "not gitignored" here would be an answer git never gave
Err(e) => warning!(
"could not tell whether {} is gitignored: {e:#}",
rel.display()
),
}
let src_meta = symlink_metadata_opt(&src)?;
let target_taken = symlink_metadata_opt(&target)?.is_some();
match src_meta {
Some(meta) if meta.is_symlink() => {
let dest = read_link(&src)?;
if dest == target {
if !target_taken {
return Err(anyhow!(
"{} already points at {}, but nothing is there",
rel.display(),
target.display()
));
}
report.line("unchanged", &rel);
return Ok(());
}
// nothing in the store to adopt, so the symlink itself moves out
if !target_taken {
// moving the link moves the pointer and leaves the contents
// where they are, so the store would hold a way back out and
// check, seeing a link into the store, would call it clean
if let Leads::Outside(end) = leads(&src, &repo.root) {
return Err(anyhow!(
"{} is a symlink to {}, outside the repository; \
externalizing it would move the link and leave its \
contents there, so repoint or remove it instead",
rel.display(),
end.display()
));
}
if !src.exists() {
warning!(
"{} is a broken symlink to {}",
rel.display(),
dest.display()
);
}
move_and_link(ctx, &src, &target)?;
report.line("moved", &rel);
return Ok(());
}
if !force {
return Err(needs_force(&target));
}
ctx.fs().place_link(&src, &target)?;
report.line("repointed", &rel);
Ok(())
}
Some(_) if target_taken => {
if !force {
return Err(needs_force(&target));
}
let backup = suffixed(&src, BACKUP_SUFFIX);
if symlink_metadata_opt(&backup)?.is_some() {
return Err(anyhow!(
"{} already exists; remove it before re-linking",
backup.display()
));
}
ctx.fs().rename(&src, &backup)?;
report.line("saved", &suffixed(&rel, BACKUP_SUFFIX));
ctx.fs().place_link(&src, &target)?;
report.line("linked", &rel);
Ok(())
}
Some(_) => {
move_and_link(ctx, &src, &target)?;
report.line("moved", &rel);
Ok(())
}
None if target_taken => {
if !force {
return Err(needs_force(&target));
}
ctx.fs().place_link(&src, &target)?;
report.line("linked", &rel);
Ok(())
}
None => Err(anyhow!(
"{} does not exist and the store has no {}",
rel.display(),
target.display()
)),
}
}
// the two halves have to end up looking like one step: with the payload moved
// but no link placed, the store holds a path nothing points at and the working
// tree has lost it altogether, which is the one outcome worse than failing
fn move_and_link(ctx: &Ctx, src: &Path, target: &Path) -> Result<()> {
ctx.fs().move_path(src, target)?;
if let Err(e) = ctx.fs().place_link(src, target) {
ctx.fs().move_path(target, src).with_context(|| {
format!(
"could not put {} back after failing to link it to {}",
src.display(),
target.display()
)
})?;
return Err(e);
}
Ok(())
}
// creating the store directories follows any symlink already standing in them,
// so a store that holds one would take the move somewhere else entirely; only
// the components at or below the store are examined, since everything above it
// is outside by definition
fn stays_in_store(repo: &Repo, rel: &Path) -> Result<()> {
let mut path = repo.store.clone();
for part in rel.components() {
path.push(part);
match symlink_metadata_opt(&path)? {
// nothing here yet, so nothing below it can be followed either
None => return Ok(()),
Some(meta) if meta.is_symlink() => {
if let Leads::Outside(end) = leads(&path, &repo.store) {
return Err(anyhow!(
"the store holds {} as a symlink to {}, outside the store; \
refusing to write through it",
path.display(),
end.display()
));
}
}
Some(_) => {}
}
}
Ok(())
}
fn needs_force(target: &Path) -> anyhow::Error {
anyhow!(
"{} already exists; pass --force to link to it",
target.display()
)
}

316
src/commands/link/check.rs Normal file
View File

@@ -0,0 +1,316 @@
use fs_err::{read_dir, read_link};
use std::ffi::OsString;
use std::os::unix::ffi::{OsStrExt, OsStringExt};
use std::path::{Path, PathBuf};
use anyhow::{Result, anyhow};
use super::store::{Leads, Repo, leads, resolve, symlink_metadata_opt};
use crate::cli::link as cli;
use crate::cmd::{Cmd, LsFiles};
use crate::ctx::Ctx;
use crate::output::{line, text, write_bytes};
// whether anything is outside the store, which main turns into an exit code
pub fn check(ctx: &Ctx, args: &cli::Check) -> Result<bool> {
let repo = Repo::discover(ctx, args.store.root())?;
let pathspecs = relative_pathspecs(&repo, &args.paths)?;
let mut exposed = Vec::new();
// git lists untracked and ignored separately
for (mark, ignored) in [(UNTRACKED, false), (IGNORED, true)] {
for entry in list_others(ctx, &repo, ignored, &pathspecs)? {
// --directory collapses a wholly untracked dir into `dir/`
match entry.strip_suffix(b"/") {
Some(dir) => exposed.extend(walk(&repo, &path_from(dir), mark)?.1),
None => exposed.extend(classify(&repo, &path_from(&entry), mark)?),
}
}
}
// and says nothing about tracked paths, where a symlink out still counts
for entry in list_tracked(ctx, &repo, &pathspecs)? {
exposed.extend(classify_tracked(&repo, &path_from(&entry))?);
}
exposed.sort_by(|a, b| a.name.cmp(&b.name));
if args.porcelain || args.null {
print_porcelain(&exposed, args.null);
} else {
print_listing(&repo, &exposed);
}
Ok(!exposed.is_empty())
}
fn print_porcelain(exposed: &[Exposed], null: bool) {
// -z is the format for scripts that must survive any filename, so its
// records are written as the bytes a path actually is. a filename can hold
// an arrow but not a NUL, as `git status -z` also assumes
if null {
for item in exposed {
let mut record = item.code().into_bytes();
record.push(b' ');
record.extend_from_slice(item.name.as_os_str().as_bytes());
if let Some(dest) = &item.dest {
record.push(0);
record.extend_from_slice(dest.as_os_str().as_bytes());
}
record.push(0);
write_bytes(&record);
}
return;
}
for item in exposed {
match &item.dest {
Some(dest) => text!(
"{} {} -> {}\n",
item.code(),
item.name.display(),
dest.display()
),
None => text!("{} {}\n", item.code(), item.name.display()),
}
}
}
fn print_listing(repo: &Repo, exposed: &[Exposed]) {
line!("store: {}", repo.store.display());
if exposed.is_empty() {
line!("nothing outside the store, a sandbox would see tracked files only");
return;
}
let sections = [
(
"Untracked paths a sandbox can read:",
" (use \"ahab link add <path>...\" to move them into the store)",
Section::Content(UNTRACKED),
),
(
"Ignored paths a sandbox can read:",
" (use \"ahab link add <path>...\" to move them into the store)",
Section::Content(IGNORED),
),
(
"Symlinks leading outside the store:",
" (their contents are not in the repository either way)",
Section::Elsewhere,
),
];
for (heading, hint, section) in sections {
let mut items = exposed.iter().filter(|item| section.holds(item)).peekable();
if items.peek().is_none() {
continue;
}
line!("\n{heading}\n{hint}");
for item in items {
match &item.dest {
Some(dest) => line!("\t{} -> {}", item.name.display(), dest.display()),
None => line!("\t{}", item.name.display()),
}
}
}
}
enum Section {
Content(char),
Elsewhere,
}
impl Section {
fn holds(&self, item: &Exposed) -> bool {
match self {
Self::Content(mark) => item.dest.is_none() && item.mark == *mark,
Self::Elsewhere => item.dest.is_some(),
}
}
}
// status codes as `git status --porcelain` spells them
const UNTRACKED: char = '?';
const IGNORED: char = '!';
const ELSEWHERE: char = '>';
// no git equivalent: a tracked path, which only ever appears as a symlink out
const TRACKED: char = 'T';
struct Exposed {
mark: char,
name: PathBuf,
dest: Option<PathBuf>,
}
impl Exposed {
fn content(mark: char, name: PathBuf) -> Self {
Self {
mark,
name,
dest: None,
}
}
fn code(&self) -> String {
let second = if self.dest.is_some() {
ELSEWHERE
} else {
self.mark
};
format!("{}{second}", self.mark)
}
}
fn classify(repo: &Repo, rel: &Path, mark: char) -> Result<Option<Exposed>> {
let src = repo.root.join(rel);
let name = rel.to_path_buf();
let Some(meta) = symlink_metadata_opt(&src)? else {
return Ok(None);
};
if !meta.is_symlink() {
return Ok(Some(Exposed::content(mark, name)));
}
let dest = read_link(&src)?;
if dest == repo.store.join(rel) {
// it names the store, but what the store holds there can be a symlink of
// its own leading straight back out, which is not being held at all
return Ok(match leads(&src, &repo.store) {
Leads::Inside | Leads::Dangling => None,
Leads::Outside(end) => Some(Exposed {
mark,
name,
dest: Some(end),
}),
});
}
Ok(Some(Exposed {
mark,
name,
dest: Some(dest),
}))
}
// git tracks symlinks, so one can lead out of the repository without ever
// showing up in an untracked or ignored listing; `add` cannot externalize it
// either, so all check can do is say it is there
fn classify_tracked(repo: &Repo, rel: &Path) -> Result<Option<Exposed>> {
let src = repo.root.join(rel);
let Some(meta) = symlink_metadata_opt(&src)? else {
return Ok(None);
};
if !meta.is_symlink() {
return Ok(None);
}
Ok(match leads(&src, &repo.root) {
Leads::Inside | Leads::Dangling => None,
Leads::Outside(end) => Some(Exposed {
mark: TRACKED,
name: rel.to_path_buf(),
dest: Some(end),
}),
})
}
fn walk(repo: &Repo, rel: &Path, mark: char) -> Result<(usize, Vec<Exposed>)> {
let dir = repo.root.join(rel);
let mut handled = 0;
let mut exposed = Vec::new();
for entry in read_dir(&dir)? {
let entry = entry?;
let child = rel.join(entry.file_name());
if entry.file_type()?.is_dir() {
let (below, inside) = walk(repo, &child, mark)?;
handled += below;
exposed.extend(inside);
continue;
}
match classify(repo, &child, mark)? {
Some(item) => exposed.push(item),
None => handled += 1,
}
}
// nothing below is in the store, so collapse to one line -- unless some of
// it leads out of the repository, which is a different thing to report and
// carries a destination the one line would drop
let all_content = exposed.iter().all(|item| item.dest.is_none());
if handled == 0 && !exposed.is_empty() && all_content {
let mut name = rel.as_os_str().to_owned();
name.push("/");
return Ok((0, vec![Exposed::content(mark, PathBuf::from(name))]));
}
Ok((handled, exposed))
}
fn relative_pathspecs(repo: &Repo, paths: &[PathBuf]) -> Result<Vec<PathBuf>> {
let mut specs = Vec::with_capacity(paths.len());
for path in paths {
let abs = resolve(path)?;
let rel = abs.strip_prefix(&repo.root).map_err(|_| {
anyhow!(
"{} is outside the repository {}",
abs.display(),
repo.root.display()
)
})?;
if rel.as_os_str().is_empty() {
return Ok(Vec::new());
}
specs.push(rel.to_path_buf());
}
Ok(specs)
}
fn list_others(
ctx: &Ctx,
repo: &Repo,
ignored: bool,
pathspecs: &[PathBuf],
) -> Result<Vec<Vec<u8>>> {
let mut listing = LsFiles::untracked(&repo.root).limited_to(pathspecs);
if ignored {
listing = listing.ignored();
}
Ok(split_nul(&listing.capture_bytes(ctx)?))
}
fn list_tracked(ctx: &Ctx, repo: &Repo, pathspecs: &[PathBuf]) -> Result<Vec<Vec<u8>>> {
let listing = LsFiles::tracked(&repo.root).limited_to(pathspecs);
Ok(split_nul(&listing.capture_bytes(ctx)?))
}
// every listing check reads is asked for with -z, and kept as the bytes git
// wrote: a path is not obliged to be utf-8, and one that is not used to fail
// the whole listing rather than the one entry
fn split_nul(out: &[u8]) -> Vec<Vec<u8>> {
out.split(|byte| *byte == 0)
.filter(|record| !record.is_empty())
.map(<[u8]>::to_vec)
.collect()
}
fn path_from(bytes: &[u8]) -> PathBuf {
PathBuf::from(OsString::from_vec(bytes.to_vec()))
}

429
src/commands/link/store.rs Normal file
View File

@@ -0,0 +1,429 @@
use fs_err as fs;
use std::env;
use std::ffi::{OsStr, OsString};
use std::os::unix::ffi::{OsStrExt, OsStringExt};
use std::path::{Component, Path, PathBuf};
use anyhow::{Context, Result, anyhow};
use crate::cmd::{CheckIgnore, Cmd, ConfigGet, LsFiles, RevParse};
use crate::ctx::Ctx;
use crate::output::note;
// a checkout with no remote to name it after
const LOCAL_NAMESPACE: &str = "_local";
pub(super) struct Repo {
pub(super) root: PathBuf,
pub(super) store: PathBuf,
// the configured store root, above the per-repository directories
pub(super) base: PathBuf,
}
impl Repo {
pub(super) fn discover(ctx: &Ctx, store: Option<&Path>) -> Result<Self> {
let root = git_root(ctx)?;
let base = normalized(match store {
Some(store) => store.to_path_buf(),
None => store_root()?,
})?;
Ok(Self {
store: base.join(repo_components(ctx, &root)?),
root,
base,
})
}
pub(super) fn relative(&self, src: &Path) -> Result<PathBuf> {
let rel = src.strip_prefix(&self.root).map_err(|_| {
anyhow!(
"{} is outside the repository {}",
src.display(),
self.root.display()
)
})?;
// empty means the whole repo
if rel.as_os_str().is_empty() {
return Err(anyhow!(
"refusing to externalize the repository root itself"
));
}
if rel.starts_with(".git") {
return Err(anyhow!("refusing to externalize anything under .git"));
}
Ok(rel.to_path_buf())
}
}
pub(super) fn resolve(path: &Path) -> Result<PathBuf> {
let abs = std::path::absolute(path)
.with_context(|| format!("resolving absolute path of {}", path.display()))?;
// the parent must exist so symlinked components resolve like git's toplevel
let Some(name) = abs.file_name().map(OsString::from) else {
return Ok(fs::canonicalize(&abs)?);
};
let parent = fs::canonicalize(abs.parent().unwrap_or(Path::new("/")))?;
Ok(parent.join(name))
}
// absolute, with `.` and `..` folded out. the store path is written into every
// symlink `add` creates, where a relative one would resolve from the link's own
// directory rather than the working one, and it is compared against the
// repository root, which a `..` would slip past
fn normalized(path: PathBuf) -> Result<PathBuf> {
let absolute = std::path::absolute(&path)
.with_context(|| format!("resolving absolute path of {}", path.display()))?;
let mut out = PathBuf::new();
for part in absolute.components() {
match part {
Component::CurDir => {}
Component::ParentDir => {
out.pop();
}
part => out.push(part),
}
}
Ok(out)
}
fn git_root(ctx: &Ctx) -> Result<PathBuf> {
// bytes, since the checkout can live at a path that is not utf-8, and with
// the cause kept: git not being installed and the directory not being a
// repository are different problems with the same one-line answer otherwise
let root = RevParse
.capture_bytes(ctx)
.context("asking git for the repository root")?;
let root = root.strip_suffix(b"\n").unwrap_or(&root);
if root.is_empty() {
return Err(anyhow!("git reported an empty repository root"));
}
Ok(fs::canonicalize(PathBuf::from(OsString::from_vec(
root.to_vec(),
)))?)
}
fn repo_components(ctx: &Ctx, root: &Path) -> Result<PathBuf> {
if let Some(url) = git_origin_url(ctx) {
if let Some(components) = components_from_remote(&url) {
return Ok(components);
}
note!(
ctx,
"could not parse git remote `{url}`, falling back to the checkout name"
);
}
let name = root
.file_name()
.ok_or_else(|| anyhow!("cannot derive a store path for {}", root.display()))?;
Ok(Path::new(LOCAL_NAMESPACE).join(sanitize_name(name)))
}
fn git_origin_url(ctx: &Ctx) -> Option<String> {
let url = ConfigGet {
key: "remote.origin.url",
}
.capture(ctx)
.ok()?;
let url = url.trim().to_string();
(!url.is_empty()).then_some(url)
}
fn components_from_remote(url: &str) -> Option<PathBuf> {
let url = url.trim();
let url = url.strip_suffix(".git").unwrap_or(url);
// `scheme://[user@]host[:port]/path`, or scp-like `[user@]host:path`
let (authority, path) = match url.split_once("://") {
Some((_, after)) => after.split_once('/')?,
None => url.split_once(':')?,
};
let host = authority.rsplit_once('@').map_or(authority, |(_, h)| h);
let host = host.split_once(':').map_or(host, |(h, _)| h);
// a local remote has no host to key on
if host.is_empty() {
return None;
}
let mut components = PathBuf::from(sanitize(&host.to_lowercase()));
let mut depth = 0;
for part in path.split('/').filter(|p| !p.is_empty()) {
components.push(sanitize(part));
depth += 1;
}
(depth > 0).then_some(components)
}
fn cleaned(s: &str) -> String {
let out: String = s
.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || matches!(c, '-' | '.' | '_') {
c
} else {
'_'
}
})
.collect();
// `.` and `..` are legal characters but not legal components
match out.chars().all(|c| c == '.') {
true => "_".repeat(out.len()),
false => out,
}
}
fn sanitize(s: &str) -> String {
let out = cleaned(s);
// every replaced character maps to the same `_`, so `my~api` and `my:api`
// would otherwise share one directory with the plain `my_api`. a component
// that came through untouched keeps its name, so the common remote keeps
// the store path it already has
match out == s {
true => out,
false => format!("{out}-{}", fingerprint(s.as_bytes())),
}
}
// a checkout name is bytes like any other path. a lossy rendering turns every
// byte it cannot read into the same replacement character, so sanitize would
// see two different names as one and fingerprint them identically: the bytes
// themselves are what has to be fingerprinted
fn sanitize_name(name: &OsStr) -> String {
match name.to_str() {
Some(text) => sanitize(text),
None => format!(
"{}-{}",
cleaned(&name.to_string_lossy()),
fingerprint(name.as_bytes())
),
}
}
// fnv-1a: the store path has to stay put across rust releases, which the hashers
// in std explicitly do not promise
fn fingerprint(bytes: &[u8]) -> String {
let mut hash: u64 = 0xcbf2_9ce4_8422_2325;
for byte in bytes {
hash ^= u64::from(*byte);
hash = hash.wrapping_mul(0x100_0000_01b3);
}
format!("{:08x}", hash as u32)
}
fn store_root() -> Result<PathBuf> {
if let Some(xdg) = non_empty_var("XDG_DATA_HOME") {
return Ok(PathBuf::from(xdg).join("ahab"));
}
let home = non_empty_var("HOME")
.filter(|v| !v.is_empty())
.ok_or_else(|| anyhow!("neither XDG_DATA_HOME nor HOME is set"))?;
Ok(PathBuf::from(home).join(".local/share/ahab"))
}
fn non_empty_var(name: &str) -> Option<OsString> {
env::var_os(name).filter(|v| !v.is_empty())
}
pub(super) fn tracked(ctx: &Ctx, repo: &Repo, rel: &Path) -> Result<bool> {
// bytes: the listing echoes back the path asked about, which is not obliged
// to be utf-8, and failing to read it would refuse the path for the wrong
// reason rather than answering whether git tracks it
let listed = LsFiles::tracked(&repo.root)
.limited_to(&[rel])
.capture_bytes(ctx)?;
Ok(!listed.is_empty())
}
// check-ignore says 0 for ignored and 1 for not; anything else means it could
// not answer at all, which is not the same as "not ignored". git cannot even be
// asked about a path whose name looks like pathspec magic, since it rejects the
// magic rather than the name, and `:(literal)` is not accepted by this command
pub(super) fn ignored(ctx: &Ctx, repo: &Repo, rel: &Path) -> Result<bool> {
let status = CheckIgnore::new(&repo.root, rel).probe_status(ctx)?;
match status.code() {
Some(0) => Ok(true),
Some(1) => Ok(false),
_ => Err(anyhow!("`git check-ignore` exited with {status}")),
}
}
pub(super) fn symlink_metadata_opt(path: &Path) -> Result<Option<std::fs::Metadata>> {
// symlink_metadata does not follow the link, so a symlink shows as one
match fs::symlink_metadata(path) {
Ok(meta) => Ok(Some(meta)),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None),
Err(e) => Err(e.into()),
}
}
// where a chain of symlinks actually ends up
pub(super) enum Leads {
// somewhere under the directory it was supposed to stay in
Inside,
// out of it, at this path
Outside(PathBuf),
// nowhere: a broken link, or too many hops for the kernel to follow
Dangling,
}
// comparing a link's target against an expected path only says what it claims;
// this says where following it really arrives, which is what decides whether a
// path is held by the store or merely points at something that is not
pub(super) fn leads(path: &Path, root: &Path) -> Leads {
let Ok(end) = fs::canonicalize(path) else {
return Leads::Dangling;
};
// the root can be reached through a symlink of its own, so resolve it too
// rather than comparing a resolved path against an unresolved prefix
let root = fs::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
match end.starts_with(&root) {
true => Leads::Inside,
false => Leads::Outside(end),
}
}
#[cfg(test)]
mod tests {
use super::{components_from_remote, fingerprint, normalized, sanitize, sanitize_name};
use std::collections::HashSet;
use std::ffi::OsStr;
use std::os::unix::ffi::OsStrExt;
use std::path::PathBuf;
#[test]
fn parses_every_spelling_of_a_remote() {
let cases = [
(
"git@git.aflabs.org:urnik/afurnik.git",
"git.aflabs.org/urnik/afurnik",
),
(
"https://git.aflabs.org/urnik/afurnik.git",
"git.aflabs.org/urnik/afurnik",
),
(
"https://git.aflabs.org/urnik/afurnik",
"git.aflabs.org/urnik/afurnik",
),
(
"https://git.aflabs.org/urnik/afurnik/",
"git.aflabs.org/urnik/afurnik",
),
(
"ssh://git@git.aflabs.org:22/urnik/afurnik.git",
"git.aflabs.org/urnik/afurnik",
),
(
"git@GIT.Aflabs.org:urnik/AFurnik.git",
"git.aflabs.org/urnik/AFurnik",
),
(
"git@git.aflabs.org:urnik/internal/afurnik.git",
"git.aflabs.org/urnik/internal/afurnik",
),
];
for (url, want) in cases {
assert_eq!(
components_from_remote(url),
Some(PathBuf::from(want)),
"url: {url}"
);
}
}
#[test]
fn rejects_remotes_without_a_host() {
assert_eq!(components_from_remote("not-a-url"), None);
assert_eq!(components_from_remote("/srv/git/afurnik.git"), None);
assert_eq!(components_from_remote("file:///srv/git/afurnik.git"), None);
assert_eq!(components_from_remote("https://git.aflabs.org/"), None);
}
#[test]
fn sanitize_never_yields_a_traversal() {
for name in ["..", ".", "a/b", "../..", "a/../b"] {
let out = sanitize(name);
assert!(!out.contains('/'), "{name} -> {out}");
assert!(out != "." && out != "..", "{name} -> {out}");
}
// a name that needed no replacing keeps the store path it already has
assert_eq!(sanitize(".env"), ".env");
assert_eq!(sanitize("afurnik"), "afurnik");
assert_eq!(sanitize("git.aflabs.org"), "git.aflabs.org");
}
#[test]
fn sanitize_keeps_names_apart_that_replacing_would_collapse() {
// every disallowed character maps to `_`, so without the fingerprint
// these would all share one store directory with a plain `my_api`
let names = ["my~api", "my:api", "my api", "my/api", "my%api"];
for name in names {
assert_ne!(sanitize(name), sanitize("my_api"), "name: {name}");
}
let distinct: HashSet<String> = names.iter().map(|name| sanitize(name)).collect();
assert_eq!(distinct.len(), names.len(), "{distinct:?}");
}
#[test]
fn a_fingerprint_does_not_drift_with_the_toolchain() {
// std's hashers make no such promise, and a moved store loses the files
assert_eq!(fingerprint(b""), "84222325");
assert_eq!(fingerprint(b"my~api"), fingerprint(b"my~api"));
assert_ne!(fingerprint(b"my~api"), fingerprint(b"my:api"));
}
#[test]
fn a_checkout_name_that_is_not_utf_8_keeps_its_own_directory() {
// both render to the same replacement character, so a fingerprint taken
// of the rendering rather than the bytes cannot tell them apart
let one = OsStr::from_bytes(b"proj\xe9");
let two = OsStr::from_bytes(b"proj\xff");
assert_ne!(sanitize_name(one), sanitize_name(two));
// and a name that is utf-8 is keyed exactly as before
assert_eq!(sanitize_name(OsStr::new("afurnik")), "afurnik");
}
#[test]
fn a_store_root_is_absolute_with_the_dots_folded_out() {
// it is written into every symlink add creates, and compared against the
// repository root, so it needs exactly one spelling
let cases = [
("/a/b/../c", "/a/c"),
("/a/./b", "/a/b"),
("/a/b/../../c", "/c"),
("/../..", "/"),
];
for (from, want) in cases {
assert_eq!(
normalized(PathBuf::from(from)).unwrap(),
PathBuf::from(want),
"from: {from}"
);
}
}
}

5
src/commands/mod.rs Normal file
View File

@@ -0,0 +1,5 @@
pub mod completions;
pub mod django;
pub mod link;
pub mod postgres;
pub mod status;

410
src/commands/postgres.rs Normal file
View File

@@ -0,0 +1,410 @@
mod server;
mod shape;
use std::io::{self, IsTerminal, Write};
use std::path::{Path, PathBuf};
use std::process::Stdio;
use anyhow::{Context, Result, bail};
use self::server::{Database, wait_until_ready, when_ready};
use self::shape::{Dump, HEADER_LEN, Kind};
use crate::cli::Format;
use crate::cli::postgres as cli;
use crate::cmd::{
Cmd, Cp, CreateDb, DropDb, Gunzip, Gzip, Head, PgDump, PgDumpAll, PgRestore, Psql, Rm,
};
use crate::ctx::Ctx;
use crate::fsops::{create_private_new, suffixed};
use crate::output::{note, plural, warning};
// unique per run: docker cp will not copy a directory over an existing path, and
// quietly leaves whatever was there for pg_restore to read instead
fn remote_dump() -> String {
// the container's /tmp is shared with whatever else runs in it, and a name
// that can be worked out in advance is one a symlink can be planted at
let spun = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map_or(0, |since| since.subsec_nanos());
format!("/tmp/ahab-dump-{}-{spun:09}", std::process::id())
}
// beside the target but unique to this run: a fixed name is a path a second
// dump would share, and one a symlink can be planted at ahead of time
fn partial_path(file: &Path) -> PathBuf {
suffixed(file, &format!(".{}.partial", std::process::id()))
}
// the database was dropped before the restore was attempted, so a restore that
// failed leaves nothing behind: said the same way wherever it happens, since
// what the user has to do about it does not depend on which tool it was
fn left_empty(tool: &str) -> anyhow::Error {
anyhow::anyhow!("{tool} failed, the database is left empty")
}
// pg_dumpall recreates roles the cluster already has, so this is the expected
// complaint rather than a failure
fn is_existing_role_error(line: &str) -> bool {
line.starts_with("ERROR:") && line.contains("role \"") && line.ends_with("already exists")
}
fn restore_cluster(ctx: &Ctx, db: &Database, restore: &dyn Cmd, file: &Path) -> Result<()> {
let out = restore
.in_container(&db.container)
.interactive()
.stdin_from_captured(ctx, file)
.context("running psql")?;
io::stdout().write_all(&out.stdout).ok();
let mut existing = 0;
let mut failed = 0;
for line in String::from_utf8_lossy(&out.stderr).lines() {
if is_existing_role_error(line) {
existing += 1;
continue;
}
// this psql runs without ON_ERROR_STOP, so it exits 0 whatever the sql
// did: these lines are the only account of what happened, which makes
// them the result rather than progress, and --quiet has to keep them
if line.starts_with("ERROR:") {
failed += 1;
warning!("{line}");
} else {
note!(ctx, "{line}");
}
}
if existing > 0 {
note!(
ctx,
"left {existing} existing role{} alone",
plural(existing)
);
}
if !out.status.success() {
return Err(left_empty("psql"));
}
if failed > 0 {
bail!(
"psql reported {failed} error{}, so the cluster restored only in part",
plural(failed)
);
}
Ok(())
}
// which shape is inside the compression, read through the container's own gunzip
// rather than assuming the host has one. this only reads, so it is a probe and
// runs even in a dry run, where the answer decides what the plan says
fn gzip_kind(ctx: &Ctx, db: &Database, file: &Path) -> Result<Kind> {
wait_until_ready(ctx, db)?;
let out = Gunzip
.pipe(&Head::bytes(HEADER_LEN))?
// head closes the pipe once it has its bytes, which kills gunzip
.allow_early_close()
.in_container(&db.container)
.interactive()
.probe_with_stdin(ctx, file)
.context("reading the compressed dump's header")?;
// head exits 0 whatever gunzip did, so an empty header is the only
// sign that decompression failed
if !out.status.success() || out.stdout.is_empty() {
let reason = String::from_utf8_lossy(&out.stderr);
let reason = reason.trim();
bail!(
"could not decompress {}: {}",
file.display(),
if reason.is_empty() {
"no data came out"
} else {
reason
}
);
}
Ok(Kind::of(&out.stdout))
}
pub fn import(ctx: &Ctx, file: &Path) -> Result<()> {
let dump = Dump::of(file)?;
let db = Database::resolve(ctx)?;
if matches!(dump, Dump::Directory) {
local_path_for_docker(file)?;
}
note!(ctx, "stopping all containers");
ctx.compose().stop().run(ctx)?;
// everything from here runs with the project down, so an error leaves it
// that way and the user has no reason to guess as much
imported(ctx, &db, &dump, file)
.map_err(|e| e.context("the project is left stopped, `docker compose up` starts it again"))
}
// docker cp reads `container:path`, splitting on the first colon, so a local
// path holding one is read as a container name and something else entirely
fn local_path_for_docker(file: &Path) -> Result<()> {
match file.to_string_lossy().contains(':') {
true => bail!(
"{} has a colon in it, which docker cp reads as a container name; \
rename it to copy it in or out",
file.display()
),
false => Ok(()),
}
}
fn imported(ctx: &Ctx, db: &Database, dump: &Dump, file: &Path) -> Result<()> {
note!(ctx, "starting db container");
ctx.compose().start(&db.service).run(ctx)?;
let remote = remote_dump();
// a directory cannot be streamed, so it is the one shape that gets copied in
if matches!(dump, Dump::Directory) {
when_ready(ctx, db, &Cp::into_container(file, &db.container, &remote))?;
}
let kind = match &dump {
Dump::Directory => Kind::Archive,
Dump::Header(header) => Kind::of(header),
Dump::Gzip => match gzip_kind(ctx, db, file) {
Ok(kind) => kind,
// looking inside needs a container to decompress with, and a dry run
// is worth printing with the project down, which is when it is most
// likely to be asked for
Err(e) if ctx.dry_run => {
note!(ctx, "planning for a single database dump: {e:#}");
Kind::Sql
}
Err(e) => return Err(e),
},
};
let restore = db.restore_with(kind);
// the name of what actually runs, so the message cannot drift from it
let tool = restore.argv().program().to_string();
note!(ctx, "restoring database with {tool}");
when_ready(
ctx,
db,
&DropDb {
username: &db.user,
dbname: &db.name,
}
.in_container(&db.container),
)?;
// a cluster dump creates the database itself, and would trip over one that
// is already there
if kind != Kind::Cluster {
when_ready(
ctx,
db,
&CreateDb {
username: &db.user,
dbname: &db.name,
}
.in_container(&db.container),
)?;
}
wait_until_ready(ctx, db)?;
let restored = restore_dump(ctx, db, dump, kind, file, &remote, &tool, restore);
// the copy inside the container holds the whole database, and its /tmp
// outlives the command: it goes whether or not the restore worked, which is
// exactly when it used to be left behind
if matches!(dump, Dump::Directory) {
let _ = Rm::recursive(&remote).in_container(&db.container).run(ctx);
}
restored?;
note!(ctx, "restarting containers");
ctx.compose().stop().run(ctx)?;
ctx.compose().up().run(ctx)?;
Ok(())
}
#[allow(clippy::too_many_arguments)]
fn restore_dump(
ctx: &Ctx,
db: &Database,
dump: &Dump,
kind: Kind,
file: &Path,
remote: &str,
tool: &str,
restore: Box<dyn Cmd + '_>,
) -> Result<()> {
if ctx.dry_run {
note!(ctx, "would restore with {tool}");
return Ok(());
}
// a directory dump was copied in whole, so pg_restore reads it from the
// container; every other shape is fed in on stdin, through gunzip when it
// arrives compressed
if matches!(dump, Dump::Directory) {
let status = PgRestore::new(&db.user, &db.name)
.from(remote)
.in_container(&db.container)
.status(ctx)?;
if !status.success() {
return Err(left_empty(tool));
}
return Ok(());
}
let restore: Box<dyn Cmd> = match dump {
Dump::Gzip => Box::new(Gunzip.pipe(&*restore)?),
_ => restore,
};
if kind == Kind::Cluster {
// psql's output is read rather than streamed here, to keep the
// expected role errors out of the way
return restore_cluster(ctx, db, &*restore, file);
}
let status = restore
.in_container(&db.container)
.interactive()
.stdin_from(ctx, file)?;
if !status.success() {
return Err(left_empty(tool));
}
Ok(())
}
pub fn psql(ctx: &Ctx, rest: &[String]) -> Result<()> {
let db = Database::resolve(ctx)?;
// a terminal only if this one has one, so `psql -c ... | cat` still works
Psql::new(&db.user, &db.name)
.args(rest)
.in_container(&db.container)
.interactive()
.tty(io::stdin().is_terminal())
.replace(ctx)
}
pub fn dump(ctx: &Ctx, args: &cli::Dump) -> Result<()> {
let (file, format, gzip) = (args.path.as_path(), args.format, args.gzip);
let db = Database::resolve(ctx)?;
if format == Format::Directory {
// main says this as an argument error before getting here; kept as the
// last word in case anything else ever calls dump
if gzip {
bail!("a directory dump is a directory of already compressed files, not a stream");
}
return dump_directory(ctx, &db, file);
}
note!(ctx, "dumping to local file {}", file.to_string_lossy());
// written beside the target and renamed once the dump succeeds, so a failure
// cannot destroy the dump that is already there
let partial = partial_path(file);
// a dry run produces no dump, so it must not lay a hand on the target either
let stdout = if ctx.dry_run {
Stdio::null()
} else {
Stdio::from(create_private_new(&partial)?)
};
let dumping = dump_command(&db, format);
let dumped = if gzip {
dumping
.pipe(&Gzip)?
.in_container(&db.container)
.stream_to(ctx, stdout)
} else {
dumping.in_container(&db.container).stream_to(ctx, stdout)
};
if let Err(e) = dumped {
let _ = ctx.fs().remove_file(&partial);
return Err(e);
}
ctx.fs().rename(&partial, file)?;
Ok(())
}
// pg_dump for one database, pg_dumpall for a cluster, which has no format letter
fn dump_command(db: &Database, format: Format) -> Box<dyn Cmd + '_> {
match PgDump::of(&db.user, &db.name, format) {
Some(dump) => Box::new(dump),
None => Box::new(PgDumpAll { username: &db.user }),
}
}
// pg_dump writes a directory format dump itself rather than to stdout, so it lands
// in the container and comes back with docker cp
fn dump_directory(ctx: &Ctx, db: &Database, target: &Path) -> Result<()> {
// symlink_metadata rather than exists(), which follows the link and so is
// false for a dangling one: docker cp would then write through it
if std::fs::symlink_metadata(target).is_ok() {
bail!(
"{} already exists; a directory dump will not be written over it",
target.display()
);
}
note!(ctx, "dumping to local directory {}", target.display());
let remote = remote_dump();
PgDump::of(&db.user, &db.name, Format::Directory)
.expect("a directory dump has a pg_dump letter")
.to(&remote)
.in_container(&db.container)
.run(ctx)?;
let copied = Cp::out_of_container(&db.container, &remote, target).run(ctx);
let _ = Rm::recursive(&remote).in_container(&db.container).run(ctx);
copied
}
#[cfg(test)]
mod tests {
use super::is_existing_role_error;
#[test]
fn only_the_existing_role_complaint_is_expected() {
assert!(is_existing_role_error(
r#"ERROR: role "postgres" already exists"#
));
for line in [
r#"ERROR: relation "things" already exists"#,
r#"ERROR: database "db" already exists"#,
"ERROR: syntax error at or near \"slect\"",
"NOTICE: role \"postgres\" already exists",
] {
assert!(!is_existing_role_error(line), "{line}");
}
}
}

View File

@@ -0,0 +1,98 @@
use std::thread;
use std::time::{Duration, Instant};
use anyhow::{Result, bail};
use super::shape::Kind;
use crate::cmd::{Cmd, PgIsReady, PgRestore, Psql};
use crate::ctx::Ctx;
use crate::project::Project;
const READY_TIMEOUT: Duration = Duration::from_secs(60);
const POLL_INTERVAL: Duration = Duration::from_secs(1);
// stands in for an id a dry run has no running container to look up
const PLANNED_CONTAINER: &str = "<db container>";
pub(super) struct Database {
pub(super) service: String,
pub(super) container: String,
pub(super) user: String,
pub(super) name: String,
}
impl Database {
pub(super) fn resolve(ctx: &Ctx) -> Result<Self> {
let compose = Project::resolve(ctx)?;
let service = compose.postgres()?;
let (user, name) = compose.postgres_credentials(&service)?;
let listed = ctx.compose().ps(&service).capture(ctx)?;
let mut ids = listed.lines().map(str::trim).filter(|id| !id.is_empty());
let container = match (ids.next(), ids.next()) {
(Some(id), None) => id.to_string(),
// one id per line: a scaled service has several, and the whole
// listing would go to docker exec as though it were a single id
(Some(_), Some(_)) => bail!(
"service {service} has more than one container running; \
scale it to one first"
),
// a dry run only prints a plan, and it is worth printing with the
// project down, which is when it is most likely to be asked for
(None, _) if ctx.dry_run => PLANNED_CONTAINER.to_string(),
(None, _) => bail!("service {service} has no running container"),
};
Ok(Self {
service,
container,
user,
name,
})
}
// what reads this shape of dump back in
pub(super) fn restore_with(&self, kind: Kind) -> Box<dyn Cmd + '_> {
match kind {
Kind::Archive => Box::new(PgRestore::new(&self.user, &self.name)),
Kind::Sql => Box::new(Psql::new(&self.user, &self.name).quiet().atomic()),
Kind::Cluster => Box::new(Psql::new(&self.user, "postgres").quiet()),
}
}
}
pub(super) fn wait_until_ready(ctx: &Ctx, db: &Database) -> Result<()> {
if ctx.dry_run {
return Ok(());
}
let deadline = Instant::now() + READY_TIMEOUT;
loop {
let ready = PgIsReady {
username: &db.user,
dbname: &db.name,
}
.in_container(&db.container)
.quietly_succeeds(ctx)?;
if ready {
return Ok(());
}
if Instant::now() >= deadline {
bail!(
"{} did not accept connections within {} seconds",
db.service,
READY_TIMEOUT.as_secs()
);
}
thread::sleep(POLL_INTERVAL);
}
}
pub(super) fn when_ready(ctx: &Ctx, db: &Database, command: &dyn Cmd) -> Result<()> {
wait_until_ready(ctx, db)?;
command.run(ctx)
}

View File

@@ -0,0 +1,104 @@
use fs_err::File;
use std::io::Read;
use std::path::Path;
use anyhow::Result;
const CUSTOM_MAGIC: &[u8] = b"PGDMP";
const TAR_MAGIC: &[u8] = b"toc.dat";
const GZIP_MAGIC: &[u8] = b"\x1f\x8b";
const CLUSTER_MARKER: &str = "PostgreSQL database cluster dump";
// wide enough for the cluster marker, which sits a few bytes into the file
pub(super) const HEADER_LEN: usize = 512;
pub(super) enum Dump {
Directory,
Gzip,
Header(Vec<u8>),
}
impl Dump {
pub(super) fn of(path: &Path) -> Result<Self> {
if path.is_dir() {
return Ok(Self::Directory);
}
let header = read_header(path)?;
if header.starts_with(GZIP_MAGIC) {
return Ok(Self::Gzip);
}
Ok(Self::Header(header))
}
}
// how the dump has to be fed back in: pg_restore for an archive, psql into the
// database for a single database dump, psql into postgres for a whole cluster
#[derive(Clone, Copy, PartialEq)]
pub(super) enum Kind {
Archive,
Sql,
Cluster,
}
impl Kind {
pub(super) fn of(header: &[u8]) -> Self {
if header.starts_with(CUSTOM_MAGIC) || header.starts_with(TAR_MAGIC) {
return Self::Archive;
}
if String::from_utf8_lossy(header).contains(CLUSTER_MARKER) {
return Self::Cluster;
}
Self::Sql
}
}
fn read_header(path: &Path) -> Result<Vec<u8>> {
let file = File::open(path)?;
let mut header = Vec::with_capacity(HEADER_LEN);
// read fills what it can rather than what was asked for, and a header that
// arrives in more than one piece would lose the cluster marker sitting a few
// bytes in, leaving a whole-cluster dump looking like a single database
file.take(HEADER_LEN as u64).read_to_end(&mut header)?;
Ok(header)
}
#[cfg(test)]
mod tests {
use super::{GZIP_MAGIC, Kind};
#[test]
fn an_archive_is_recognised_by_its_magic() {
assert!(matches!(Kind::of(b"PGDMP\x01\x0f"), Kind::Archive));
assert!(matches!(Kind::of(b"toc.dat\x00\x00"), Kind::Archive));
}
#[test]
fn a_cluster_dump_is_recognised_by_its_header() {
let header = b"--\n-- PostgreSQL database cluster dump\n--\n\n\\restrict abc\n";
assert!(matches!(Kind::of(header), Kind::Cluster));
}
#[test]
fn anything_else_is_a_single_database_dump() {
for header in [
&b"--\n-- PostgreSQL database dump\n"[..],
&b"BEGIN;"[..],
&b""[..],
&b"PGD"[..],
&b"toc.da"[..],
] {
assert!(matches!(Kind::of(header), Kind::Sql), "{header:?}");
}
}
#[test]
fn gzip_is_recognised_by_its_magic() {
assert!(b"\x1f\x8b\x08\x00rest".starts_with(GZIP_MAGIC));
assert!(!b"PGDMP".starts_with(GZIP_MAGIC));
}
}

92
src/commands/status.rs Normal file
View File

@@ -0,0 +1,92 @@
use std::path::Path;
use anyhow::Result;
use crate::cmd::Cmd;
use crate::commands::link;
use crate::ctx::Ctx;
use crate::output::line;
use crate::project::Project;
// which services ahab picked, what it would talk to, what the store holds
pub fn status(ctx: &Ctx, store: Option<&Path>) -> Result<()> {
// a role it cannot pick out is worth reporting; a project it cannot read is not
let project = match Project::resolve(ctx) {
Ok(project) => project,
Err(e) => {
stored(ctx, store);
return Err(e);
}
};
line!("services: {}", project.names().join(", "));
role(ctx, "django", project.django(), &project);
role(ctx, "postgres", project.postgres(), &project);
stored(ctx, store);
Ok(())
}
// what the store holds, and where to look for what it does not
fn stored(ctx: &Ctx, store: Option<&Path>) {
match link::stored_summary(ctx, store) {
Err(e) => line!("store: {e:#}"),
Ok((store, linked, other)) => {
line!("store: {}", store.display());
line!("\tlinked: {linked}");
if other > 0 {
line!("\tnot linked: {other} (see `ahab link list`)");
}
line!("\t`ahab link check` lists what a sandbox can still read");
}
}
}
// one detected service: the name, the image behind it, and whether it is up
fn role(ctx: &Ctx, role: &str, detected: Result<String>, project: &Project) {
let service = match detected {
Ok(service) => service,
Err(e) => {
line!("{role}: {e:#}");
return;
}
};
let image = project
.image(&service)
.unwrap_or("built from the project")
.to_string();
line!("{role}: {service} ({image})");
match ctx.compose().ps(&service).capture(ctx) {
Ok(id) if id.trim().is_empty() => line!("\tcontainer: not running"),
Ok(id) => line!("\tcontainer: {}", short(id.trim())),
Err(e) => line!("\tcontainer: {e:#}"),
}
if role == "postgres" {
// status reports what it cannot work out rather than stopping, so a name
// the postgres tools would refuse is something to say, not to fail on
match project.postgres_credentials(&service) {
Ok((user, database)) => {
let source = |key: &str| match project.env(&service, key) {
Some(_) => "",
None => " (default, nothing in the environment)",
};
line!("\tuser: {user}{}", source("POSTGRES_USER"));
line!("\tdatabase: {database}{}", source("POSTGRES_DB"));
}
Err(e) => line!("\tcredentials: {e:#}"),
}
}
}
// a container id is 64 characters and only the first few are ever typed
fn short(id: &str) -> String {
id.chars().take(12).collect()
}

22
src/ctx.rs Normal file
View File

@@ -0,0 +1,22 @@
use crate::cmd::Compose;
use crate::fsops::Fs;
// how ahab was invoked, handed to everything that runs commands or writes files
pub struct Ctx {
pub verbose: bool,
pub dry_run: bool,
pub quiet: bool,
}
impl Ctx {
// the two things these flags actually decide, asked for by name rather than
// read field by field wherever they are needed: a dry run writes nothing,
// and --quiet stops compose narrating
pub fn fs(&self) -> Fs<'_> {
Fs::new(self)
}
pub fn compose(&self) -> Compose<'_> {
Compose::new(self)
}
}

215
src/fsops.rs Normal file
View File

@@ -0,0 +1,215 @@
use std::io::Write;
use std::os::unix::fs::{DirBuilderExt, OpenOptionsExt};
use std::path::{Path, PathBuf};
use anyhow::{Context, Result, anyhow};
// std::fs with the path and the operation already in the error
use fs_err as fs;
use fs_err::os::unix::fs::symlink;
use crate::ctx::Ctx;
pub fn suffixed(path: &Path, suffix: &str) -> PathBuf {
let mut out = path.as_os_str().to_owned();
out.push(suffix);
PathBuf::from(out)
}
// the working tree, as this invocation is allowed to touch it. every write goes
// through here, so what a dry run means is settled in one place rather than at
// the top of each operation, where the next one added could simply not ask
pub struct Fs<'a> {
ctx: &'a Ctx,
}
impl<'a> Fs<'a> {
pub(crate) fn new(ctx: &'a Ctx) -> Self {
Self { ctx }
}
// a dry run writes nothing and reports that nothing went wrong, which for
// every operation here is its empty answer
fn unless_planned<T: Default>(&self, act: impl FnOnce() -> Result<T>) -> Result<T> {
match self.ctx.dry_run {
true => Ok(T::default()),
false => act(),
}
}
pub fn move_path(&self, src: &Path, target: &Path) -> Result<()> {
self.unless_planned(|| {
ensure_parent(target)?;
// rename cannot cross filesystems, and the store often is another one
match fs::rename(src, target) {
Ok(()) => Ok(()),
Err(rename_err) => match copy_recursive(src, target) {
Ok(()) => remove_recursive(src),
Err(copy_err) => Err(copy_err).with_context(|| format!("after {rename_err}")),
},
}
})
}
pub fn rename(&self, src: &Path, target: &Path) -> Result<()> {
self.unless_planned(|| Ok(fs::rename(src, target)?))
}
pub fn place_link(&self, link_path: &Path, target: &Path) -> Result<()> {
self.unless_planned(|| {
// the parent can be missing when the store holds a path the
// repository does not have any more, and symlink would only report
// the bare ENOENT
ensure_parent(link_path)?;
// symlink under a temp name and rename over the path: the rename is
// atomic
let tmp = suffixed(link_path, ".ahab-tmp");
// only ahab's own leftover is cleared away: anything else here
// belongs to the project, and silently unlinking it would lose it
match fs::symlink_metadata(&tmp) {
Ok(meta) if meta.is_symlink() => fs::remove_file(&tmp)?,
Ok(_) => {
return Err(anyhow!(
"{} is in the way and is not a symlink ahab left behind; \
move it aside",
tmp.display()
));
}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => return Err(e.into()),
}
symlink(target, &tmp)?;
Ok(fs::rename(&tmp, link_path)?)
})
}
pub fn remove_file(&self, path: &Path) -> Result<()> {
self.unless_planned(|| Ok(fs::remove_file(path)?))
}
pub fn create_dir(&self, path: &Path) -> Result<()> {
self.unless_planned(|| Ok(fs::create_dir(path)?))
}
// created if it is missing, left as it is otherwise, which is what a caller
// making an empty __init__.py wants
pub fn touch(&self, path: &Path) -> Result<()> {
self.unless_planned(|| {
fs::OpenOptions::new()
.write(true)
.create(true)
.truncate(false)
.open(path)?;
Ok(())
})
}
// fails rather than writing over a file that is already there
pub fn write_new(&self, path: &Path, contents: &[u8]) -> Result<()> {
self.unless_planned(|| {
let mut file = fs::File::create_new(path)?;
file.write_all(contents)?;
Ok(())
})
}
// the store holds what should not be reachable from the repository, so the
// directories it is kept in are the owner's alone. only the tree at or
// below `base` is created here; what is above it is the user's own business
pub fn ensure_private_parent(&self, base: &Path, target: &Path) -> Result<()> {
self.unless_planned(|| {
let Some(parent) = target.parent().filter(|dir| dir.starts_with(base)) else {
return Ok(());
};
// a recursive create applies the mode to every directory it makes
std::fs::DirBuilder::new()
.recursive(true)
.mode(0o700)
.create(parent)
.with_context(|| format!("creating {}", parent.display()))
})
}
// a restored path can leave the store holding nothing but empty directories
pub fn prune_empty(&self, dir: Option<&Path>, stop: &Path) {
let pruned: Result<()> = self.unless_planned(|| {
let mut dir = dir;
while let Some(path) = dir {
if path == stop || !path.starts_with(stop) || fs::remove_dir(path).is_err() {
break;
}
dir = path.parent();
}
Ok(())
});
// an empty directory left behind is untidy, not a failure
let _ = pruned;
}
}
// 0600 and only where nothing is yet. a dump is the whole database, and
// pg_dumpall's is every role's password hash, so it is not the owner's to share
// by default; create_new also refuses to follow a symlink planted at the path,
// which File::create would open and truncate.
//
// not on Fs: a dry run has no file to hand back, so its one caller asks for
// this only once it knows it is writing for real
pub fn create_private_new(path: &Path) -> Result<std::fs::File> {
std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o600)
.open(path)
.with_context(|| format!("creating {}", path.display()))
}
fn ensure_parent(target: &Path) -> Result<()> {
match target.parent() {
Some(parent) if !parent.as_os_str().is_empty() => Ok(fs::create_dir_all(parent)?),
_ => Ok(()),
}
}
// whatever is there, file, directory or symlink
fn remove_recursive(path: &Path) -> Result<()> {
if fs::symlink_metadata(path)?.is_dir() {
fs::remove_dir_all(path)?;
} else {
fs::remove_file(path)?;
}
Ok(())
}
fn copy_recursive(src: &Path, target: &Path) -> Result<()> {
let meta = fs::symlink_metadata(src)?;
if meta.is_dir() {
fs::create_dir_all(target)?;
for entry in fs::read_dir(src)? {
let entry = entry?;
copy_recursive(&entry.path(), &target.join(entry.file_name()))?;
}
return Ok(());
}
if meta.is_symlink() {
return Ok(symlink(fs::read_link(src)?, target)?);
}
fs::copy(src, target)?;
Ok(())
}

View File

@@ -1,24 +0,0 @@
#![allow(unused)]
use std::{
fs::{File, OpenOptions},
path::PathBuf,
};
pub mod cli;
pub mod command_builder;
pub mod scripts;
// NOTE: stolen from https://docs.rs/debug_print/latest/debug_print/
#[macro_export]
macro_rules! debug_println {
($($arg:tt)*) => (if ::std::cfg!(debug_assertions) { ::std::println!($($arg)*); })
}
fn safe_create_file(path: PathBuf) -> Result<File, std::io::Error> {
OpenOptions::new().write(true).create_new(true).open(path)
}
fn create_file(path: PathBuf) -> Result<File, std::io::Error> {
OpenOptions::new().write(true).create(true).open(path)
}

View File

@@ -1,50 +1,114 @@
use ahab::{cli, scripts};
use std::process::ExitCode;
mod cli;
mod cmd;
mod commands;
mod ctx;
mod fsops;
mod output;
mod project;
use anyhow::Result;
use clap::{CommandFactory, Parser};
fn main() -> Result<()> {
// always load dotenv on start
dotenvy::dotenv().ok();
use crate::ctx::Ctx;
use crate::output::{note, write_err};
// TODO: (matej) some prints about project
// eg: DJANGO_SETTINGS_MODULE for django projects, used docker-compose, etc.
// 0 ran with nothing to report, 1 could not finish, 2 bad arguments, 4 found something
const FINDINGS: u8 = 4;
fn main() -> ExitCode {
let args = cli::Ahab::parse();
match args.command {
cli::Commands::Completion { shell } => {
clap_complete::generate(
shell,
&mut cli::Ahab::command(),
"ahab",
&mut std::io::stdout(),
);
Ok(())
let ctx = Ctx {
verbose: args.verbose,
dry_run: args.dry_run,
quiet: args.quiet,
};
// said once here rather than by each command, so every line that follows
// reads as the plan it is
if ctx.dry_run {
note!(ctx, "dry run, nothing will be changed");
}
// the command's own verdict first, then whether it managed to say it
match run(&ctx, args.command).and_then(|code| output::delivered().map(|()| code)) {
Ok(code) => code,
Err(e) => {
write_err(format_args!("Error: {e:#}"));
ExitCode::FAILURE
}
}
}
// reported the way clap reports its own argument errors, which is what makes it
// exit 2 rather than 1
fn usage(message: &str) -> ! {
cli::Ahab::command()
.error(clap::error::ErrorKind::ArgumentConflict, message)
.exit()
}
fn run(ctx: &Ctx, command: cli::Commands) -> Result<ExitCode> {
let done = ExitCode::SUCCESS;
match command {
cli::Commands::Django { command } => {
match command {
cli::Django::Bash => commands::django::bash(ctx),
cli::Django::Run { rest } => commands::django::run(ctx, &rest),
cli::Django::MakeCommand { app, name } => {
commands::django::make_command(ctx, &app, &name)
}
cli::Django::Makemigrations => commands::django::makemigrations(ctx),
cli::Django::Manage { rest } => commands::django::manage(ctx, &rest),
cli::Django::Migrate { rest } => commands::django::migrate(ctx, &rest),
cli::Django::Shell => commands::django::shell(ctx),
}?;
Ok(done)
}
cli::Commands::Postgres { command } => {
match command {
cli::Postgres::Import { path } => commands::postgres::import(ctx, &path),
cli::Postgres::Psql { rest } => commands::postgres::psql(ctx, &rest),
cli::Postgres::Dump(args) => {
// clap cannot say that a flag conflicts with one value of
// another, and this is still an argument error: it belongs
// with the usage and the exit code the others get
if args.gzip && args.format == cli::Format::Directory {
usage(
"a directory dump is a directory of already \
compressed files, not a stream",
);
}
commands::postgres::dump(ctx, &args)
}
}?;
Ok(done)
}
cli::Commands::Link { command } => match command {
cli::Link::Add(args) => commands::link::add(ctx, &args).map(|()| done),
cli::Link::List(args) => commands::link::list(ctx, &args).map(|()| done),
cli::Link::Restore(args) => commands::link::restore(ctx, &args).map(|()| done),
// the one command with something to say through its exit code
cli::Link::Check(args) => match commands::link::check(ctx, &args)? && args.exit_code {
true => Ok(ExitCode::from(FINDINGS)),
false => Ok(done),
},
},
cli::Commands::Status { store } => {
commands::status::status(ctx, store.root.as_deref())?;
Ok(done)
}
cli::Commands::Completions { shell } => {
commands::completions::completions(shell)?;
Ok(done)
}
cli::Commands::Docker { command } => match command {
cli::Docker::StopAll => scripts::docker::stop_all(),
},
cli::Commands::Compose { command } => match command {
cli::DockerCompose::Build => scripts::docker_compose::build(),
cli::DockerCompose::Down => scripts::docker_compose::down(),
cli::DockerCompose::Rebuild => scripts::docker_compose::rebuild(),
cli::DockerCompose::Restart => scripts::docker_compose::restart(),
cli::DockerCompose::Start => scripts::docker_compose::start(None),
cli::DockerCompose::Stop => scripts::docker_compose::stop(),
cli::DockerCompose::Up => scripts::docker_compose::up(),
},
cli::Commands::Django { command } => match command {
cli::Django::MakeCommand { app, name } => scripts::django::make_command(&app, &name),
cli::Django::Makemigrations => scripts::django::makemigrations(),
cli::Django::Manage { rest } => scripts::django::manage(&rest),
cli::Django::Migrate { rest } => scripts::django::migrate(&rest),
cli::Django::Shell => scripts::django::shell(),
cli::Django::Test => scripts::django::test(),
},
cli::Commands::Postgres { command } => match command {
cli::Postgres::Import { path } => scripts::postgres::import(&path),
cli::Postgres::Dump { path } => scripts::postgres::dump(&path),
},
}
}

117
src/output.rs Normal file
View File

@@ -0,0 +1,117 @@
use std::fmt::Arguments;
use std::io::{self, Write};
use std::sync::OnceLock;
use std::sync::atomic::{AtomicU8, Ordering};
use anyhow::{Result, anyhow};
// progress, on stderr and only when it was asked for
macro_rules! note {
($ctx:expr, $($arg:tt)*) => {
if !$ctx.quiet {
$crate::output::write_err(format_args!($($arg)*))
}
};
}
// not progress: it speaks about the result, so --quiet keeps it
macro_rules! warning {
($($arg:tt)*) => {
$crate::output::write_err(format_args!("warning: {}", format_args!($($arg)*)))
};
}
// what a command was asked for, on stdout
macro_rules! line {
($($arg:tt)*) => { $crate::output::write_line(format_args!($($arg)*)) };
}
// the same, for a caller that terminates its own entries
macro_rules! text {
($($arg:tt)*) => { $crate::output::write_text(format_args!($($arg)*)) };
}
// stdout is open, or it is not and why
const OPEN: u8 = 0;
// the reader left: nothing is wrong, there is just nowhere to write
const CLOSED: u8 = 1;
// the write itself failed, so what was asked for was never delivered
const FAILED: u8 = 2;
static STDOUT: AtomicU8 = AtomicU8::new(OPEN);
static REASON: OnceLock<String> = OnceLock::new();
pub(crate) fn write_line(args: Arguments) {
if writable() {
finish(writeln!(io::stdout(), "{args}"));
}
}
pub(crate) fn write_text(args: Arguments) {
if writable() {
finish(write!(io::stdout(), "{args}"));
}
}
// a path on unix is bytes, not text, so a record naming one is written as bytes
pub(crate) fn write_bytes(bytes: &[u8]) {
if writable() {
finish(io::stdout().write_all(bytes));
}
}
// stderr is commentary about the work, not the work itself. there is nowhere to
// report that it could not be written, so the error goes nowhere -- eprintln!
// panics instead, which turns a closed pipe into a crash
pub(crate) fn write_err(args: Arguments) {
let _ = writeln!(io::stderr(), "{args}");
}
fn writable() -> bool {
STDOUT.load(Ordering::Relaxed) == OPEN
}
// a failed write must not end the process: a command halfway through moving
// files would leave the rest undone and still report the success it had planned
// on. writing stops, the command runs to its end, and main asks how it went
fn finish(written: io::Result<()>) {
let Err(e) = written else {
return;
};
match e.kind() {
io::ErrorKind::BrokenPipe => STDOUT.store(CLOSED, Ordering::Relaxed),
_ => {
STDOUT.store(FAILED, Ordering::Relaxed);
let _ = REASON.set(e.to_string());
}
}
}
// whether everything a command was asked for actually reached stdout. the
// buffered writes text! makes are flushed here rather than by the runtime after
// main returns, which discards the error and truncates the output in silence
pub(crate) fn delivered() -> Result<()> {
if writable() {
finish(io::stdout().flush());
}
if STDOUT.load(Ordering::Relaxed) != FAILED {
return Ok(());
}
Err(match REASON.get() {
Some(reason) => anyhow!("writing to stdout failed: {reason}"),
None => anyhow!("writing to stdout failed"),
})
}
pub(crate) use {line, note, text, warning};
// the `s` on a counted noun, so a message reads right for one and for many
pub(crate) fn plural(n: usize) -> &'static str {
match n {
1 => "",
_ => "s",
}
}

334
src/project.rs Normal file
View File

@@ -0,0 +1,334 @@
use anyhow::{Context, Result, anyhow};
use serde_json::Value;
use crate::cmd::Cmd;
use crate::ctx::Ctx;
const POSTGRES_IMAGES: [&str; 4] = ["postg", "timescale", "pgvector", "citus"];
const DJANGO_SETTINGS_MODULE: &str = "DJANGO_SETTINGS_MODULE";
pub struct Project {
services: Value,
}
impl Project {
pub fn resolve(ctx: &Ctx) -> Result<Self> {
let json = ctx.compose().config().capture(ctx)?;
let config: Value = serde_json::from_str(&json).context("parsing docker compose config")?;
let services = config
.get("services")
.cloned()
.context("compose config has no services")?;
Ok(Self { services })
}
pub fn django(&self) -> Result<String> {
let candidates: Vec<&str> = self
.names()
.into_iter()
.filter(|name| {
let service = &self.services[name];
service.get("build").is_some() && env_var(service, DJANGO_SETTINGS_MODULE).is_some()
})
.collect();
match candidates.as_slice() {
[] => Err(anyhow!(
"no service both builds an image and sets {DJANGO_SETTINGS_MODULE}, looked at {}",
self.names().join(", ")
)),
[only] => Ok(only.to_string()),
several => {
let serving: Vec<&&str> = several
.iter()
.filter(|name| self.publishes_ports(name))
.collect();
match serving.as_slice() {
[only] => Ok(only.to_string()),
// publishing ports is the tie-break, so say which way it
// failed: none of them serving is the opposite complaint
// from all of them serving
[] => Err(anyhow!(
"cannot tell which service runs django, {} all build an image \
and set {DJANGO_SETTINGS_MODULE}, and none publishes ports \
to tell them apart",
several.join(", ")
)),
_ => Err(anyhow!(
"cannot tell which service runs django, {} all build an image, set \
{DJANGO_SETTINGS_MODULE} and publish ports",
serving
.iter()
.map(|name| **name)
.collect::<Vec<&str>>()
.join(", ")
)),
}
}
}
}
pub fn postgres(&self) -> Result<String> {
let candidates: Vec<&str> = self
.names()
.into_iter()
.filter(|name| {
self.services[name]
.get("image")
.and_then(Value::as_str)
.is_some_and(is_postgres_image)
})
.collect();
match candidates.as_slice() {
[] => Err(anyhow!(
"no service uses a postgres image, looked at {}",
self.names().join(", ")
)),
[only] => Ok(only.to_string()),
several => Err(anyhow!(
"cannot tell which service is the database, {} all use a postgres image",
several.join(", ")
)),
}
}
pub fn postgres_credentials(&self, service: &str) -> Result<(String, String)> {
let entry = &self.services[service];
let user = env_var(entry, "POSTGRES_USER").unwrap_or_else(|| "db".to_string());
let database = env_var(entry, "POSTGRES_DB").unwrap_or_else(|| "db".to_string());
Ok((
usable_name(service, "POSTGRES_USER", user)?,
usable_name(service, "POSTGRES_DB", database)?,
))
}
pub fn names(&self) -> Vec<&str> {
let mut names: Vec<&str> = self
.services
.as_object()
.map(|services| services.keys().map(String::as_str).collect())
.unwrap_or_default();
names.sort_unstable();
names
}
// the image a service runs, which a service that builds its own does not have
pub fn image(&self, service: &str) -> Option<&str> {
self.services[service].get("image")?.as_str()
}
// whether a value was set in the project or is ahab's own fallback
pub fn env(&self, service: &str, key: &str) -> Option<String> {
env_var(&self.services[service], key)
}
fn publishes_ports(&self, service: &str) -> bool {
self.services[service]
.get("ports")
.and_then(Value::as_array)
.is_some_and(|ports| !ports.is_empty())
}
}
fn env_var(service: &Value, key: &str) -> Option<String> {
service
.get("environment")?
.get(key)?
.as_str()
.map(str::to_string)
}
// the compose file chooses these, and every postgres tool ahab runs takes them
// as a role or database name. libpq reads a name holding `=` or a uri as a whole
// connection string, so one could send a dump to another server entirely, and a
// leading dash is read as an option however the argv is quoted
fn usable_name(service: &str, key: &str, value: String) -> Result<String> {
let wrong = if value.is_empty() {
Some("is empty")
} else if value.starts_with('-') {
Some("starts with a dash, which the postgres tools read as an option")
} else if value.contains('=') {
Some("holds an `=`, which libpq reads as a connection string")
} else if value.contains("://") {
Some("holds a url, which libpq reads as a connection string")
} else if value.chars().any(|c| c.is_whitespace() || c.is_control()) {
Some("holds whitespace")
} else {
None
};
match wrong {
None => Ok(value),
Some(wrong) => Err(anyhow!(
"{key} of service {service} {wrong}: {value:?}. \
ahab passes it to psql, pg_dump and dropdb as a name, \
so it has to be one"
)),
}
}
fn is_postgres_image(image: &str) -> bool {
let image = image.to_lowercase();
POSTGRES_IMAGES.iter().any(|kind| image.contains(kind))
}
#[cfg(test)]
mod tests {
use super::{Project, is_postgres_image, usable_name};
use serde_json::json;
fn compose(services: serde_json::Value) -> Project {
Project { services }
}
#[test]
fn a_name_the_postgres_tools_would_read_as_something_else_is_refused() {
// libpq takes a dbname holding `=` or a url as a whole conninfo string,
// so a compose file could otherwise pick the server a dump goes to
let refused = [
"postgresql://myproject:pw@evil.example.net:5432/loot",
"host=evil.example.net dbname=loot",
"--file=/var/lib/postgresql/data/pg_hba.conf",
"-h evil.example.net",
"my db",
"",
];
for value in refused {
assert!(
usable_name("db", "POSTGRES_DB", value.to_string()).is_err(),
"should have been refused: {value:?}"
);
}
// and the names real projects actually use are left alone
for value in ["myproject_db", "my-db", "db", "app.prod", "DB_2"] {
assert_eq!(
usable_name("db", "POSTGRES_DB", value.to_string()).unwrap(),
value,
"should have been accepted: {value:?}"
);
}
}
#[test]
fn recognises_postgres_flavours() {
for image in [
"postgres:18",
"postgis/postgis:16-3.4",
"timescale/timescaledb:latest-pg16",
"pgvector/pgvector:pg16",
"citusdata/citus:12",
"PostgreSQL:16",
] {
assert!(is_postgres_image(image), "{image} should count as postgres");
}
for image in ["redis:7-alpine", "mysql:8", "nginx", "local-django"] {
assert!(!is_postgres_image(image), "{image} should not");
}
}
#[test]
fn picks_the_service_that_builds_and_has_the_settings_module() {
let compose = compose(json!({
"db": {"image": "postgres:18", "environment": {"POSTGRES_DB": "n"}},
"django": {
"build": {"context": "."},
"environment": {"DJANGO_SETTINGS_MODULE": "config.settings.local"},
"ports": [{"target": 8000}],
},
}));
assert_eq!(compose.django().unwrap(), "django");
assert_eq!(compose.postgres().unwrap(), "db");
}
#[test]
fn breaks_a_worker_tie_on_published_ports() {
let compose = compose(json!({
"django": {
"build": {"context": "."},
"environment": {"DJANGO_SETTINGS_MODULE": "config.settings.local"},
"ports": [{"target": 8000}],
},
"worker": {
"build": {"context": "."},
"environment": {"DJANGO_SETTINGS_MODULE": "config.settings.local"},
},
}));
assert_eq!(compose.django().unwrap(), "django");
}
#[test]
fn refuses_when_two_candidates_both_serve() {
let serving = json!({
"build": {"context": "."},
"environment": {"DJANGO_SETTINGS_MODULE": "config.settings.local"},
"ports": [{"target": 8000}],
});
let compose = compose(json!({"api": serving, "web": serving}));
let error = compose.django().unwrap_err().to_string();
assert!(error.contains("api, web"), "{error}");
}
#[test]
fn refuses_when_nothing_matches() {
let compose = compose(json!({
"db": {"image": "postgres:18"},
"cache": {"image": "redis:7"},
}));
assert!(
compose
.django()
.unwrap_err()
.to_string()
.contains("cache, db")
);
assert_eq!(compose.postgres().unwrap(), "db");
}
#[test]
fn refuses_two_postgres_services() {
let compose = compose(json!({
"db": {"image": "postgres:18"},
"replica": {"image": "postgres:18"},
}));
let error = compose.postgres().unwrap_err().to_string();
assert!(error.contains("db, replica"), "{error}");
}
#[test]
fn reads_credentials_off_the_detected_service() {
let compose = compose(json!({
"db": {
"image": "postgres:18",
"environment": {"POSTGRES_USER": "myproject", "POSTGRES_DB": "myproject_db"},
},
}));
let (user, database) = compose.postgres_credentials("db").unwrap();
assert_eq!(
(user.as_str(), database.as_str()),
("myproject", "myproject_db")
);
}
#[test]
fn falls_back_to_db_when_the_service_says_nothing() {
let compose = compose(json!({"db": {"image": "postgres:18"}}));
let (user, database) = compose.postgres_credentials("db").unwrap();
assert_eq!((user.as_str(), database.as_str()), ("db", "db"));
}
}

View File

@@ -1,89 +0,0 @@
use std::env;
use std::fs::create_dir;
use std::io::Write;
use std::path::{Path, PathBuf};
use anyhow::{anyhow, Result};
use crate::command_builder::CommandBuilder;
use crate::{create_file, safe_create_file};
fn get_django_settings_module() -> Result<String> {
let dsm = env::var("DJANGO_SETTINGS_MODULE")?;
println!("USING: {dsm}");
Ok(dsm)
}
const DEBUG_TEMPLATE: &str = r#"from django.core.management.base import BaseCommand
class Command(BaseCommand):
def handle(self, *args, **options):
pass
"#;
pub fn make_command(app: &PathBuf, name: &str) -> Result<()> {
let app_name = app.to_string_lossy();
let app_dir = Path::new(&app);
let not_app_exists = !app_dir.is_dir();
if not_app_exists {
return Err(anyhow!("directory {app_name} does not exist"));
}
println!("found app {app_name}");
let management_dir = app_dir.join("management");
let not_management_exists = !management_dir.exists();
if not_management_exists {
create_dir(&management_dir)?;
create_file(management_dir.join("__init__.py"))?;
println!("created module {app_name}.management")
};
let commands_dir = management_dir.join("commands");
let not_commands_exists = !commands_dir.exists();
if not_commands_exists {
create_dir(&commands_dir)?;
create_file(commands_dir.join("__init__.py"))?;
println!("created module {app_name}.management.commands")
};
let mut file = safe_create_file(commands_dir.join(format!("{name}.py")))?;
file.write_all(DEBUG_TEMPLATE.as_bytes())?;
println!("created command {app_name}.management.commands.{name}");
Ok(())
}
pub fn manage(rest: &[String]) -> Result<()> {
let dsm = get_django_settings_module()?;
let joined = rest.join(" ");
let command = format!("exec appserver python manage.py {joined} --settings={dsm}");
CommandBuilder::docker_compose().args(&command).exec()
}
// shortcuts
pub fn makemigrations() -> Result<()> {
manage(&["makemigrations".to_string()])
}
pub fn migrate(rest: &[String]) -> Result<()> {
let mut full_rest = vec!["migrate".to_string()];
full_rest.extend_from_slice(rest);
manage(&full_rest)
}
pub fn shell() -> Result<()> {
manage(&["shell".to_string()])
}
pub fn test() -> Result<()> {
manage(&["test".to_string()])
}

View File

@@ -1,15 +0,0 @@
use anyhow::Result;
use crate::command_builder::CommandBuilder;
pub fn stop_all() -> Result<()> {
let running_containers = CommandBuilder::docker().args("ps -q").exec_get_stdout()?;
if running_containers.is_empty() {
return Ok(());
}
CommandBuilder::docker()
.args(&format!("stop {running_containers}"))
.exec()
}

View File

@@ -1,43 +0,0 @@
use anyhow::Result;
use crate::command_builder::CommandBuilder;
// simple commands
pub fn build() -> Result<()> {
CommandBuilder::docker_compose().args("build").exec()
}
pub fn down() -> Result<()> {
CommandBuilder::docker_compose().args("down").exec()
}
/// Start containers via `docker compose start`. Optionally pass containers to be started.
/// ```
/// start(None);
/// ```
/// is roughly the same as
/// `docker compose --env-file ./.env -f docker/local/docker-compose.yaml up start`
pub fn start(containers: Option<&str>) -> Result<()> {
let args = format!("start {}", containers.unwrap_or(""));
CommandBuilder::docker_compose().args("start").exec()
}
pub fn stop() -> Result<()> {
CommandBuilder::docker_compose().args("stop").exec()
}
pub fn up() -> Result<()> {
CommandBuilder::docker_compose().args("up -d").exec()
}
// shortcuts
pub fn rebuild() -> Result<()> {
stop()?;
build()?;
up()
}
pub fn restart() -> Result<()> {
stop()?;
up()
}

View File

@@ -1,4 +0,0 @@
pub mod django;
pub mod docker;
pub mod docker_compose;
pub mod postgres;

View File

@@ -1,79 +0,0 @@
use anyhow::{anyhow, Context, Result};
use std::{
fs::File,
path::{Path, PathBuf},
process::Stdio,
};
use super::docker_compose;
use crate::command_builder::CommandBuilder;
fn get_containers() -> Result<[String; 2]> {
// get db container
// FIX: we assume we are running db in service named "postgresbd"
let db_container = CommandBuilder::docker_compose()
.args("ps -q postgresdb")
.exec_get_stdout()?
.trim()
.to_string();
let no_result = db_container.is_empty();
if no_result {
return Err(anyhow!("no container"));
}
// get all containers and filter out db container
let app_containers = CommandBuilder::docker_compose()
.args("ps -q")
.exec_get_stdout()?
.split_whitespace()
.filter(|x| x != &db_container)
.collect::<Vec<&str>>()
.join(" ");
Ok([db_container, app_containers])
}
pub fn import(file: &Path) -> Result<()> {
let [db_container, app_containers] = get_containers()?;
let dump_file = file.to_string_lossy();
println!("stopping all containers");
docker_compose::stop()?;
println!("starting db container");
docker_compose::start(Some(&db_container))?;
println!("restoring database");
let commands = [
format!("cp {dump_file} {db_container}:/tmp/dbdump"),
format!("exec {db_container} dropdb -U db db"),
format!("exec {db_container} createdb -U db -E utf8 -T template0 db"),
format!("exec {db_container} pg_restore -U db --dbname=db /tmp/dbdump"),
];
for command in commands {
CommandBuilder::docker().args(&command).exec()?;
}
println!("restarting containers");
docker_compose::stop()?;
docker_compose::up()?;
Ok(())
}
pub fn dump(file: &PathBuf) -> Result<()> {
let [db_container, _] = get_containers()?;
println!("dumping to local file {}", file.to_string_lossy());
let file = File::create(file)?;
let stdout = Stdio::from(file);
let command = format!("exec {db_container} pg_dump -U db --format=c db");
CommandBuilder::docker()
.args(&command)
.exec_redirect_stdout(stdout)?;
Ok(())
}

262
tests/link_store.rs Normal file
View File

@@ -0,0 +1,262 @@
// what the store is for: a path a sandbox should not read living somewhere it
// cannot. every case here is one that was once wrong.
mod support;
use std::fs;
use support::Case;
#[test]
fn a_path_moves_into_the_store_and_reads_back_through_the_link() {
let case = Case::new("roundtrip");
case.write(".env", b"SECRET=1\n");
case.write("secrets/token", b"tok\n");
case.gitignore(".env\nsecrets/\n");
case.ahab(&["link", "add", ".env", "secrets"]).ok();
assert!(case.is_symlink(".env"));
assert_eq!(fs::read(case.path(".env")).unwrap(), b"SECRET=1\n");
assert_eq!(fs::read(case.path("secrets/token")).unwrap(), b"tok\n");
case.ahab(&["link", "check", "--exit-code"]).ok();
case.ahab(&["link", "restore", "--all"]).ok();
assert!(!case.is_symlink(".env"));
assert_eq!(fs::read(case.path(".env")).unwrap(), b"SECRET=1\n");
}
#[test]
fn add_refuses_a_symlink_that_already_leads_out_of_the_repository() {
let case = Case::new("launder");
fs::write(case.outside.join("key"), b"KEY\n").unwrap();
case.link(&case.path("cache"), &case.outside);
// moving the link would move the pointer and leave the contents there,
// and check would then see a link into the store and call it clean
case.ahab(&["link", "add", "cache"])
.failed()
.says("outside the repository");
case.ahab(&["link", "check", "--exit-code"]).code_is(4);
}
#[test]
fn check_reports_a_store_entry_that_leads_back_out() {
let case = Case::new("poisoned");
fs::create_dir_all(&case.store).unwrap();
fs::write(case.outside.join("key"), b"KEY\n").unwrap();
// the state an older ahab left: the store holds a way back out
case.link(&case.store.join("cache"), &case.outside);
case.link(&case.path("cache"), &case.store.join("cache"));
case.ahab(&["link", "check", "--exit-code"]).code_is(4);
}
#[test]
fn check_reports_a_tracked_symlink_leading_out() {
let case = Case::new("tracked-link");
fs::create_dir_all(case.outside.join("aws")).unwrap();
case.link(&case.path("awsdir"), &case.outside.join("aws"));
case.git(&["add", "-f", "awsdir"]);
case.git(&["commit", "-qm", "commit a symlink out"]);
// git tracks symlinks, so this is in no untracked or ignored listing
case.ahab(&["link", "check", "--exit-code"]).code_is(4);
case.ahab(&["link", "check", "--porcelain"]).says("T>");
// and add cannot fix it, so saying so is all check can do
case.ahab(&["link", "add", "awsdir"])
.failed()
.says("tracked by git");
}
#[test]
fn a_stored_symlink_is_one_entry_rather_than_a_tree_to_walk() {
let case = Case::new("walk-out");
fs::create_dir_all(case.outside.join("private")).unwrap();
fs::write(case.outside.join("private/diary"), b"x\n").unwrap();
fs::write(case.outside.join(".netrc"), b"x\n").unwrap();
fs::create_dir_all(&case.store).unwrap();
case.link(&case.store.join("cache"), &case.outside);
let run = case.ahab(&["link", "list"]);
run.ok().says("cache");
// is_dir() would follow the link and enumerate what is behind it
run.silent_about(".netrc");
run.silent_about("diary");
}
#[test]
fn a_store_inside_the_repository_is_refused_however_it_is_spelled() {
let case = Case::new("store-inside");
case.write(".env", b"SECRET=1\n");
case.gitignore(".env\n");
// named directly
case.ahab(&["link", "add", "--store", "./within", ".env"])
.failed()
.says("inside the repository");
// and reached through a symlink, which a prefix test does not catch
case.mkdir("within");
let sneaky = case.repo.parent().unwrap().join("sneaky");
case.link(&sneaky, &case.path("within"));
let run = case.ahab(&[
"link".as_ref(),
"add".as_ref(),
"--store".as_ref(),
sneaky.as_os_str(),
".env".as_ref(),
]);
run.failed().says("inside the repository");
assert!(!case.is_symlink(".env"));
}
#[test]
fn a_relative_store_root_still_resolves_from_anywhere() {
let case = Case::new("relative-store");
case.write("sub/.env", b"SECRET=1\n");
case.gitignore("sub/.env\n");
case.ahab(&["link", "add", "--store", "../store", "sub/.env"])
.ok();
// the target is written into the symlink, so a relative one would resolve
// from the link's own directory rather than the working one
assert!(case.is_symlink("sub/.env"));
assert_eq!(fs::read(case.path("sub/.env")).unwrap(), b"SECRET=1\n");
}
#[test]
fn the_store_directories_are_the_owners_alone() {
let case = Case::new("modes");
case.write("deep/nested/.env", b"SECRET=1\n");
case.gitignore("deep\n");
case.ahab(&["link", "add", "deep/nested/.env"]).ok();
for dir in [
case.store.as_path(),
&case.store.join("deep"),
&case.store.join("deep/nested"),
] {
assert_eq!(case.mode(dir), 0o700, "{}", dir.display());
}
}
#[test]
fn a_failed_link_puts_the_payload_back() {
let case = Case::new("rollback");
case.write("x.env", b"SECRET=1\n");
// ahab's own temp name, but a real file: it must not be removed, and the
// payload must not be left in the store with nothing pointing at it
case.write("x.env.ahab-tmp", b"THE PROJECT OWNS THIS\n");
case.gitignore("x.env\n");
case.ahab(&["link", "add", "x.env"]).failed();
assert!(!case.is_symlink("x.env"));
assert_eq!(fs::read(case.path("x.env")).unwrap(), b"SECRET=1\n");
assert_eq!(
fs::read(case.path("x.env.ahab-tmp")).unwrap(),
b"THE PROJECT OWNS THIS\n"
);
assert!(!case.store.join("x.env").exists());
}
#[test]
fn a_directory_of_symlinks_out_keeps_its_destinations() {
let case = Case::new("collapse");
fs::write(case.outside.join("a"), b"A\n").unwrap();
case.link(&case.path("bundle/one"), &case.outside.join("a"));
// collapsing to `bundle/` would drop both the destination and the code
let run = case.ahab(&["link", "check", "--porcelain"]);
run.code_is(0).says("?>").says("bundle/one");
}
#[test]
fn a_filename_that_is_not_utf_8_is_reported_and_moved_as_itself() {
use std::ffi::OsStr;
use std::os::unix::ffi::OsStrExt;
let case = Case::new("latin1");
let name = OsStr::from_bytes(b"caf\xe9.env");
fs::write(case.repo.join(name), b"SECRET=1\n").unwrap();
// the whole listing used to fail on the one entry
let run = case.ahab(&["link", "check", "-z"]);
run.code_is(0);
assert!(
run.stdout.windows(4).any(|w| w == b"caf\xe9"),
"-z must write the bytes the name actually has"
);
case.gitignore("caf\u{e9}.env\n");
case.ahab(&["link".as_ref(), "add".as_ref(), name]).ok();
assert!(
fs::symlink_metadata(case.repo.join(name))
.unwrap()
.is_symlink()
);
}
#[test]
fn a_tracked_filename_that_is_not_utf_8_is_still_refused() {
use std::ffi::OsStr;
use std::os::unix::ffi::OsStrExt;
let case = Case::new("latin1-tracked");
let name = OsStr::from_bytes(b"caf\xe9.env");
fs::write(case.repo.join(name), b"TRACKED\n").unwrap();
case.git(&["add".as_ref(), "-f".as_ref(), name]);
case.git(&["commit", "-qm", "track a latin-1 name"]);
// asked about lossily, git answers about a path nothing has and says
// "not tracked", and a tracked file leaves the working tree
case.ahab(&["link".as_ref(), "add".as_ref(), name])
.failed()
.says("tracked by git");
assert!(
!fs::symlink_metadata(case.repo.join(name))
.unwrap()
.is_symlink()
);
}
#[test]
fn a_pathspec_is_asked_about_as_a_name_not_a_pattern() {
let case = Case::new("pathspec");
// git reads pathspec magic after `--` too, so this used to have git list
// every tracked file *except* the named one, which read as "it is tracked"
case.write(":!untracked.env", b"SECRET=1\n");
case.gitignore(":!untracked.env\n");
case.ahab(&["link", "add", ":!untracked.env"]).ok();
assert!(case.is_symlink(":!untracked.env"));
}
#[test]
fn distinct_remotes_do_not_share_one_store_directory() {
let case = Case::new("remotes");
case.git(&[
"remote",
"set-url",
"origin",
"https://git.example.org/a/my_api",
]);
let plain = case.ahab(&["link", "list"]).out();
case.git(&[
"remote",
"set-url",
"origin",
"https://git.example.org/a/my~api",
]);
let awkward = case.ahab(&["link", "list"]).out();
assert_ne!(plain, awkward);
// the ordinary remote keeps the path it already had
assert!(plain.contains("/a/my_api"), "{plain}");
}

99
tests/output_io.rs Normal file
View File

@@ -0,0 +1,99 @@
// what a command reports and how it exits, including when the write fails.
mod support;
use std::fs;
use support::Case;
#[test]
fn findings_are_reported_through_the_exit_code() {
let case = Case::new("findings");
case.write("loose.txt", b"x\n");
case.ahab(&["link", "check"]).ok();
case.ahab(&["link", "check", "--exit-code"]).code_is(4);
}
#[test]
fn an_argument_error_exits_two() {
let case = Case::new("usage");
case.ahab(&["link", "restore"]).code_is(2);
case.ahab(&["link", "restore", "--all", "some/path"])
.code_is(2);
case.ahab(&["postgres", "dump", "-F", "directory", "-z", "d"])
.code_is(2);
case.ahab(&["nonsense"]).code_is(2);
}
#[test]
fn a_write_that_cannot_be_delivered_is_a_failure_rather_than_a_success() {
let case = Case::new("devfull");
for n in 0..200 {
case.write(&format!("file-{n}.txt"), b"x\n");
}
// /dev/full accepts the open and fails the write. exiting 0 there would
// report a listing that was never delivered
let Ok(full) = fs::File::create("/dev/full") else {
eprintln!("skipped: this system has no /dev/full to fail a write against");
return;
};
let out = std::process::Command::new(env!("CARGO_BIN_EXE_ahab"))
.current_dir(&case.repo)
.env("XDG_DATA_HOME", case.repo.join("../xdg"))
.args(["link", "check", "--exit-code", "--porcelain"])
.stdout(full)
.output()
.expect("running ahab");
assert_eq!(out.status.code(), Some(1), "{:?}", out.status);
assert!(
String::from_utf8_lossy(&out.stderr).contains("writing to stdout failed"),
"{}",
String::from_utf8_lossy(&out.stderr)
);
}
#[test]
fn every_named_path_is_moved_even_when_the_reader_leaves() {
let case = Case::new("partial");
for name in ["a.env", "b.env", "c.env"] {
case.write(name, b"x\n");
}
case.gitignore("a.env\nb.env\nc.env\n");
// a closed stdout must stop the writing, not the moving: the command had
// planned to move all three and reported success for doing so
let run = case.ahab(&["link", "add", "a.env", "b.env", "c.env"]);
run.ok();
for name in ["a.env", "b.env", "c.env"] {
assert!(case.is_symlink(name), "{name} was left behind");
}
}
#[test]
fn quiet_keeps_the_reason_a_path_failed() {
let case = Case::new("quiet");
case.write("ok.env", b"x\n");
case.gitignore("ok.env\n");
let run = case.ahab(&["-q", "link", "add", "ok.env", "missing.env"]);
run.failed()
.says("1 of 2 paths failed")
// the line saying *which* and *why* is a result, not progress
.says("missing.env");
}
#[test]
fn a_dry_run_changes_nothing_on_disk() {
let case = Case::new("dryrun");
case.write(".env", b"SECRET=1\n");
case.gitignore(".env\n");
case.ahab(&["--dry-run", "link", "add", ".env"]).ok();
assert!(!case.is_symlink(".env"));
assert!(!case.store.join(".env").exists());
}

207
tests/support/mod.rs Normal file
View File

@@ -0,0 +1,207 @@
// enough scaffolding to run ahab against a throwaway repository, hand-rolled
// rather than pulled in: the crate has no dependencies it does not need, and a
// dev-dependency is still something to trust and keep current.
// each integration test file compiles this module separately, so a helper only
// one of them needs looks unused to the others
#![allow(dead_code)]
use std::ffi::OsStr;
use std::fmt::Write as _;
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
use std::sync::atomic::{AtomicU32, Ordering};
use std::{env, fs};
// unique per case even with the suite running in parallel
static NEXT: AtomicU32 = AtomicU32::new(0);
pub struct Case {
root: PathBuf,
pub repo: PathBuf,
pub store: PathBuf,
pub outside: PathBuf,
}
impl Case {
// a repository with an origin remote, a store of its own, and one commit
pub fn new(label: &str) -> Self {
let root = env::temp_dir().join(format!(
"ahab-{label}-{}-{}",
std::process::id(),
NEXT.fetch_add(1, Ordering::Relaxed)
));
let _ = fs::remove_dir_all(&root);
let case = Self {
repo: root.join("repo"),
store: root
.join("xdg/ahab/git.example.org/acme/proj")
.to_path_buf(),
outside: root.join("outside"),
root,
};
for dir in [&case.repo, &case.outside] {
fs::create_dir_all(dir).expect("creating the case directories");
}
case.git(&["init", "-q", "."]);
case.git(&["config", "user.email", "test@example.org"]);
case.git(&["config", "user.name", "test"]);
case.git(&["config", "commit.gpgsign", "false"]);
case.git(&[
"remote",
"add",
"origin",
"https://git.example.org/acme/proj.git",
]);
case.write("tracked.txt", b"tracked\n");
case.git(&["add", "tracked.txt"]);
case.git(&["commit", "-qm", "init"]);
case
}
pub fn git<S: AsRef<OsStr>>(&self, args: &[S]) -> Output {
let out = Command::new("git")
.current_dir(&self.repo)
.args(args)
.output()
.expect("running git");
assert!(
out.status.success(),
"git {:?} failed: {}",
args.iter()
.map(|a| a.as_ref().to_string_lossy())
.collect::<Vec<_>>(),
String::from_utf8_lossy(&out.stderr)
);
out
}
// ahab, in the repository, with a store nothing else shares
pub fn ahab<S: AsRef<OsStr>>(&self, args: &[S]) -> Run {
let out = Command::new(env!("CARGO_BIN_EXE_ahab"))
.current_dir(&self.repo)
.env("XDG_DATA_HOME", self.root.join("xdg"))
// the git commands ahab runs must not read the developer's own config
.env("GIT_CONFIG_GLOBAL", "/dev/null")
.env("GIT_CONFIG_SYSTEM", "/dev/null")
.args(args)
.output()
.expect("running ahab");
Run {
code: out.status.code(),
stdout: out.stdout,
stderr: String::from_utf8_lossy(&out.stderr).into_owned(),
}
}
pub fn path(&self, rel: &str) -> PathBuf {
self.repo.join(rel)
}
pub fn write(&self, rel: &str, contents: &[u8]) {
let path = self.path(rel);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent).expect("creating a parent directory");
}
fs::write(path, contents).expect("writing a file");
}
pub fn mkdir(&self, rel: &str) {
fs::create_dir_all(self.path(rel)).expect("creating a directory");
}
pub fn link(&self, at: &Path, to: &Path) {
if let Some(parent) = at.parent() {
fs::create_dir_all(parent).expect("creating a parent directory");
}
std::os::unix::fs::symlink(to, at).expect("creating a symlink");
}
pub fn gitignore(&self, lines: &str) {
self.write(".gitignore", lines.as_bytes());
self.git(&["add", ".gitignore"]);
self.git(&["commit", "-qm", "ignore"]);
}
pub fn is_symlink(&self, rel: &str) -> bool {
fs::symlink_metadata(self.path(rel)).is_ok_and(|meta| meta.is_symlink())
}
pub fn mode(&self, path: &Path) -> u32 {
use std::os::unix::fs::PermissionsExt;
fs::symlink_metadata(path)
.expect("reading a mode")
.permissions()
.mode()
& 0o777
}
}
impl Drop for Case {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.root);
}
}
pub struct Run {
pub code: Option<i32>,
pub stdout: Vec<u8>,
pub stderr: String,
}
impl Run {
pub fn out(&self) -> String {
String::from_utf8_lossy(&self.stdout).into_owned()
}
pub fn ok(&self) -> &Self {
assert_eq!(self.code, Some(0), "{}", self.report());
self
}
pub fn failed(&self) -> &Self {
assert_ne!(self.code, Some(0), "{}", self.report());
self
}
pub fn code_is(&self, want: i32) -> &Self {
assert_eq!(self.code, Some(want), "{}", self.report());
self
}
pub fn says(&self, needle: &str) -> &Self {
assert!(
self.out().contains(needle) || self.stderr.contains(needle),
"expected {needle:?}\n{}",
self.report()
);
self
}
pub fn silent_about(&self, needle: &str) -> &Self {
assert!(
!self.out().contains(needle) && !self.stderr.contains(needle),
"did not expect {needle:?}\n{}",
self.report()
);
self
}
fn report(&self) -> String {
let mut out = String::new();
let _ = write!(
out,
"exit: {:?}\n--- stdout\n{}--- stderr\n{}",
self.code,
self.out(),
self.stderr
);
out
}
}