Container image: pulling, running, verifying¶
- Issue: #40; threat T24 in docs/security.md.
- Image:
ghcr.io/horae-dev/horae, forlinux/amd64andlinux/arm64.
Tags¶
| Tag | Points at | Moves |
|---|---|---|
sha-<short> (7 hex digits) |
the image built from that commit of main |
never |
edge |
the image of the newest commit of main |
on every push to main (never backwards: a re-run of an older commit does not move it) |
X.Y.Z |
the main image of the commit tagged vX.Y.Z |
never |
X.Y |
the newest X.Y.Z release |
on each patch release |
X |
the newest X.* release, from 1.0 only |
on each release; there is no floating 0 |
There is no latest tag in 0.x. Pin by digest in production
(ghcr.io/horae-dev/horae@sha256:...): tags can move, digests cannot.
How they are made:
- Push to
main: thepublishjob of.github/workflows/ci.yml, after CI OK passed for that commit. Each platform is built and smoke-tested (scripts/smoke-image.sh; arm64 under QEMU), the multi-platform image is pushed by digest, untagged, its layers are checked to be the tested ones, its SBOM is checked, it is signed and the signature verified, and only then areedgeandsha-<short>added. A tag never points at an untested or unsigned digest. - Tag
vX.Y.Z:.github/workflows/image.yml, called byrelease.yml(see releasing), builds nothing. The tagged commit must be onmain; the workflow takes thesha-<short>digest of that commit, checks its revision label and its signature, smoke-tests it (itshorae --versionmust reportX.Y.Z), then adds the release tags to that same digest. Ifmainnever published that commit (for instance its CI failed or was cancelled), the release fails: re-run that commit's CI onmain, then the tag workflow. Only plain semver tags (vX.Y.Z) are releases; anything else fails.
horae --version reports __version__ from src/horae/__init__.py at the image's commit: bump
it in the commit you tag (the release check refuses a mismatch). The image label
org.opencontainers.image.version holds the same value and org.opencontainers.image.revision
the commit.
Verify before you run¶
Every published digest is signed with cosign keyless signing
(GitHub OIDC, no long-lived key) by the publish job of ci.yml on main, and carries an SPDX
SBOM and build provenance for each platform. Release tags reuse that digest, so the identity is
the same for edge, sha-* and releases:
cosign verify \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity https://github.com/horae-dev/horae/.github/workflows/ci.yml@refs/heads/main \
--certificate-github-workflow-repository horae-dev/horae \
--certificate-github-workflow-trigger push \
ghcr.io/horae-dev/horae@sha256:<digest>
Add --certificate-github-workflow-sha <commit> to also bind it to the commit you expect. The
signature is on the multi-platform index digest, which is what every tag points at.
Read the SBOM (one per platform):
docker buildx imagetools inspect ghcr.io/horae-dev/horae:edge --format '{{ json .SBOM }}'
Running¶
The image runs as the non-root user 10001:10001, under tini
as PID 1 (ENTRYPOINT ["tini", "--", "horae"]), and works with a read-only root filesystem. It
declares no VOLUME: mount one on /instance yourself (-v), or your instance lives in the
container and is lost with it. The only writable places are:
/instance, the instance directory:horae.yaml,horae.lock,pipelines/,plugins/,state/andvault/.HORAE_INSTANCE=/instanceand it is the working directory. A new named volume starts owned by 10001 (mode 0700); a bind mount must be owned by uid 10001./tmp, which must be a tmpfs (HOMEandTMPDIRpoint there).
Create the instance once, then run the example pipeline:
docker volume create horae
opts=(--rm --read-only --tmpfs /tmp --cap-drop ALL --security-opt no-new-privileges
-v horae:/instance)
docker run "${opts[@]}" ghcr.io/horae-dev/horae:edge init /instance
docker run "${opts[@]}" ghcr.io/horae-dev/horae:edge doctor
docker run "${opts[@]}" --network none ghcr.io/horae-dev/horae:edge run daily-fixture
The note lands in the volume's vault/Daily/. Compose equivalent: read_only: true,
tmpfs: [/tmp], cap_drop: [ALL], security_opt: [no-new-privileges:true], a volume on
/instance.
The image has no pip, no ensurepip and no compiler. It does carry uv and git, for
horae plugin add only (see below).
The image ships no Docker CLI and mounts no Docker socket, so plugins run as processes in
it: runtime: container (#49) reports runtime_unavailable. Giving a containerised Horae
access to an engine (a socket mount is host root) is a deliberate choice:
plugin-containers.md.
Installing plugins in the container¶
#169. Plugins run without uv: each installed
plugin has its own venv, built at install time from its uv.lock and linked to the Python that
built it (schemas decision 69), and the host sets UV_OFFLINE=1, UV_PYTHON_DOWNLOADS=never and
PIP_NO_INDEX=1 in every plugin child (/usr/local/bin, where uv lives, is on a plugin's PATH, but a uv it
ran would be offline and could not download an interpreter). Installing one is the only thing
that uses uv, so the image ships:
- uv, copied in the final stage from the same pinned official image the build stage uses
(
UV_IMAGEin theDockerfile:ghcr.io/astral-sh/uv:<version>@sha256:<digest>, a static binary at/usr/local/bin/uv). Renovate bumps the tag and digest together, grouped with CI's uv. It is not used byhorae serve,runor any pipeline. - git, which
horae plugin addclones marketplace indexes and plugin repositories with (https only, no hook). It brings Perl with it, which is most of the image's size increase.
The venv is built by the image's own interpreter (/usr/local/bin/python3.12), so it works in
that container with no network. Everything it writes is under /instance, the one writable
volume: the plugin tree and its .venv in plugins/<name>, uv's cache in state/uv-cache
(owned by 10001 like the rest; nothing else is writable, and /tmp may stay noexec).
Installing needs network access (git over https to the marketplace and the plugin's
repository, and the package index named in the plugin's uv.lock); running does not, and
--network none still works for everything but plugin add.
docker compose exec horae horae plugin add obsidian # compose; Coolify: the Terminal, or docker exec
A plugin installed this way is not enabled until horae.lock lists it with the permissions
you approved (#46, PR #168). Until #168 is
merged plugin add only installs and prints the pending notice; once it is, plugin add
shows the plugin's permissions and asks for your approval in the terminal, so use an interactive
exec (docker compose exec, or docker exec -it). For a reproducible, non-interactive
install, commit the horae.lock of a reviewed install into your instance repository and install
with --yes, which approves only when horae.lock already lists that exact version, commit and
permissions, and is refused otherwise (this part depends on #168 too; the flag does not exist
before it). Without a terminal and without --yes the install is refused. A plugin is installed into the volume, so it survives restarts and
image upgrades: after an upgrade, the plugin's venv still points at the same interpreter path
(/usr/local/bin/python3.12), which is stable within a Python minor version; a release that
changes the image's Python minor version says so in its notes, and the plugin must then be
reinstalled (horae plugin upgrade, #47).
The image's size grew by about 53 MB for uv and about 120 MB for git and Perl, so the runtime image went from 341 MB to 514 MB (arm64, uncompressed).
horae serve, the HTTP API and the health check¶
The default command is horae serve (#37,
decision 51): the built-in scheduler and the HTTP API
(GET /healthz, GET /runs, GET /runs/<id>), read-only, on port 8080. The image sets
HORAE_HOST=0.0.0.0 so that the reverse proxy can reach it on the container network. The API
has no authentication of its own: put it behind the proxy's SSO middleware (only /act/ is
exempt, #41) and never publish the port (-p 8080:8080 on a public interface exposes the run
list to anyone). Set HORAE_HOST=127.0.0.1 when nothing else needs to reach it.
Set HORAE_ALLOWED_HOSTS to the public hostname the proxy serves Horae on (space-separated if
several, e.g. HORAE_ALLOWED_HOSTS=horae.example.org): the API refuses (400 invalid_host)
any request whose Host is not loopback or one of these, its defence against DNS rebinding.
Without it horae serve warns at start, and each refused Host is logged once. A neighbour
container calling http://horae:8080/ needs horae in the list too (_ is allowed, as in
Docker service names). The Host check is a defence against browsers, not neighbours: any
container on Horae's network can reach port 8080 and send any Host, so put Horae on a network
shared only with the proxy.
docker run -d --name horae "${opts[@]}" --stop-timeout 20 ghcr.io/horae-dev/horae:edge
docker inspect --format '{{.State.Health.Status}}' horae # healthy once /healthz answers 200
Its HEALTHCHECK asks /healthz over loopback with the standard library (python -c and
urllib, no proxy; there is no curl or wget in the image): 200 when the store is readable and
the scheduler runs (or HORAE_SCHEDULER=external), 503 otherwise. A container running
another command (init, doctor, run) reports unhealthy, which is harmless for a one-off. If
you set HORAE_PORT, the check follows it; a serve --port N command does not (the check keeps
asking 8080), so change the port with HORAE_PORT. If you set HORAE_HOST to :: (IPv6 only)
or to an address other than 0.0.0.0 or loopback, override the check. A healthy container does
not prove the proxy can reach Horae: the check uses a loopback Host, so a wrong
HORAE_ALLOWED_HOSTS only shows in the logs. Give the container at least 20 s to stop
(--stop-timeout 20, compose stop_grace_period: 20s).
Stable paths, and upgrading¶
/opt/venv (the virtualenv, whose bin/python runs Horae) and /instance are a stable
contract of the image: they will not change between releases without a migration note.
horae init writes absolute paths into the instance: horae.lock holds the vault's path
(/instance/vault), and plugins/<name>/plugin.yaml runs each built-in plugin as
/opt/venv/bin/python -m horae.plugins.builtin.<module>.
On an image upgrade, nothing in the instance is rewritten. The built-in plugins' code is the
new image's (their command runs the image's interpreter), but their installed manifests in
plugins/ and their pins in horae.lock (version 0.1.0, a placeholder commit, the approved
permissions) stay as init wrote them. Horae checks that each installed manifest matches its pin,
so the instance keeps working; it does not notice that the image ships a newer built-in plugin
version, and v1 has no command to reinstall them (plugin install and verify is #44, #46).
Manual step when a release notes a built-in plugin version or permission change: run
horae init into a new empty volume, review the new plugins/ and horae.lock entries, and copy
them into your instance (or move your pipelines/, vault/ and state/ to the new one).
Bumping the deploy templates¶
The deploy templates pin a sha-<short> tag by default, written once per file: the
x-horae-image line of deploy/coolify/docker-compose.yaml
and of deploy/compose/docker-compose.yaml
(.env.example leaves HORAE_IMAGE_TAG empty, so it uses that default). Renovate does not
bump it: a sha-* tag is not a version it can order. Release step, after main published
the commit (its sha-<short> exists and is signed) and after the 7-day wait the repository
applies to its own dependencies (minimumReleaseAge in renovate.json) when the change is not
urgent:
- Verify the signature of the new tag's digest (above).
- Replace the tag in both
x-horae-imagelines (the same value; once releases exist, the release tagX.Y.Z), and in the twoimage:lines ofdeploy/coolify/upstream/horae.yaml(the template proposed to Coolify; a test checks it names the same image). If it was already merged upstream, a new upstream pull request carries the bump. uv run pytest tests/test_deploy_templates.py(it checks both files carry one, identical, default) and commit.
A deployed Coolify resource keeps the tag it stored at its first deployment; operators update
by changing HORAE_IMAGE_TAG (coolify.md). In production, operators pin
the digest as well: HORAE_IMAGE_TAG=<tag>@sha256:<digest>.
Testing locally¶
scripts/smoke-image.sh IMAGE [PLATFORM] is the CI smoke test: non-root, PID 1 is tini,
--version format (and EXPECTED_VERSION if set), a read-only root filesystem probed as root
against a writable control, no pip/ensurepip/compiler, uv --version (the Dockerfile's pin), a plugin venv built inside the container offline (scripts/smoke-plugin-env.py: a fixture plugin locked against a local file index, installed by PluginEnvironment on the volume, linked to the image's Python, runnable renamed), then init, doctor and
run daily-fixture on named volumes with no network, and finally the default horae serve:
healthy by the image's own HEALTHCHECK, /runs answering, exit 0 on docker stop. Run it after docker build -t horae:test .;
pull requests run it on native amd64.