The Release Process¶

How a version of EduMatcher is cut and published: the scripts and workflows involved, the developer checklist, and the pitfalls that catch newcomers. The day-to-day development loop that precedes a release is in Development Practice.

How a release is produced¶

Two scripts and two GitHub workflows. One tag produces the Python package and the container images together, all carrying the same version, which is what lets the installer pin a whole system with a single number.

flowchart TD
    MK["scripts/mkrelease.sh\nbump version, build docs, tag"]
    TAG["git tag vX.Y.Z\npushed to GitHub"]
    GH["scripts/mkghrelease.sh\ngh release create + artifacts"]
    REL["GitHub release published"]

    PYPI["publish-to-pypi.yml\npoetry build and publish"]
    IMG["publish-images.yml"]

    AMD["build amd64\nubuntu-24.04"]
    ARM["build arm64\nubuntu-24.04-arm"]
    MERGE["merge digests into\none manifest list\ntags X.Y.Z and latest"]

    OUTP["PyPI\nedumatcher X.Y.Z"]
    OUTC["GHCR\n7 multi-arch images X.Y.Z"]

    USER1["pipx install edumatcher"]
    USER2["curl .../install.sh | bash"]

    MK --> TAG --> GH --> REL
    REL --> PYPI --> OUTP --> USER1
    REL --> IMG
    IMG --> AMD --> MERGE
    IMG --> ARM --> MERGE
    MERGE --> OUTC --> USER2
    GH -.->|"phase 6B waits for the run"| MERGE

Each image is built natively on both architectures rather than emulated, then the two are joined into one manifest list. A user on Intel and a user on Apple Silicon pull the same tag and each gets the right binary.

The seven published images are:

ghcr.io/johan162/edumatcher                 the exchange, all pm-* processes
ghcr.io/johan162/edumatcher-ai-sim          the AI-trader simulation runner
ghcr.io/johan162/edumatcher-terminal-gui    TapeDeck (pm-terminal)
ghcr.io/johan162/edumatcher-log-gui         the Log Operator Console
ghcr.io/johan162/edumatcher-config-gui      the Configuration GUI
ghcr.io/johan162/edumatcher-trader-gui      the Trading GUI
ghcr.io/johan162/edumatcher-book-gui        the order-book viewer

latest is only moved for an exact vMAJOR.MINOR.PATCH tag, so a pre-release never becomes what a new user gets by default — the same rule the PyPI workflow uses to choose between PyPI and TestPyPI.

Publishing by hand¶

make ghcr-push in deployment/docker/ builds all six images from your checkout and pushes them, for when the workflow cannot run:

export GITHUB_USER=<you> GHCR_TOKEN=<token with write:packages>
make ghcr-push                              # all six, tagged :dev
make ghcr-push TAG=0.20.6 FORCE=1 LATEST=1  # as a release tag

It builds only for the architecture you are on. Pushing a single-architecture image over a release tag replaces the manifest list, and users on the other architecture then get "no matching manifest" — which you will not notice, because your own machine keeps working. That is why a release-looking tag needs FORCE=1 and why latest is never moved unless asked.

Developer release checklist¶

For the maintainer cutting a release. Steps 1-2 are local, 3-5 are automated but need watching, and 6-7 are the checks that the release actually works for somebody who is not you.

Before tagging¶

Working tree is clean and tests pass.
```
./scripts/mkbld.sh
```
CHANGELOG.md has an entry for this version.
Add a new CHANGELOGENTRY.md. Using the custom copilot skill /changelog-entry to create a draft version based on the git-logs
or if there is no copilot available Use the script scripts/mkchlogentry.sh drafts one from the commit logs.

Tag and release¶

Run scripts/mkrelease.sh. It bumps the version, builds the documentation bundles, commits and tags.
Wait for the CI workflows on the tag to go green before creating the release.
Run scripts/mkghrelease.sh. It validates the artifacts in dist/, creates the GitHub release, and then waits for the container image workflow.

Option	Effect
`--dry-run`	Show what would happen; create nothing
`--pre-release`	Force pre-release marking regardless of the tag
`--skip-images`	Do not wait for the image workflow
`IMAGE_WAIT_MINUTES=n`	How long to wait (default 30)

```bash
git switch main && git pull --ff-only
./scripts/mkghrelease.sh
```

If the image workflow fails, the GitHub release still exists — only the images are missing. Re-run just that part:

gh run view <run-id> --log-failed
gh workflow run publish-images.yml -f tag=vX.Y.Z

After the release¶

Verify the one-line install as a stranger would. First stop any stack you already have running — the released deployment and the source-built one use the same container names and host ports, so an install started beside a running stack silently attaches to it and verifies nothing:

make -C deployment/docker down-all

Then install into a throwaway directory so your own instance is untouched:

curl -fsSL https://raw.githubusercontent.com/johan162/EduMatcher/vX.Y.Z/deployment/curl/install.sh \
    | bash -s -- --dir /tmp/em-release-test

Then open http://localhost:8090, and clean up with cd /tmp/em-release-test && ./edumatcher.sh uninstall --data.

Verify the PyPI install in a fresh environment:
```
pipx install edumatcher==X.Y.Z
```

Where releases usually go wrong

Two failures are quiet rather than loud. An image built from PyPI instead of the checkout looks like a successful build but ships the previous release — step 5's image-workflow log should show Installing local wheel (the line the Dockerfile prints when it finds a wheel staged by the build, rather than installing from PyPI). And private GHCR packages fail only for other people, never for the maintainer who is already authenticated — step 6, done as a stranger with no credentials, is what catches that.

Summary: To make a release, step-by-step¶

# 1. Bump version in pyproject.toml
poetry version 0.3.2

# 2. Create the changelog template with help of copilot using the custom skill 
/changelog-entry 0.3.2 patch

# 3. Review and edit CHANGELOG.md as needed. The commit the new changelog
git add CHANGELOG.md
git commit -m "chore(changelog): v0.3.2"

# 4. Build and validate release artifacts.
# Use --intro for real releases because mkghrelease.sh expects the intro bundle.
./scripts/mkbld.sh --intro

# 5. Commit release-ready changes on develop, then preview release actions
GITHUB_USER=<your-gh-user> ./scripts/mkrelease.sh patch --dry-run

# 6. Execute the local release flow (squash merge develop -> main, tag, push, sync back)
GITHUB_USER=<your-gh-user> ./scripts/mkrelease.sh patch

# 7. After CI is green on main, create the GitHub release from main
git switch main && git pull --ff-only
./scripts/mkghrelease.sh

Appendix: The release scripts¶

What `mkbld.sh` does¶

mkbld.sh is the build gate before release scripts. It currently:

validates environment and required Poetry tools
runs black, flake8, mypy (and pyright if available)
runs all npm tests for the web-apps/
runs pytyest tests with coverage threshold 80%
updates coverage badge when coverage.xml changed
builds and verifies Python packages
builds all documentations, user-guide PDF+EPUB, doc-site, training bundles, etc
optionally builds Exchange Intro bundle when --intro is provided

For release publishing, prefer running ./scripts/mkbld.sh --intro.

What `mkrelease.sh` expects¶

The script is designed around this model:

GITHUB_USER environment variable is set
you are on a clean develop branch
local develop is synced with remote
the requested version is not already tagged
CHANGELOG.md already contains an entry for that version
release artifacts already exist in dist/ and pass twine check

Important usage detail: mkrelease.sh argument is only the release type (major, minor, or patch). The version is read from pyproject.toml.

During execution, mkrelease.sh will:

validate repo state and changelog/version/tag preconditions
squash-merge develop into main and create v<version> tag
push main and the tag
merge main back into develop and push develop
wait for GitHub Actions completion with gh run watch --exit-status

What `mkghrelease.sh` expects¶

The script is designed around this model:

GITHUB_USER environment variable is set
authenticated gh CLI on PATH
you are on a clean main branch synced with remote
latest release tag already exists on main
required artifacts exist:
wheel in dist/
sdist in dist/
user-guide bundle in docs/dist/
exchange-intro bundle in docs-exchange-intro/dist/

It auto-detects pre-releases from tags ending in rcN (or you can force with --pre-release) and creates the GitHub release using notes extracted from CHANGELOG.md.

Common pitfalls for new developers¶

Forgetting which data directory is live¶

In a source checkout, src/data/ — not ~/.local/share/edumatcher/ — is the data directory every local process and every test reads and writes. A "compiled config has been modified since it was compiled" error from pytest, or a pm-config-deploy you ran that doesn't seem to take effect, is almost always this. See Where your data directory actually is above.

Starting clients before the engine¶

Most processes assume the engine sockets already exist. Start pm-engine first.

Using a gateway ID that is not in `engine_config.yaml`¶

pm-alf-console --id SOMEONE only works if that gateway is configured.

Forgetting that observer processes depend on each other¶

pm-ticker and pm-board rely on statistics written by pm-stats.

Assuming docs-only changes need no validation¶

They still need:

poetry run mkdocs build

Treating helper scripts as canonical truth¶

Some scripts are polished automation; others still show drift from earlier repo history. Read before relying on them.

Ignoring end-to-end behavior¶

A change that passes unit tests can still break:

startup order
message topics
gateway acknowledgements
persistence restore
UI observers

That is why a minimal live run is worth doing.

Suggested first-week path for a new developer¶

If you are onboarding, this is a good sequence:

Set up Poetry and build the docs
Run the minimal live system once
Submit a few manual orders through pm-alf-console
Run the normal test suite
Read the deterministic verification page
Pick one small bug fix or documentation improvement
Run the full quality gate before opening a PR

That path teaches both the theory and the operational shape of the system before you attempt deeper engine work.