pristontale-eu-linux/RELEASE_PIPELINE_WORKPLAN.md

# PristonTale EU Linux Launcher Release Pipeline Workplan

This is the tracked implementation plan for manual Gitea releases today and the
deferred automatic publication path, stable and prerelease channels, signed
manifests, release history, and channel-aware rollback. A checked item means the
repository contains the implementation or the audit evidence named beside it.
Publishing tests that require a configured Gitea server remain unchecked until
they have run against that server.

Legend: `[x]` complete, `[ ]` pending.

## Current milestone

- [x] Save the release workplan in the repository.
- [x] Inspect the existing release, update, rollback, settings, UI, and telemetry paths.
- [x] Implement and validate the automatic publication pipeline for future use.
- [x] Switch to temporary manual-publication mode: `release.sh` now creates a
  signed, verified `dist/v<VERSION>/` package without commits, tags, pushes, API
  calls, or uploads.
- [x] Disable automatic workflow push triggers so manual Gitea releases cannot
  race an Actions publication job.
- [x] Store normal update payloads directly at the tar root while retaining
  runtime extraction support for historical archives and the `0.9.10`–`0.9.11`
  compatibility window.
- [ ] Re-enable automatic Gitea publication when requested.
- [x] P1: add stable/prerelease discovery and launcher channel support.
- [x] P2: harden remote rollback and managed-file reconciliation.
- [ ] P3: complete signing migration and release quality improvements.

## Phase 0 — Existing-system audit

- [x] Repository structure inspected. Runtime sources are in `ptlinux/` and
  `ptdiag/`; installer assets are in `packaging/`; the checked-in channel is in
  `updates/`; local, ignored development tools and tests are in `tools/` and
  `tests/`.
- [x] Authoritative launcher version identified as `ptlinux/__init__.py`.
  `pyproject.toml` is a required synchronized metadata mirror.
- [x] Current update archive identified as
  `pristontale-eu-linux-update-v<VERSION>.tar.gz`.
- [x] Current archive layout identified. Its root is
  `pristontale-eu-linux-update-v<VERSION>/`, containing `ptlinux/`, a compact
  `ptdiag/` subset, and `release.json`.
- [x] Current update manifest identified as signed JSON schema 1 in
  `updates/stable.json`, with version, channel, minimum updater version, archive
  URL/size/SHA-256, publication time, and notes URL.
- [x] Current update URL identified as the raw `main` branch
  `updates/stable.json`; `PTLINUX_UPDATE_MANIFEST_URL` can override it.
- [x] Latest-release discovery identified as the mutable signed stable manifest.
  The launcher does not use Gitea `/releases/latest`.
- [x] Current rollback identified as local retained release directories selected
  by atomic `current` and `previous` symlinks.
- [x] Cached archives identified in the launcher download cache and reused only
  when their size and SHA-256 match the authenticated manifest.
- [x] Current staging identified: download to a `.part` file, verify while
  downloading, safely extract to a staging directory, validate/self-import,
  move into the immutable release store, then atomically replace symlinks.
- [x] Atomic activation confirmed through same-directory temporary symlink plus
  `os.replace()` and directory `fsync`.
- [x] Removed launcher files are handled because each release is a complete,
  separate directory. No in-place overlay is used.
- [x] Fixed release asset names are not required by the current launcher; the
  authenticated manifest supplies the archive URL.
- [x] Existing manually generated files identified: installer tar/zip archives,
  launcher update tar archive, SHA-256 files, stable manifest/signature, and
  release uploads.
- [x] Existing manual exclusions identified. The launcher update builder selects
  Python/runtime files explicitly and excludes installer-only modules; the AIO
  builder selects runtime, packaging, installer, and README files explicitly.
- [x] SHA-256 generation and verification confirmed for archives.
- [x] Existing asymmetric signature support confirmed. The manifest is signed
  and verified with the bundled public key through OpenSSL (currently RSA with
  SHA-256); the private key is ignored under `release-keys/`.
- [x] Existing telemetry inspected and extended with channel, target release,
  manifest identity, and complete update/rollback lifecycle metadata.
- [x] Decision: extend the existing signed-manifest and atomic release-directory
  updater instead of introducing a second updater.

## Release model and invariants (phases 1–2, 19, 24–25, 32–38, 45–51, 61–63, 77, 84)

- [x] Support exactly `stable` and `prerelease` channels in the initial model.
- [x] Accept stable tags `vMAJOR.MINOR.PATCH` and prerelease tags ending in
  `-beta.N` or `-rc.N`; reject channel/tag mismatches.
- [x] Treat the manually selected Gitea release tag as the authoritative,
  immutable release identity.
- [ ] Restore annotated-tag and channel-pointer monotonicity enforcement when
  automatic publication is re-enabled. Manual publishers must currently check
  the tag in Gitea and the version ordering before copying a channel pointer.
- [x] Use one semantic-version parser and one channel/update resolver; never use
  lexical version ordering.
- [x] Preserve ordering from beta to RC to final, including multi-digit fields.
- [x] Keep the internal launcher version without the leading `v`.
- [x] Build immutable, versioned assets named
  `PristonTale-EU-<VERSION>.tar.gz`; normal releases place `ptlinux/`, `ptdiag/`,
  and `release.json` directly at the archive root.
- [x] Accept both new flat archives and legacy version-folder archives in the
  updated launcher so retained releases and rollback remain compatible.
- [x] Configure stable `0.9.10` and its `0.9.11` UI repair as legacy-wrapped
  bridge releases so deployed `0.9.8` launchers can install either one
  automatically.
- [x] Mark flat manifests as requiring updater `0.9.9` so legacy clients fail
  closed before download/extraction rather than entering a doomed transaction.
- [x] Keep final manifests/signatures/checksums outside the archive they describe.
- [x] Keep signed metadata mandatory for trusted automatic update operations.
- [x] Stable clients can install stable releases only.
- [x] Prerelease clients resolve the greatest eligible stable or prerelease
  semantic version.
- [x] Changing prerelease to stable never causes an automatic downgrade.
- [x] Model release status as `active`, `withdrawn`, or `blocked`; do not offer
  withdrawn/blocked releases as new automatic updates.
- [x] Use each historical release's own signed manifest and immutable versioned
  archive; verify the embedded archive checksum and every target file hash.
- [x] Persist the signed per-file inventory at install time and re-verify it
  before reactivating a locally retained release; preserve compatibility for
  releases installed before integrity records existed.
- [x] Never reuse or overwrite a published tag or its assets.
- [x] Fail rather than duplicating an existing Gitea release.
- [x] Preserve compatibility for existing stable installations during migration.

## Release tooling and configuration (phases 3–7, 12–14, 18, 20–23, 26–27, 65–66)

- [x] Add `.gitea/workflows/release.yml` for the deferred automatic publisher;
  keep it manual-dispatch only while manual upload mode is active.
- [x] Add tracked `scripts/release.sh`, `scripts/build-release.sh`,
  `scripts/generate-manifest.py`, and `scripts/verify-release.sh`.
- [x] Add `release/config.json` with application, release branch, archive naming,
  manifest schema, channels, selected payload, required files, and exclusions.
- [x] Keep generated `dist/` output ignored.
- [x] Explicitly exclude Git/IDE metadata, logs, caches, credentials, developer
  environment files, downloads, test output, and other project-specific files.
- [x] Implement the primary CLI as `./scripts/release.sh CHANNEL VERSION` with
  aliases normalized to `prerelease`.
- [x] Validate the repository root and channel/version pair, then run configured
  preflight checks before creating local artifacts.
- [x] Support `--dry-run`, `--skip-tests`, and an explicit local signing key.
- [x] Isolate preflight stdin so nested CLI tests cannot open an interactive
  release-note prompt, and prefer the repository `.venv` interpreter when present.
- [x] Set the requested version only in the staged package so a manual build does
  not edit the authoritative source version or synchronized project metadata.
- [x] Leave commit, tag, push, Gitea release creation, and asset upload to the
  developer while manual publication mode is active.
- [ ] Restore remote, branch, clean-worktree, tag-absence, version-commit,
  annotated-tag, and ordered-push handling when automatic publication resumes.
- [x] Make dry-run perform validation, notes preview, and optional tests without
  commit, tag, push, or release creation.
- [x] Make `build-release.sh VERSION CHANNEL OUTPUT_DIR` independently build an
  explicitly selected staging tree and clean generated output.
- [x] Normalize archive ordering, timestamps, ownership, group, and permissions
  where practical.
- [x] Generate archive SHA-256 and a sorted per-file manifest without absolute paths.
- [x] Include schema, application, version, channel, tag, commit, archive
  metadata, release metadata, timestamp, status, compatibility, and file hashes.
- [x] Verify archive extraction, required entrypoints, forbidden content,
  checksum, manifest consistency, every managed file hash, and signature before
  publication.
- [x] Give each archive download a signed SHA-256 query key so a CDN cannot keep
  serving a pre-upload empty response from the immutable Gitea asset path.

## Manual Gitea publication milestone

- [x] Produce the versioned archive, archive checksum, signed manifest, and
  release notes together under `dist/v<VERSION>/`.
- [x] Preserve other locally built versions and clean only the selected version
  directory when rebuilding it.
- [x] Verify both the normal flat archive and configured legacy-wrapped bridge;
  the updater installs either payload successfully.
- [x] Print the exact five paths to upload and the matching `v<VERSION>` tag.
- [x] Print whether Gitea's Pre-release checkbox must be selected for the chosen
  channel.
- [x] Generate a signed `dist/v<VERSION>/channel/<channel>.json{,.sig}` pair for the
  developer to copy to `updates/` after the Gitea assets exist.
- [x] Require all five Gitea attachments to be uploaded with nonzero sizes before
  publishing the generated channel pointer.
- [x] Repair the live `v0.9.11` stable pointer after Cloudflare cached a
  pre-upload zero-byte archive response. The pointer now uses the archive's
  signed SHA-256 cache key, and the deployed `0.9.10` updater completed its full
  remote verification against the repaired channel.
- [x] Safely discard generated-only `__pycache__` content before validating a
  signed managed release, while rejecting unexpected cache contents; remove the
  local diagnostic cache that blocked activation and successfully complete the
  `0.9.10` to `0.9.11` transaction.
- [x] Keep the other channel pointer unchanged.
- [x] Manually regenerate signed `updates/history.json{,.sig}` from the live
  manifest-backed Gitea releases so remote history and rollback discovery no
  longer return HTTP 404; automatic regeneration remains deferred.

## Release notes (phases 8–11)

- [x] Support `--description` and Markdown-preserving `--description-file`.
- [x] Generate commit notes from the previous stable tag for stable releases.
- [x] For prereleases, prefer the previous prerelease of the same target version;
  use the previous stable tag for the first prerelease.
- [x] Put developer text before generated commit notes by default.
- [x] Support `--no-auto-changelog`, `--changelog-only`, and `--editor` using
  `${EDITOR:-nano}`.
- [x] Allow interactive Ctrl+D release-note entry only on an interactive stdin;
  use generated notes in noninteractive environments.
- [x] Preview the final release notes before building and copy them unchanged to
  `dist/v<VERSION>/release-notes.md` for manual use as the Gitea release body.

## Deferred Gitea Actions publication (phases 15–17, 28–31, 58–60, 64)

- [x] Implement the `v*` tag-push publisher and CI tag validation for future use.
- [x] Remove its tag-push trigger during manual publication mode; only an
  intentional manual dispatch can start the retained workflow.
- [x] Derive tag, version, channel, prerelease flag, and dereferenced commit SHA
  from the exact checked-out annotated tag.
- [x] Prefer `GITHUB_REF_NAME`, `GITHUB_REPOSITORY`,
  `GITHUB_API_URL`, `GITHUB_SERVER_URL`, `GITHUB_OUTPUT`, `GITHUB_ENV`, and
  `GITHUB_STEP_SUMMARY` context values.
- [x] Use `actions/checkout@v4` and build the exact tagged commit.
- [x] Require a minimally scoped `RELEASE_TOKEN` without logging it.
- [x] Keep the manifest private signing key only in an Actions secret, write it
  with mode 0600 when needed, and remove it immediately after signing.
- [x] Run validation/tests, build, manifest generation, signing, and complete
  verification before creating a Gitea release.
- [x] Create `draft=false` releases through the Gitea API with the correct
  prerelease flag, target commit, title, and release notes; require a successful
  response and release ID.
- [x] Upload archive, checksum, manifest, manifest signature, and release notes
  separately; fail on every required upload error.
- [x] Re-query the release and verify attachment names/sizes and the prerelease flag.
- [x] Print a release summary containing title, channel, tag, commit, archive,
  SHA-256, manifest, and publication status.
- [x] Document Gitea Actions, repository Actions, isolated `act_runner`, required
  commands, trusted tag-only secret exposure, and minimum permissions in
  `release/README.md`.
- [ ] Verify Actions enablement and the registered release runner on the live Gitea.

## Channel indexes and history (phases 32–33)

- [x] Publish independently signed `stable.json` and `prerelease.json` channel
  metadata at stable URLs without cross-channel mutation.
- [x] Publish signed release history with version, channel, tag, manifest URL,
  signature URL, date, notes URL, status, and compatibility metadata.
- [x] Follow the Gitea release-list pagination contract so old rollback releases
  remain in history after the first 100 API results.
- [x] List only real Gitea releases that retain their own manifest and signature.
- [x] Rebuild status-aware channel manifests after a withdrawal/block so legacy
  channel consumers no longer receive an inactive channel head.

## Launcher channels, resolver, and UI (phases 34–44)

- [x] Add validated `update_channel` setting with `stable` default so existing
  users remain stable.
- [x] Add Stable/Prerelease settings UI with clear descriptions.
- [x] Immediately check for updates after changing the selected channel.
- [x] Do not downgrade automatically after switching prerelease to stable.
- [x] Refactor discovery, update eligibility, semantic ordering, channel
  transitions, and inactive-release filtering through one `UpdateResolver`.
- [x] Connect signed remote history and remote rollback choices to `UpdateResolver`.
- [x] Show current/target versions, target channel, release date/title/summary,
  notes, and a clear prerelease label in the update UI.
- [x] Populate version history from actual signed release history and show
  version, channel, date, state, notes, and install/rollback action.
- [x] Add All/Stable/Prerelease history filters with channel-appropriate defaults.
- [x] Stable rollback lists stable releases by default; prerelease rollback lists
  earlier prereleases and stable releases with labels.
- [x] Require explicit action and a configuration-compatibility warning for
  prerelease-to-stable rollback.
- [x] Label the channel of each locally retained release and warn on an explicit
  cross-channel local rollback.

## Rollback transaction hardening (phases 46–50, 52–53)

- [x] Reconcile managed files by activating a complete, target-manifest-verified
  immutable release directory; files absent from the target disappear without
  deleting any unmanaged path.
- [x] Keep config, logs, saves, downloads, and user-created/unmanaged files outside
  managed-file deletion.
- [x] Add configuration schema compatibility metadata and reject unsafe rollback.
- [x] Preserve each installed release's signed configuration compatibility in
  its local integrity record and allow signed release-history overrides for
  withdrawn or compatibility-reclassified historical releases.
- [x] Precheck release/manifest/signature/archive, free space, processes,
  writability, and supported schemas.
- [x] Snapshot current managed state through immutable release links, then
  download/verify/stage/reconcile/apply/health
  check, and restore the snapshot after local transaction failure.
- [x] Permit automatic local restore after failed update application, while
  prohibiting telemetry-driven remote downgrade.
- [ ] Optionally add advanced latest/pinned update policy.
- [x] Retain 30 bounded local update/rollback history records.

## Telemetry and hotfix integration (phases 54–57)

- [x] Include current/target version and channel, selected channel, tag, manifest
  schema/hash, and rollback flag in update telemetry.
- [x] Add discover/download/apply success and failure event families.
- [x] Add requested/download/apply/restore rollback event families with from/to
  versions/channels and an enumerated reason.
- [x] Keep full Markdown descriptions out of telemetry; use version/tag/channel
  and optional release ID only.
- [ ] Make hotfix targeting channel-aware and exact-version aware while reusing
  the release hash/signature verification path.

## Failure and security behavior (phases 27–31, 59–63)

- [x] Reject `.env`, private keys, SSH keys, Git/IDE metadata, logs, caches, and
  bytecode in published archives.
- [x] Never expose release credentials to pull-request workflows or untrusted code.
- [x] Stop publication after test, build, manifest, signing, release-create, or
  required-asset failure; never report a partial release as successful.
- [x] Keep failed tags immutable and publish a new version after correcting a failure.

## Verification matrix (phases 67–76)

- [ ] End-to-end stable release: stable flag, artifacts, signed manifest, stable
  index update, unchanged prerelease index, and stable discovery.
- [ ] End-to-end beta/RC release: prerelease flag, artifacts, prerelease index
  update, unchanged stable index, prerelease discovery, and stable exclusion.
- [x] Exercise inline/file/multiline Markdown, generated notes, no-auto,
  changelog-only, and editor modes; compare preview with Gitea body.
- [x] Exercise both channel preferences and resolver transitions, including no
  implicit downgrade after returning to stable.
- [x] Verify semantic ordering, including `1.10.0 > 1.9.9`, stable filtering,
  and prerelease-to-final transition.
- [x] Exercise stable rollback, prerelease rollback, cross-channel rollback,
  managed-file removal, config/log survival, and failed-apply restoration.
- [x] Exercise withdrawn-release filtering/warnings and valid successor selection.
- [x] Simulate Gitea/API 401, 403, 404, 409, 422, 500, lookup/create/upload
  timeouts, and attachment
  failure without false success.
- [x] Seed forbidden path cases and verify exclusion/failure and absence from the
  deterministic archive.
- [x] Verify valid signatures and reject modified manifest metadata; require
  signed metadata in CI.

## Acceptance criteria (phases 78–83)

- [x] `./scripts/release.sh stable 1.8.0 --description-file ...` validates,
  previews, builds, signs, and verifies the five files needed for a manual
  stable Gitea release without changing Git or contacting Gitea.
- [x] `./scripts/release.sh prerelease 1.9.0-beta.1 --description ...` produces
  the equivalent prerelease upload set and clearly instructs the developer to
  select Gitea's Pre-release checkbox.
- [x] Stable-to-prerelease checks immediately; prerelease-to-stable retains the
  installed beta until a greater eligible stable exists.
- [x] Cross-channel rollback fetches and verifies the target release, warns,
  snapshots, reconciles managed files, applies and health-checks, records
  telemetry, and preserves user state.
- [x] Normal development remains ordinary commits/pushes. A manual release begins
  with `./scripts/release.sh stable VERSION` or
  `./scripts/release.sh prerelease VERSION`; archive, checksum, manifest,
  signature, and release notes generation require no manual work.
- [ ] Re-enable Gitea release creation, asset upload, and signed channel/history
  publication after the manual release process has been proven in production.

## Implementation priority

### P0 — Build first

- [x] Inspect existing updater/release system.
- [x] Semantic-version handling.
- [x] Manual release CLI and all description/changelog/preview modes.
- [x] Stable/prerelease tag validation.
- [x] Retained Gitea publisher implementation with archive, SHA-256, manifest,
  signature, release creation, prerelease flag, upload, and verification.
- [x] Disable its automatic trigger for the current manual publication milestone.

### P1 — Channel support

- [x] `update_channel` setting and settings UI.
- [x] Signed channel metadata and channel-aware discovery.
- [x] Central `UpdateResolver` and semantic ordering.
- [x] Signed release history, release notes in update UI, and channel-aware
  rollback candidates.
- [x] Keep the inline release-notes preview in a fixed-height scrollable area so
  the version table, channel filter, and update controls remain visible.

### P2 — Rollback hardening

- [x] Managed-file reconciliation and snapshot/restore validation.
- [x] Cross-channel rollback and rollback telemetry.
- [x] Withdrawn state and bounded local update history.

### P3 — Security and quality

- [x] Plan a compatible Ed25519 key migration from the deployed RSA verifier.
- [x] Improve and measure reproducibility for archive and manifest output.
- [ ] Integrate channel-aware targeted hotfix verification.
- [ ] Optional version pinning and release notifications.

## Deployment prerequisites

- [x] Manual publication needs no Actions runner, `RELEASE_TOKEN`, or Gitea
  signing secret; the developer needs permission to create a repository release
  and upload attachments in the Gitea UI.
- [ ] Add `RELEASE_TOKEN` with repository release/content permissions to Gitea Actions.
- [ ] Add the manifest signing private key as a protected Gitea Actions secret.
- [ ] Confirm Actions, the repository workflow, and an isolated release runner are enabled.
- [ ] Confirm the runner image provides Git, Bash, curl, jq, Python 3, tar, gzip,
  sha256sum, rsync, and OpenSSL/signing support.

The configured SSH remote is reachable and `origin/main` resolves successfully.
The current development environment does not expose either Actions secret, and
the public Gitea API currently returns empty HTTP 200 responses, so runner,
repository-Actions, and secret state cannot be authenticated from this checkout.

## Validation evidence

- [x] 68 release-pipeline, resolver, channel, history, signing, rollback,
  telemetry, CLI, publisher, status, and API-failure tests pass locally.
- [x] All 575 current project tests pass during the complete manual release preflight.
- [x] The published flat `stable 0.9.9` package demonstrated the expected
  compatibility boundary: deployed `0.9.8` clients reject its declared minimum
  updater instead of attempting an archive layout they cannot extract.
- [x] A real wrapped `stable 0.9.10` bridge under `dist/v0.9.10/` built, signed,
  verified, passed updater staging, and is offered automatically to `0.9.8`.
  Its only tar root is `pristontale-eu-linux-update-v0.9.10`; archive SHA-256:
  `76df8942d2f0b3b533eb967f8a2379b2a7bdd44fd58305d0f8bf061778fbd9ea`.
- [x] A real wrapped `stable 0.9.11` UI repair under `dist/v0.9.11/` built,
  signed, and verified with minimum updater `0.5.0`. Its only tar root is
  `pristontale-eu-linux-update-v0.9.11`; archive SHA-256:
  `60b13d6fbc8fe27714c7fe1a65fa73785a13962d96d03decfac1aa62c4e3f6db`.
- [ ] Publish the prepared `v0.9.11` release and its five assets in Gitea. The
  remote currently has no `v0.9.11` tag, so its stable channel still resolves
  to `0.9.10` even when the PyCharm checkout is running `0.9.11`.
- [ ] After the Gitea assets exist, commit and push the prepared
  `updates/stable.json` and `updates/stable.json.sig` pair that points stable
  discovery to `0.9.11`.
- [x] Keep the `0.9.11` embedded notes to one concise change line so clients
  still running the overflowing dialog can reach the install action.
- [x] Hidden-window Tk geometry validation at the default scale confirms the
  notes viewport, channel filter, 124-pixel history table, and footer all fit
  inside the 820×680 update dialog.
- [x] Reproduce the reported `0.9.10` layout from its published archive: the old
  dialog requests 820×828 pixels while forcing a 760×620 window. Confirm the
  `0.9.11` archive uses the repaired dialog and keeps notes, history, and footer
  in separate non-overlapping bounds; its deliberately short channel notes also
  leave the old `0.9.10` install button reachable during the repair upgrade.
- [x] Synchronize the workspace version source and project metadata to `0.9.11`,
  and make the shared PyCharm launcher configuration put `$PROJECT_DIR$` first
  on `PYTHONPATH`, so debug runs load the current checkout rather than an older
  installed release.
- [x] Keep update-dialog version labels tied to the running launcher. When a
  PyCharm checkout is newer than the active installed symlink, show the checkout
  as `Current` and describe the older on-disk release as `Installed`.
- [x] Prior deterministic-build validation produced byte-identical stable
  `0.9.8` archives with SHA-256
  `72346f216856235df3c6071409e2bbce17853b9f9589bf3f012743ddf7823f4b`.
- [x] Shell syntax, JSON parsing, Git whitespace checks, PHP telemetry endpoint
  syntax, Python compilation, and IDE error inspections pass.
- [ ] Run stable and prerelease publication acceptance tests against the live
  Gitea after the protected secrets and isolated runner are configured.