diff --git a/RELEASE_PIPELINE_WORKPLAN.md b/RELEASE_PIPELINE_WORKPLAN.md deleted file mode 100644 index fc96228..0000000 --- a/RELEASE_PIPELINE_WORKPLAN.md +++ /dev/null @@ -1,444 +0,0 @@ -# PristonTale EU Linux Launcher Release Pipeline Workplan - -This is the tracked implementation plan for manual Gitea releases today and the -deferred automatic publication path, stable and prerelease channels, signed -manifests, release history, and channel-aware rollback. A checked item means the -repository contains the implementation or the audit evidence named beside it. -Publishing tests that require a configured Gitea server remain unchecked until -they have run against that server. - -Legend: `[x]` complete, `[ ]` pending. - -## Current milestone - -- [x] Save the release workplan in the repository. -- [x] Inspect the existing release, update, rollback, settings, UI, and telemetry paths. -- [x] Implement and validate the automatic publication pipeline for future use. -- [x] Switch to temporary manual-publication mode: `release.sh` now creates a - signed, verified `dist/v/` package without commits, tags, pushes, API - calls, or uploads. -- [x] Disable automatic workflow push triggers so manual Gitea releases cannot - race an Actions publication job. -- [x] Store normal update payloads directly at the tar root while retaining - runtime extraction support for historical archives and the `0.9.10`–`0.9.11` - compatibility window. -- [ ] Re-enable automatic Gitea publication when requested. -- [x] P1: add stable/prerelease discovery and launcher channel support. -- [x] P2: harden remote rollback and managed-file reconciliation. -- [ ] P3: complete signing migration and release quality improvements. - -## Phase 0 — Existing-system audit - -- [x] Repository structure inspected. Runtime sources are in `ptlinux/` and - `ptdiag/`; installer assets are in `packaging/`; the checked-in channel is in - `updates/`; local, ignored development tools and tests are in `tools/` and - `tests/`. -- [x] Authoritative launcher version identified as `ptlinux/__init__.py`. - `pyproject.toml` is a required synchronized metadata mirror. -- [x] Current update archive identified as - `pristontale-eu-linux-update-v.tar.gz`. -- [x] Current archive layout identified. Its root is - `pristontale-eu-linux-update-v/`, containing `ptlinux/`, a compact - `ptdiag/` subset, and `release.json`. -- [x] Current update manifest identified as signed JSON schema 1 in - `updates/stable.json`, with version, channel, minimum updater version, archive - URL/size/SHA-256, publication time, and notes URL. -- [x] Current update URL identified as the raw `main` branch - `updates/stable.json`; `PTLINUX_UPDATE_MANIFEST_URL` can override it. -- [x] Latest-release discovery identified as the mutable signed stable manifest. - The launcher does not use Gitea `/releases/latest`. -- [x] Current rollback identified as local retained release directories selected - by atomic `current` and `previous` symlinks. -- [x] Cached archives identified in the launcher download cache and reused only - when their size and SHA-256 match the authenticated manifest. -- [x] Current staging identified: download to a `.part` file, verify while - downloading, safely extract to a staging directory, validate/self-import, - move into the immutable release store, then atomically replace symlinks. -- [x] Atomic activation confirmed through same-directory temporary symlink plus - `os.replace()` and directory `fsync`. -- [x] Removed launcher files are handled because each release is a complete, - separate directory. No in-place overlay is used. -- [x] Fixed release asset names are not required by the current launcher; the - authenticated manifest supplies the archive URL. -- [x] Existing manually generated files identified: installer tar/zip archives, - launcher update tar archive, SHA-256 files, stable manifest/signature, and - release uploads. -- [x] Existing manual exclusions identified. The launcher update builder selects - Python/runtime files explicitly and excludes installer-only modules; the AIO - builder selects runtime, packaging, installer, and README files explicitly. -- [x] SHA-256 generation and verification confirmed for archives. -- [x] Existing asymmetric signature support confirmed. The manifest is signed - and verified with the bundled public key through OpenSSL (currently RSA with - SHA-256); the private key is ignored under `release-keys/`. -- [x] Existing telemetry inspected and extended with channel, target release, - manifest identity, and complete update/rollback lifecycle metadata. -- [x] Decision: extend the existing signed-manifest and atomic release-directory - updater instead of introducing a second updater. - -## Release model and invariants (phases 1–2, 19, 24–25, 32–38, 45–51, 61–63, 77, 84) - -- [x] Support exactly `stable` and `prerelease` channels in the initial model. -- [x] Accept stable tags `vMAJOR.MINOR.PATCH` and prerelease tags ending in - `-beta.N` or `-rc.N`; reject channel/tag mismatches. -- [x] Treat the manually selected Gitea release tag as the authoritative, - immutable release identity. -- [ ] Restore annotated-tag and channel-pointer monotonicity enforcement when - automatic publication is re-enabled. Manual publishers must currently check - the tag in Gitea and the version ordering before copying a channel pointer. -- [x] Use one semantic-version parser and one channel/update resolver; never use - lexical version ordering. -- [x] Preserve ordering from beta to RC to final, including multi-digit fields. -- [x] Keep the internal launcher version without the leading `v`. -- [x] Build immutable, versioned assets named - `PristonTale-EU-.tar.gz`; normal releases place `ptlinux/`, `ptdiag/`, - and `release.json` directly at the archive root. -- [x] Accept both new flat archives and legacy version-folder archives in the - updated launcher so retained releases and rollback remain compatible. -- [x] Configure stable `0.9.10` and its `0.9.11` UI repair as legacy-wrapped - bridge releases so deployed `0.9.8` launchers can install either one - automatically. -- [x] Mark flat manifests as requiring updater `0.9.9` so legacy clients fail - closed before download/extraction rather than entering a doomed transaction. -- [x] Keep final manifests/signatures/checksums outside the archive they describe. -- [x] Keep signed metadata mandatory for trusted automatic update operations. -- [x] Stable clients can install stable releases only. -- [x] Prerelease clients resolve the greatest eligible stable or prerelease - semantic version. -- [x] Changing prerelease to stable never causes an automatic downgrade. -- [x] Model release status as `active`, `withdrawn`, or `blocked`; do not offer - withdrawn/blocked releases as new automatic updates. -- [x] Use each historical release's own signed manifest and immutable versioned - archive; verify the embedded archive checksum and every target file hash. -- [x] Persist the signed per-file inventory at install time and re-verify it - before reactivating a locally retained release; preserve compatibility for - releases installed before integrity records existed. -- [x] Never reuse or overwrite a published tag or its assets. -- [x] Fail rather than duplicating an existing Gitea release. -- [x] Preserve compatibility for existing stable installations during migration. - -## Release tooling and configuration (phases 3–7, 12–14, 18, 20–23, 26–27, 65–66) - -- [x] Add `.gitea/workflows/release.yml` for the deferred automatic publisher; - keep it manual-dispatch only while manual upload mode is active. -- [x] Add tracked `scripts/release.sh`, `scripts/build-release.sh`, - `scripts/generate-manifest.py`, and `scripts/verify-release.sh`. -- [x] Add `release/config.json` with application, release branch, archive naming, - manifest schema, channels, selected payload, required files, and exclusions. -- [x] Keep generated `dist/` output ignored. -- [x] Explicitly exclude Git/IDE metadata, logs, caches, credentials, developer - environment files, downloads, test output, and other project-specific files. -- [x] Implement the primary CLI as `./scripts/release.sh CHANNEL VERSION` with - aliases normalized to `prerelease`. -- [x] Validate the repository root and channel/version pair, then run configured - preflight checks before creating local artifacts. -- [x] Support `--dry-run`, `--skip-tests`, and an explicit local signing key. -- [x] Isolate preflight stdin so nested CLI tests cannot open an interactive - release-note prompt, and prefer the repository `.venv` interpreter when present. -- [x] Set the requested version only in the staged package so a manual build does - not edit the authoritative source version or synchronized project metadata. -- [x] Leave commit, tag, push, Gitea release creation, and asset upload to the - developer while manual publication mode is active. -- [ ] Restore remote, branch, clean-worktree, tag-absence, version-commit, - annotated-tag, and ordered-push handling when automatic publication resumes. -- [x] Make dry-run perform validation, notes preview, and optional tests without - commit, tag, push, or release creation. -- [x] Make `build-release.sh VERSION CHANNEL OUTPUT_DIR` independently build an - explicitly selected staging tree and clean generated output. -- [x] Normalize archive ordering, timestamps, ownership, group, and permissions - where practical. -- [x] Generate archive SHA-256 and a sorted per-file manifest without absolute paths. -- [x] Include schema, application, version, channel, tag, commit, archive - metadata, release metadata, timestamp, status, compatibility, and file hashes. -- [x] Verify archive extraction, required entrypoints, forbidden content, - checksum, manifest consistency, every managed file hash, and signature before - publication. -- [x] Give each archive download a signed SHA-256 query key so a CDN cannot keep - serving a pre-upload empty response from the immutable Gitea asset path. - -## Manual Gitea publication milestone - -- [x] Produce the versioned archive, archive checksum, signed manifest, and - release notes together under `dist/v/`. -- [x] Preserve other locally built versions and clean only the selected version - directory when rebuilding it. -- [x] Verify both the normal flat archive and configured legacy-wrapped bridge; - the updater installs either payload successfully. -- [x] Print the exact five paths to upload and the matching `v` tag. -- [x] Print whether Gitea's Pre-release checkbox must be selected for the chosen - channel. -- [x] Generate a signed `dist/v/channel/.json{,.sig}` pair for the - developer to copy to `updates/` after the Gitea assets exist. -- [x] Require all five Gitea attachments to be uploaded with nonzero sizes before - publishing the generated channel pointer. -- [x] Repair the live `v0.9.11` stable pointer after Cloudflare cached a - pre-upload zero-byte archive response. The pointer now uses the archive's - signed SHA-256 cache key, and the deployed `0.9.10` updater completed its full - remote verification against the repaired channel. -- [x] Safely discard generated-only `__pycache__` content before validating a - signed managed release, while rejecting unexpected cache contents; remove the - local diagnostic cache that blocked activation and successfully complete the - `0.9.10` to `0.9.11` transaction. -- [x] Keep the other channel pointer unchanged. -- [x] Manually regenerate signed `updates/history.json{,.sig}` from the live - manifest-backed Gitea releases so remote history and rollback discovery no - longer return HTTP 404; automatic regeneration remains deferred. - -## Release notes (phases 8–11) - -- [x] Support `--description` and Markdown-preserving `--description-file`. -- [x] Generate commit notes from the previous stable tag for stable releases. -- [x] For prereleases, prefer the previous prerelease of the same target version; - use the previous stable tag for the first prerelease. -- [x] Put developer text before generated commit notes by default. -- [x] Support `--no-auto-changelog`, `--changelog-only`, and `--editor` using - `${EDITOR:-nano}`. -- [x] Allow interactive Ctrl+D release-note entry only on an interactive stdin; - use generated notes in noninteractive environments. -- [x] Preview the final release notes before building and copy them unchanged to - `dist/v/release-notes.md` for manual use as the Gitea release body. - -## Deferred Gitea Actions publication (phases 15–17, 28–31, 58–60, 64) - -- [x] Implement the `v*` tag-push publisher and CI tag validation for future use. -- [x] Remove its tag-push trigger during manual publication mode; only an - intentional manual dispatch can start the retained workflow. -- [x] Derive tag, version, channel, prerelease flag, and dereferenced commit SHA - from the exact checked-out annotated tag. -- [x] Prefer `GITHUB_REF_NAME`, `GITHUB_REPOSITORY`, - `GITHUB_API_URL`, `GITHUB_SERVER_URL`, `GITHUB_OUTPUT`, `GITHUB_ENV`, and - `GITHUB_STEP_SUMMARY` context values. -- [x] Use `actions/checkout@v4` and build the exact tagged commit. -- [x] Require a minimally scoped `RELEASE_TOKEN` without logging it. -- [x] Keep the manifest private signing key only in an Actions secret, write it - with mode 0600 when needed, and remove it immediately after signing. -- [x] Run validation/tests, build, manifest generation, signing, and complete - verification before creating a Gitea release. -- [x] Create `draft=false` releases through the Gitea API with the correct - prerelease flag, target commit, title, and release notes; require a successful - response and release ID. -- [x] Upload archive, checksum, manifest, manifest signature, and release notes - separately; fail on every required upload error. -- [x] Re-query the release and verify attachment names/sizes and the prerelease flag. -- [x] Print a release summary containing title, channel, tag, commit, archive, - SHA-256, manifest, and publication status. -- [x] Document Gitea Actions, repository Actions, isolated `act_runner`, required - commands, trusted tag-only secret exposure, and minimum permissions in - `release/README.md`. -- [ ] Verify Actions enablement and the registered release runner on the live Gitea. - -## Channel indexes and history (phases 32–33) - -- [x] Publish independently signed `stable.json` and `prerelease.json` channel - metadata at stable URLs without cross-channel mutation. -- [x] Publish signed release history with version, channel, tag, manifest URL, - signature URL, date, notes URL, status, and compatibility metadata. -- [x] Follow the Gitea release-list pagination contract so old rollback releases - remain in history after the first 100 API results. -- [x] List only real Gitea releases that retain their own manifest and signature. -- [x] Rebuild status-aware channel manifests after a withdrawal/block so legacy - channel consumers no longer receive an inactive channel head. - -## Launcher channels, resolver, and UI (phases 34–44) - -- [x] Add validated `update_channel` setting with `stable` default so existing - users remain stable. -- [x] Add Stable/Prerelease settings UI with clear descriptions. -- [x] Immediately check for updates after changing the selected channel. -- [x] Do not downgrade automatically after switching prerelease to stable. -- [x] Refactor discovery, update eligibility, semantic ordering, channel - transitions, and inactive-release filtering through one `UpdateResolver`. -- [x] Connect signed remote history and remote rollback choices to `UpdateResolver`. -- [x] Show current/target versions, target channel, release date/title/summary, - notes, and a clear prerelease label in the update UI. -- [x] Populate version history from actual signed release history and show - version, channel, date, state, notes, and install/rollback action. -- [x] Add All/Stable/Prerelease history filters with channel-appropriate defaults. -- [x] Stable rollback lists stable releases by default; prerelease rollback lists - earlier prereleases and stable releases with labels. -- [x] Require explicit action and a configuration-compatibility warning for - prerelease-to-stable rollback. -- [x] Label the channel of each locally retained release and warn on an explicit - cross-channel local rollback. - -## Rollback transaction hardening (phases 46–50, 52–53) - -- [x] Reconcile managed files by activating a complete, target-manifest-verified - immutable release directory; files absent from the target disappear without - deleting any unmanaged path. -- [x] Keep config, logs, saves, downloads, and user-created/unmanaged files outside - managed-file deletion. -- [x] Add configuration schema compatibility metadata and reject unsafe rollback. -- [x] Preserve each installed release's signed configuration compatibility in - its local integrity record and allow signed release-history overrides for - withdrawn or compatibility-reclassified historical releases. -- [x] Precheck release/manifest/signature/archive, free space, processes, - writability, and supported schemas. -- [x] Snapshot current managed state through immutable release links, then - download/verify/stage/reconcile/apply/health - check, and restore the snapshot after local transaction failure. -- [x] Permit automatic local restore after failed update application, while - prohibiting telemetry-driven remote downgrade. -- [ ] Optionally add advanced latest/pinned update policy. -- [x] Retain 30 bounded local update/rollback history records. - -## Telemetry and hotfix integration (phases 54–57) - -- [x] Include current/target version and channel, selected channel, tag, manifest - schema/hash, and rollback flag in update telemetry. -- [x] Add discover/download/apply success and failure event families. -- [x] Add requested/download/apply/restore rollback event families with from/to - versions/channels and an enumerated reason. -- [x] Keep full Markdown descriptions out of telemetry; use version/tag/channel - and optional release ID only. -- [ ] Make hotfix targeting channel-aware and exact-version aware while reusing - the release hash/signature verification path. - -## Failure and security behavior (phases 27–31, 59–63) - -- [x] Reject `.env`, private keys, SSH keys, Git/IDE metadata, logs, caches, and - bytecode in published archives. -- [x] Never expose release credentials to pull-request workflows or untrusted code. -- [x] Stop publication after test, build, manifest, signing, release-create, or - required-asset failure; never report a partial release as successful. -- [x] Keep failed tags immutable and publish a new version after correcting a failure. - -## Verification matrix (phases 67–76) - -- [ ] End-to-end stable release: stable flag, artifacts, signed manifest, stable - index update, unchanged prerelease index, and stable discovery. -- [ ] End-to-end beta/RC release: prerelease flag, artifacts, prerelease index - update, unchanged stable index, prerelease discovery, and stable exclusion. -- [x] Exercise inline/file/multiline Markdown, generated notes, no-auto, - changelog-only, and editor modes; compare preview with Gitea body. -- [x] Exercise both channel preferences and resolver transitions, including no - implicit downgrade after returning to stable. -- [x] Verify semantic ordering, including `1.10.0 > 1.9.9`, stable filtering, - and prerelease-to-final transition. -- [x] Exercise stable rollback, prerelease rollback, cross-channel rollback, - managed-file removal, config/log survival, and failed-apply restoration. -- [x] Exercise withdrawn-release filtering/warnings and valid successor selection. -- [x] Simulate Gitea/API 401, 403, 404, 409, 422, 500, lookup/create/upload - timeouts, and attachment - failure without false success. -- [x] Seed forbidden path cases and verify exclusion/failure and absence from the - deterministic archive. -- [x] Verify valid signatures and reject modified manifest metadata; require - signed metadata in CI. - -## Acceptance criteria (phases 78–83) - -- [x] `./scripts/release.sh stable 1.8.0 --description-file ...` validates, - previews, builds, signs, and verifies the five files needed for a manual - stable Gitea release without changing Git or contacting Gitea. -- [x] `./scripts/release.sh prerelease 1.9.0-beta.1 --description ...` produces - the equivalent prerelease upload set and clearly instructs the developer to - select Gitea's Pre-release checkbox. -- [x] Stable-to-prerelease checks immediately; prerelease-to-stable retains the - installed beta until a greater eligible stable exists. -- [x] Cross-channel rollback fetches and verifies the target release, warns, - snapshots, reconciles managed files, applies and health-checks, records - telemetry, and preserves user state. -- [x] Normal development remains ordinary commits/pushes. A manual release begins - with `./scripts/release.sh stable VERSION` or - `./scripts/release.sh prerelease VERSION`; archive, checksum, manifest, - signature, and release notes generation require no manual work. -- [ ] Re-enable Gitea release creation, asset upload, and signed channel/history - publication after the manual release process has been proven in production. - -## Implementation priority - -### P0 — Build first - -- [x] Inspect existing updater/release system. -- [x] Semantic-version handling. -- [x] Manual release CLI and all description/changelog/preview modes. -- [x] Stable/prerelease tag validation. -- [x] Retained Gitea publisher implementation with archive, SHA-256, manifest, - signature, release creation, prerelease flag, upload, and verification. -- [x] Disable its automatic trigger for the current manual publication milestone. - -### P1 — Channel support - -- [x] `update_channel` setting and settings UI. -- [x] Signed channel metadata and channel-aware discovery. -- [x] Central `UpdateResolver` and semantic ordering. -- [x] Signed release history, release notes in update UI, and channel-aware - rollback candidates. -- [x] Keep the inline release-notes preview in a fixed-height scrollable area so - the version table, channel filter, and update controls remain visible. - -### P2 — Rollback hardening - -- [x] Managed-file reconciliation and snapshot/restore validation. -- [x] Cross-channel rollback and rollback telemetry. -- [x] Withdrawn state and bounded local update history. - -### P3 — Security and quality - -- [x] Plan a compatible Ed25519 key migration from the deployed RSA verifier. -- [x] Improve and measure reproducibility for archive and manifest output. -- [ ] Integrate channel-aware targeted hotfix verification. -- [ ] Optional version pinning and release notifications. - -## Deployment prerequisites - -- [x] Manual publication needs no Actions runner, `RELEASE_TOKEN`, or Gitea - signing secret; the developer needs permission to create a repository release - and upload attachments in the Gitea UI. -- [ ] Add `RELEASE_TOKEN` with repository release/content permissions to Gitea Actions. -- [ ] Add the manifest signing private key as a protected Gitea Actions secret. -- [ ] Confirm Actions, the repository workflow, and an isolated release runner are enabled. -- [ ] Confirm the runner image provides Git, Bash, curl, jq, Python 3, tar, gzip, - sha256sum, rsync, and OpenSSL/signing support. - -The configured SSH remote is reachable and `origin/main` resolves successfully. -The current development environment does not expose either Actions secret, and -the public Gitea API currently returns empty HTTP 200 responses, so runner, -repository-Actions, and secret state cannot be authenticated from this checkout. - -## Validation evidence - -- [x] 68 release-pipeline, resolver, channel, history, signing, rollback, - telemetry, CLI, publisher, status, and API-failure tests pass locally. -- [x] All 575 current project tests pass during the complete manual release preflight. -- [x] The published flat `stable 0.9.9` package demonstrated the expected - compatibility boundary: deployed `0.9.8` clients reject its declared minimum - updater instead of attempting an archive layout they cannot extract. -- [x] A real wrapped `stable 0.9.10` bridge under `dist/v0.9.10/` built, signed, - verified, passed updater staging, and is offered automatically to `0.9.8`. - Its only tar root is `pristontale-eu-linux-update-v0.9.10`; archive SHA-256: - `76df8942d2f0b3b533eb967f8a2379b2a7bdd44fd58305d0f8bf061778fbd9ea`. -- [x] A real wrapped `stable 0.9.11` UI repair under `dist/v0.9.11/` built, - signed, and verified with minimum updater `0.5.0`. Its only tar root is - `pristontale-eu-linux-update-v0.9.11`; archive SHA-256: - `60b13d6fbc8fe27714c7fe1a65fa73785a13962d96d03decfac1aa62c4e3f6db`. -- [ ] Publish the prepared `v0.9.11` release and its five assets in Gitea. The - remote currently has no `v0.9.11` tag, so its stable channel still resolves - to `0.9.10` even when the PyCharm checkout is running `0.9.11`. -- [ ] After the Gitea assets exist, commit and push the prepared - `updates/stable.json` and `updates/stable.json.sig` pair that points stable - discovery to `0.9.11`. -- [x] Keep the `0.9.11` embedded notes to one concise change line so clients - still running the overflowing dialog can reach the install action. -- [x] Hidden-window Tk geometry validation at the default scale confirms the - notes viewport, channel filter, 124-pixel history table, and footer all fit - inside the 820×680 update dialog. -- [x] Reproduce the reported `0.9.10` layout from its published archive: the old - dialog requests 820×828 pixels while forcing a 760×620 window. Confirm the - `0.9.11` archive uses the repaired dialog and keeps notes, history, and footer - in separate non-overlapping bounds; its deliberately short channel notes also - leave the old `0.9.10` install button reachable during the repair upgrade. -- [x] Synchronize the workspace version source and project metadata to `0.9.11`, - and make the shared PyCharm launcher configuration put `$PROJECT_DIR$` first - on `PYTHONPATH`, so debug runs load the current checkout rather than an older - installed release. -- [x] Keep update-dialog version labels tied to the running launcher. When a - PyCharm checkout is newer than the active installed symlink, show the checkout - as `Current` and describe the older on-disk release as `Installed`. -- [x] Prior deterministic-build validation produced byte-identical stable - `0.9.8` archives with SHA-256 - `72346f216856235df3c6071409e2bbce17853b9f9589bf3f012743ddf7823f4b`. -- [x] Shell syntax, JSON parsing, Git whitespace checks, PHP telemetry endpoint - syntax, Python compilation, and IDE error inspections pass. -- [ ] Run stable and prerelease publication acceptance tests against the live - Gitea after the protected secrets and isolated runner are configured.