Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ jobs:
promotion_revision: ${{ steps.identity.outputs.promotion_revision }}
workflow_revision: ${{ steps.candidate.outputs.workflow_revision }}
docs_sha256: ${{ steps.candidate.outputs.docs_sha256 }}
identifier_catalog_sha256: ${{ steps.candidate.outputs.identifier_catalog_sha256 }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add the required DCO sign-off

The commit object for b573544b5b1e40d77cfe679a8fb57dc55d7dffe5 has no Signed-off-by: trailer, so this commit does not satisfy the repository's mandatory DCO policy; recreate or squash the change into a signed-off commit before merging.

AGENTS.md reference: AGENTS.md:L405-L407

Useful? React with 👍 / 👎.

client_registries: ${{ steps.candidate.outputs.client_registries }}
client_registry_matrix: ${{ steps.candidate.outputs.client_registry_matrix }}
client_registry_pypi_matrix: ${{ steps.candidate.outputs.client_registry_pypi_matrix }}
Expand Down Expand Up @@ -233,6 +234,11 @@ jobs:
--deny-self-hosted-runners
done
docs_sha256=
identifier_catalog_sha256="$(
git show \
"${{ steps.identity.outputs.source_sha }}:products/identifiers/generated/catalog.v1.json" \
| sha256sum | awk '{print $1}'
)"
Comment on lines +237 to +241

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip identifier hashing for v0.19.0

When the maintained publication workflow is used for its accepted historical v0.19.0 case, this unconditional git show fails under set -euo pipefail because identifier catalogs begin at v0.19.1: IDENTIFIER_CATALOG_RELEASE_MINIMUM_VERSION is (0, 19, 1), and the v0.19.0 manifest has no catalog binding. This prevents even the already-published closeout path from reaching destination classification, so compute and publish the digest only for versions at or above the catalog minimum.

Useful? React with 👍 / 👎.

IFS=. read -r major minor patch <<<"${{ steps.identity.outputs.version }}"
if ((
major > 0 ||
Expand Down Expand Up @@ -300,6 +306,7 @@ jobs:
echo "release_id=${release_id}"
echo "workflow_revision=${workflow_revision}"
echo "docs_sha256=${docs_sha256}"
echo "identifier_catalog_sha256=${identifier_catalog_sha256}"
echo "client_registries=${client_registries}"
echo "client_registry_matrix=${client_registry_matrix}"
echo "client_registry_pypi_matrix=${client_registry_pypi_matrix}"
Expand Down Expand Up @@ -1548,3 +1555,70 @@ jobs:
-f "released_tag=${{ needs.verify.outputs.tag }}" \
-f "docs_sha256=${{ needs.verify.outputs.docs_sha256 }}" \
-f "request_id=${{ inputs.request_id }}"

publish-identifiers:
name: Publish exact identifier catalog
if: >-
always() &&
(needs.publish.result == 'success' ||
needs.closeout-published.result == 'success')
needs:
- verify
- publish
- closeout-published
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Dispatch and verify exact identifier publication
env:
GH_TOKEN: ${{ secrets.IDENTIFIER_PUBLISHER_TOKEN }}
RELEASE_REQUEST_ID: ${{ inputs.request_id }}
shell: bash
run: |
set -euo pipefail
request_id="${RELEASE_REQUEST_ID}"
if [[ -z "${request_id}" ]]; then
request_id="${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
fi
title="Publish Registry Stack identifiers ${{ needs.verify.outputs.tag }} (${request_id})"
previous_run_id="$(
gh run list \
--repo registrystack/registrystack-id \
--workflow deploy-cloudflare-workers.yml \
--event workflow_dispatch \
--limit 100 \
--json databaseId,displayTitle \
--jq "[.[] | select(.displayTitle == \"${title}\") | .databaseId] | max // 0"
)"
gh workflow run deploy-cloudflare-workers.yml \
--repo registrystack/registrystack-id \
--ref main \
-f "released_tag=${{ needs.verify.outputs.tag }}" \
-f "source_sha=${{ needs.verify.outputs.source_sha }}" \
-f "catalog_sha256=${{ needs.verify.outputs.identifier_catalog_sha256 }}" \
-f "request_id=${request_id}"

deadline=$((SECONDS + 120))
target_run_id=
while [[ -z "${target_run_id}" ]]; do
target_run_id="$(
gh run list \
--repo registrystack/registrystack-id \
--workflow deploy-cloudflare-workers.yml \
--event workflow_dispatch \
--limit 100 \
--json databaseId,displayTitle \
--jq "[.[] | select(.displayTitle == \"${title}\" and .databaseId > ${previous_run_id}) | .databaseId] | max // empty"
)"
if [[ -n "${target_run_id}" ]]; then
break
fi
if (( SECONDS >= deadline )); then
echo "Timed out correlating the identifier publication run" >&2
exit 1
fi
sleep 2
done
gh run watch "${target_run_id}" \
--repo registrystack/registrystack-id \
--exit-status
Comment on lines +1622 to +1624

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid watching runs with a fine-grained PAT

When this job uses the documented fine-grained IDENTIFIER_PUBLISHER_TOKEN, gh run watch cannot authenticate: gh run watch --help states, “This command does not support authenticating via fine grained PATs” because the required checks:read permission is unavailable. Consequently, the identifier deployment may succeed but every release run and retry will fail while waiting for it after the release is already immutable; poll the run through an Actions-compatible API/command or use an authentication method that supports Checks.

Useful? React with 👍 / 👎.

27 changes: 16 additions & 11 deletions products/identifiers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,12 @@ products/identifiers/scripts/check.sh

The publisher imports this catalog from one exact Registry Stack commit. It
copies artifact bytes only after checking their recorded digest and publishes
exactly the identifiers in that catalog. An identifier removed from current
source is removed from the resolver. A current entry may update the metadata
for the same identifier, but neither the catalog nor the publisher may change
an identifier's kind or reuse it for a different meaning.
the active identifiers in that catalog. A previously published identifier

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Split the change by owning area

This change combines release automation under .github/workflows/ and release/ with the identifier product contract under products/identifiers/, contrary to the repository requirement that one change remain within one owning area; separate the product-contract update from the release-owned workflow and operational changes.

AGENTS.md reference: AGENTS.md:L413-L415

Useful? React with 👍 / 👎.

removed from current source remains resolvable as `deprecated`, using its last
reviewed metadata. A current entry may update the metadata for the same
identifier, but neither the catalog nor the publisher may change an
identifier's kind or reuse it for a different meaning. Immutable artifact paths
remain available after the corresponding canonical artifact advances.

The canonical schema URI may identify the current schema within its named
compatibility line. The publisher also exposes the exact imported bytes by
Expand All @@ -63,8 +65,9 @@ or release blocker.
- Trust boundary: identifier metadata describes existing product behavior. It
creates no principal, claim, permission, disclosure rule, credential, or
signing authority.
- Recovery: fix forward with a reviewed catalog and redeploy. A removed URI may
remain absent, but it must never be reused or repurposed.
- Recovery: rerun publication for the exact released source and catalog digest,
or fix forward with a reviewed catalog. A removed URI remains reserved and
resolvable; it must never be reused or repurposed.

## Definition of Done

Expand All @@ -79,8 +82,9 @@ true:
- Product problem generation preserves each closed value-free problem inventory
and adds no authentication or disclosure behavior.
- Retired Relay V1, Registry Notary, Registry Platform operations, registryctl,
and release-lock identifiers are absent from the generated catalog and
resolver.
and release-lock identifiers are absent from the generated active catalog.
A path that was already published remains available as historical resolver
output and cannot be reused.
- Registry-owned Relay V2 vocabulary identifiers are published; Solmara demo
identifiers and legacy SHACL fixture identifiers remain outside the public
catalog because they are not Registry Stack contracts.
Expand All @@ -90,9 +94,10 @@ true:
- Future release manifests from version `0.19.1` bind the catalog path, digest,
and entry count without making live resolver availability a release gate.
- The publisher imports an exact Registry Stack commit, verifies every digest,
publishes exactly the active catalog, generates the complete static site,
and has pull-request validation, automated source synchronization, and a
scheduled live smoke check.
publishes the active catalog plus deprecated historical identifiers and
retained immutable artifacts, generates the complete static site, and has
pull-request validation, automated source synchronization, and a scheduled
exact-byte live smoke check.
- Focused Rust, generator, CI-routing, documentation, publisher, and
cross-repository checks pass with no unrelated changes.
- Separate draft pull requests exist for `registry-stack` and
Expand Down
28 changes: 22 additions & 6 deletions release/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ Start release preparation when:
workflow write access.
- The `npm`, `pypi`, and `pypi-evidence` GitHub environments exist with the
intended release approvers.
- The `IDENTIFIER_PUBLISHER_TOKEN` repository secret is a fine-grained token
with Actions read/write access only to `registrystack/registrystack-id`.
Cloudflare credentials remain in that publisher repository.
- A reviewed `registrystack-id` synchronization pull request for the exact
release source commit and identifier catalog digest has been merged. The
release dispatch deploys that committed bundle and fails closed if it differs.
- Every dependency pinned to a long-term-support line has been checked for
remaining upstream support: more than six months must remain, or the
next-LTS migration is already promoted to a release blocker for this cut.
Expand Down Expand Up @@ -819,10 +825,10 @@ The command re-verifies the plan against the exact candidate, protected-main
ancestry, source commit policy, and current public destinations. It creates and
inspects the exact annotated candidate tag, pushes only that tag ref, dispatches
publication with a unique correlation ID, follows only the correlated run, runs
the public verifier, and waits for the correlated documentation deployment. The
tag is annotated but not cryptographically signed. The command never approves a
protected environment. It prints the environment and exact run URL when an
authorized reviewer must use **Review deployments**.
the public verifier, and waits for the correlated documentation and identifier
deployments. The tag is annotated but not cryptographically signed. The command
never approves a protected environment. It prints the environment and exact run
URL when an authorized reviewer must use **Review deployments**.

The command is resumable. Rerun the same `publish --plan ... --wait` invocation
after an interruption. It accepts only an exact local-only or immutable remote
Expand All @@ -831,8 +837,10 @@ through the fail-closed workflow, and treats an already public release as
complete only after public verification. If publication became immutable before
its documentation dispatch completed, the same command reuses a healthy
correlated docs run or dispatches a new exact tag-and-digest-bound docs run. It
never moves a tag or overwrites mismatched public state. Add `--verbose-wait`
only when raw job output is useful.
also dispatches the identifier publisher again with the same released tag,
source commit, and catalog digest; that deployment is idempotent and performs
its live smoke before succeeding. It never moves a tag or overwrites mismatched
public state. Add `--verbose-wait` only when raw job output is useful.

For break-glass manual operation, omitting `--plan-output` from
`verify-candidate` still prints the three low-level tag, push, and dispatch
Expand Down Expand Up @@ -867,6 +875,13 @@ Publication:
The same workflow rebuilds `/dev/` on every push to protected `main` while
retaining the latest authenticated docs-bearing release at the canonical
and versioned routes.
8. Dispatches `registrystack-id` with the exact released tag, source commit,
and catalog SHA-256, waits for the correlated deployment, and requires the
complete live catalog smoke. The publisher verifies the tag target and main
ancestry, requires its reviewed committed bundle to match that source and
digest, retains older identifiers and digest-addressed artifacts, and deploys
only after its deterministic checks pass. It never imports unreviewed source
during deployment.

The candidate attestations bind the payload build, while the signed checksum
chain and checksum provenance authenticate the public inventory and its
Expand Down Expand Up @@ -922,6 +937,7 @@ workflow, and it adds no release gate.
| Candidate expires before the tag is pushed | Request and verify a new candidate |
| Bound draft or publication step fails while the candidate remains valid | Fix the workflow on protected `main` if needed, then rerun `registry-release publish --plan <candidate-plan.json> --wait` |
| Documentation dispatch or deployment fails after publication | Rerun `registry-release publish --plan <candidate-plan.json> --wait`; it verifies the immutable release and safely dispatches a new exact docs request when no healthy correlated run remains |
| Identifier dispatch, deployment, or live smoke fails after publication | Rerun `registry-release publish --plan <candidate-plan.json> --wait`; the already-public release takes the exact closeout path and republishes the same tag, source commit, and catalog digest. If the publisher bundle does not match, merge the reviewed synchronization for that exact released source before retrying. If it reports conflicting immutable bytes, stop and fix forward with a new release |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Implement the documented identifier recovery path

When identifier publication fails after the GitHub Release has become public, this prescribed rerun does not republish identifiers: publish_candidate_plan enters its already-published branch at release/scripts/registry-release:3574, verifies the public release, and only recovers documentation through line 3677, while dispatch_publication_run is reached only in the later unpublished branch. Thus an identifier dispatch, deployment, or smoke failure remains unrecoverable by the documented command; add explicit identifier recovery or redispatch the closeout workflow for this state.

Useful? React with 👍 / 👎.

| One final image tag already has the expected digest | Retry; publication accepts and re-verifies the exact digest |
| npm or PyPI already has every expected client byte | Retry; publication accepts and re-verifies the exact registry state |
| npm or PyPI has only an exact subset of the client packages | Retry; publication uploads only the absent exact packages |
Expand Down
48 changes: 48 additions & 0 deletions release/scripts/test_release_workflow_structure.py
Original file line number Diff line number Diff line change
Expand Up @@ -1246,6 +1246,7 @@ def test_is_a_manual_main_workflow_with_recoverable_jobs(self) -> None:
"publish_client_npm",
"publish_client_pypi",
"dispatch-docs",
"publish-identifiers",
],
)
self.assertEqual(
Expand Down Expand Up @@ -1636,6 +1637,53 @@ def test_dispatches_docs_for_current_release_candidates(self) -> None:
dispatch_run,
)

def test_dispatches_and_waits_for_exact_identifier_publication(self) -> None:
_, document = workflow("release.yml")
verify = step_run(
document,
"verify",
"Verify binding, candidate, and attestations",
)
self.assertIn(
"products/identifiers/generated/catalog.v1.json",
verify,
)
self.assertIn(
"identifier_catalog_sha256=${identifier_catalog_sha256}",
verify,
)
self.assertEqual(
document["jobs"]["verify"]["outputs"]["identifier_catalog_sha256"],
"${{ steps.candidate.outputs.identifier_catalog_sha256 }}",
)

publication = document["jobs"]["publish-identifiers"]
self.assertIn("needs.publish.result == 'success'", publication["if"])
self.assertIn(
"needs.closeout-published.result == 'success'",
publication["if"],
)
self.assertEqual(
publication["needs"],
["verify", "publish", "closeout-published"],
)
run = step_run(
document,
"publish-identifiers",
"Dispatch and verify exact identifier publication",
)
self.assertIn("registrystack/registrystack-id", run)
self.assertIn(
'-f "source_sha=${{ needs.verify.outputs.source_sha }}"',
run,
)
self.assertIn(
'-f "catalog_sha256=${{ needs.verify.outputs.identifier_catalog_sha256 }}"',
run,
)
self.assertIn('gh run watch "${target_run_id}"', run)
self.assertIn("--exit-status", run)

def test_promotes_exact_client_packages_with_oidc_and_retry_safety(self) -> None:
text, document = workflow("release.yml")
npm = document["jobs"]["publish_client_npm"]
Expand Down
Loading