Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 42 additions & 4 deletions .github/workflows/batch-sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@ name: Batch Sync
# Security: dispatch inputs reach run: scripts only via env — `${{ }}`
# interpolation inside run: lets a crafted input inject shell.
#
# Secrets: GHA_PAT_BASIC needs contents:write on ethersphere/batch-archive;
# PRIVATE_GNOSIS_RPC_URL is required; GNOSIS_RPC_USER / GNOSIS_RPC_PASSWORD
# are needed only off-allowlist.
# Secrets: BEE_RUNNER_APP_ID / BEE_RUNNER_KEY mint a short-lived App token
# scoped to ethersphere/batch-archive for the publish push (the App is already
# installed org-wide with contents:write); PRIVATE_GNOSIS_RPC_URL is required;
# GNOSIS_RPC_USER / GNOSIS_RPC_PASSWORD are needed only off-allowlist.

on:
workflow_dispatch:
Expand Down Expand Up @@ -63,6 +64,20 @@ jobs:
timeout-minutes: 120

steps:
# Preflight: the real token is minted after the export (see below), which
# would push an App misconfiguration — lost contents:write, key no longer
# shared with this repo — to the end of a run that can take two hours.
# Minting once up front fails in seconds instead. This token is never
# used; create-github-app-token revokes it in its post step.
- name: Verify batch-archive App access
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.BEE_RUNNER_APP_ID }}
private-key: ${{ secrets.BEE_RUNNER_KEY }}
owner: ethersphere
repositories: batch-archive
permission-contents: write

- name: Checkout
uses: actions/checkout@v5

Expand All @@ -82,7 +97,11 @@ jobs:
uses: actions/checkout@v5
with:
repository: ethersphere/batch-archive
token: ${{ secrets.GHA_PAT_BASIC }}
# Public repo: the default token can read it. Write happens at the end
# with an App token, so no long-lived credential sits in .git/config
# for the length of the export.
token: ${{ github.token }}
persist-credentials: false
path: batch-archive
fetch-depth: 0

Expand Down Expand Up @@ -151,9 +170,23 @@ jobs:
--slim=true \
--verbosity "${VERBOSITY}"

# Minted here, not at checkout: an App token lives ~1h and the export
# above can outrun that. permission-contents keeps it to what the push
# needs, and repositories keeps it off every other repo in the org.
- name: Generate token for batch-archive

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minting the token here rather than at checkout makes sense given the ~1h token lifetime, but it does move credential failures to the end of the run. Previously a bad credential failed at Checkout batch-archive within seconds — that is how the empty GHA_PAT_BASIC got caught quickly. Now, if the App loses contents:write or the key visibility changes, it surfaces after an export that can run up to the full 120-minute timeout.

That is worse than just lost time: snapshot.ndjson.gzip only exists in the runner workspace, and the next run's git clean -ffdx wipes it, so the retry re-exports everything from scratch.

Two cheap options if you want to keep the late mint:

  • a pre-flight before Export that mints and discards a token — it validates the installation and the permission in a couple of seconds;
  • an actions/upload-artifact with if: failure() so a failed publish does not cost the whole export.

Worth noting the same loss applies to a non-fast-forward rejection on the final push.

id: archive-token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.BEE_RUNNER_APP_ID }}
private-key: ${{ secrets.BEE_RUNNER_KEY }}
owner: ethersphere
repositories: batch-archive
permission-contents: write

- name: Publish to batch-archive
env:
TRIGGERED_BY: ${{ github.actor }}
ARCHIVE_TOKEN: ${{ steps.archive-token.outputs.token }}
run: |
set -euo pipefail
cp snapshot.ndjson.gzip batch-archive/archive/export.ndjson.gzip
Expand Down Expand Up @@ -193,5 +226,10 @@ jobs:
git tag "${new_tag}"
# --atomic: a rejected push to main rejects the tag too, so no
# orphaned tag can become a later run's resume point.
# A token in the remote URL lands in git's argv, readable via
# `ps -eo args` by anything else on these long-lived shared runners.
# A credential helper reads it from the environment instead; the
# helper written to .git/config holds no secret itself.
git config credential.helper '!f() { echo username=x-access-token; echo "password=${ARCHIVE_TOKEN}"; }; f'
git push --atomic origin HEAD:main "refs/tags/${new_tag}"
echo "::notice::published snapshot at block ${last_block} as ${new_tag} (resumed from ${ARCHIVE_TAG})"
Loading