perf(cicd): shallow checkouts for release build and prepare phases - #37177
perf(cicd): shallow checkouts for release build and prepare phases#37177wezell wants to merge 4 commits into
Conversation
fetch-depth: 0 fetches every branch ref (~2k+ branches, ~1.1 GB pack) which cost ~19-20 min per job in release runs (see run 32759840677: Prepare Release 22m of which ~20m was the fetch; Initial Artifact Build 34m of which ~19m was the fetch). - build-phase: the ref-based checkout only needs the ref snapshot. buildnumber-maven-plugin (scmRevision) reads HEAD only, which works on a shallow clone. The PR-check steps that need history stay on the fetch-depth: 0 no-ref path. - release-prepare: checkout HEAD of the default branch shallowly; when an explicit release commit is supplied, fetch just that commit (GitHub enables uploadpack.allowAnySHA1InWant) before reset --hard. Measured from a cold clone: shallow single-tag fetch ~77s vs ~19min in CI for the full fetch. Tarball snapshot alternative (~24s) was rejected because it loses .git and the scmRevision stamp falls back to UNKNOWN.
|
Claude finished @wezell's task in 1m 34s —— View job Code Review — shallow checkouts for release build & prepare phases
Reviewed the full diff ( Resolved
New IssuesNo new issues. I verified the two paths most worth scrutinizing in the new code:
All other changes ( LGTM. 🟢 · branch |
Same fix as the release phases, but multiplied: the test phase fans out to up to 26 jobs (setup-matrix + 25 suites from .github/test-matrix.yml), each doing a fetch-depth: 0 checkout that pulls every branch ref (~2k+ branches, ~1.1 GB pack, ~19-20 min per job). Nothing in these jobs reads git history: no git commands run after checkout, the maven-job action has no git usage, Sonar runs in a separate workflow, and buildnumber-maven-plugin (scmRevision) reads HEAD only, which works on a shallow clone.
…nd nightly setup Completes the audit of every fetch-depth: 0 in .github/: the semantics were misread — fetch-depth: 0 means UNLIMITED history for ALL refs (every branch, ~2k+ branches, ~1.1 GB pack, ~19-20 min per job), not 'no history'. - deployment-phase: no git commands after checkout -> fetch-depth: 1 - cicd_6-release bump-min-sdk-version: only needs HEAD of main (it does its own before checkout -B) -> fetch-depth: 1 - cicd_4-nightly setup: Find Build Commit walks git log on main back to midnight UTC (max observed ~19 commits/day on main) -> fetch-depth: 100 Left at fetch-depth: 0 intentionally: - cicd_comp_build-phase.yml no-ref path: diff --git a/.github/workflows/cicd_comp_build-phase.yml b/.github/workflows/cicd_comp_build-phase.yml index ef92201..3f0fe0d 100644 --- a/.github/workflows/cicd_comp_build-phase.yml +++ b/.github/workflows/cicd_comp_build-phase.yml @@ -91,12 +91,18 @@ jobs: if: inputs.ref == '' with: fetch-depth: 0 + # Shallow checkout (fetch-depth: 1): the build only needs the ref snapshot. + # fetch-depth: 0 pulled every branch ref (~2k+ branches, ~1.1 GB pack) and + # cost ~19-20 min per job. buildnumber-maven-plugin (scmRevision) only reads + # HEAD, which works fine on a shallow clone. The `run-pr-checks` steps that + # need real history (git diff origin/main...HEAD, git status) are only used + # by the inputs.ref == '' path below, which keeps fetch-depth: 0. - name: Checkout code with ref ${{ inputs.ref }} if: inputs.ref != '' uses: actions/checkout@v4 with: ref: ${{ inputs.ref }} - fetch-depth: 0 + fetch-depth: 1 # Check if .mvn/maven.config is modified in PR (only for PR checks) - name: Check if .mvn/maven.config is in the PR commit diff --git a/.github/workflows/cicd_comp_release-prepare-phase.yml b/.github/workflows/cicd_comp_release-prepare-phase.yml index dff9e11..484e5f4 100644 --- a/.github/workflows/cicd_comp_release-prepare-phase.yml +++ b/.github/workflows/cicd_comp_release-prepare-phase.yml @@ -93,10 +93,14 @@ jobs: env: GITHUB_CONTEXT: ${{ toJson(github) }} + # Shallow checkout (fetch-depth: 1): this job only needs HEAD of the default + # branch (or the explicitly supplied release commit, fetched below) to cut + # the release branch from. fetch-depth: 0 pulled every branch ref + # (~2k+ branches, ~1.1 GB pack) and cost ~20 min per release run. - name: Checkout core uses: actions/checkout@v4 with: - fetch-depth: 0 + fetch-depth: 1 token: ${{ secrets.CI_MACHINE_TOKEN || github.token }} - uses: ./.github/actions/core-cicd/cleanup-runner @@ -146,7 +150,15 @@ jobs: git push origin :refs/tags/${release_tag} fi - git reset --hard ${{ steps.set-version.outputs.release_commit }} + release_commit=${{ steps.set-version.outputs.release_commit }} + # Shallow checkout (fetch-depth: 1) only guarantees HEAD of the default + # branch. When an explicit release commit was supplied, fetch just that + # commit (GitHub enables uploadpack.allowAnySHA1InWant) before resetting. + if ! git cat-file -e "${release_commit}^{commit}" 2>/dev/null; then + echo "Release commit ${release_commit} not present locally, fetching it" + git fetch --depth=1 origin "${release_commit}" + fi + git reset --hard "${release_commit}" release_version=${{ steps.set-version.outputs.release_version }} release_branch=${{ steps.set-version.outputs.release_branch }} diff --git a/.github/workflows/cicd_comp_test-phase.yml b/.github/workflows/cicd_comp_test-phase.yml index 7c77510..01c020e 100644 --- a/.github/workflows/cicd_comp_test-phase.yml +++ b/.github/workflows/cicd_comp_test-phase.yml @@ -96,7 +96,9 @@ jobs: - name: Checkout code uses: actions/checkout@v4 with: - fetch-depth: 0 + # Shallow checkout: this job only reads .github/test-matrix.yml from HEAD. + # fetch-depth: 0 pulled every branch ref (~2k+ branches, ~1.1 GB pack). + fetch-depth: 1 - name: Parse test configuration id: parse-config @@ -249,7 +251,12 @@ jobs: - name: Checkout code uses: actions/checkout@v4 with: - fetch-depth: 0 + # Shallow checkout: test jobs run Maven suites against the checked-out + # snapshot only — no git history is read anywhere in this job or in the + # maven-job action (buildnumber-maven-plugin reads HEAD, which works on + # a shallow clone). fetch-depth: 0 pulled every branch ref (~2k+ + # branches, ~1.1 GB pack) at ~19-20 min per matrix job. + fetch-depth: 1 # The libvips image engine (IMAGE_API_USE_LIBVIPS) is exercised by VipsParityTest. # Install native libvips on the JVM unit-test runner so those tests run instead of needs the merge base with main (PR checks) - ai_claude-backend-reviewer.yml: agent runs git diff/log/show between base and head SHAs - legacy-release_maven-release-process.yml: deprecated workflow (follow-up tracked in #37178)
✅ Validated via Manual Deploy run 32769863759Dispatched -8 Manual Deploy with Use workflow from: Build / Initial Artifact Build: ✅ success — the ref-path checkout with Measured fetch time (ref-path checkout, this branch)
Savings ~13 min per build job. Note the ~77s figure in the PR description was measured from a fast local network; GitHub-hosted runners get ~1 MB/s to the git backend, so ~6 min for the ~360 MB shallow snapshot is the realistic number. The remaining ~6 min is pure snapshot transfer — irreducible without the tarball approach (loses |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
dotbot flagged two items: 1. [P2] build-phase ref checkout depth was keyed on the wrong condition. The PR-check steps (.mvn/maven.config guard, clean-tree check) are gated on run-pr-checks, not ref == '' as the comment claimed. Drive depth by run-pr-checks so a future caller passing both ref and run-pr-checks gets full history. Uses string literals: GH expressions treat 0 as falsy, so the naive run-pr-checks && 0 || 1 collapses to 1 in the true branch. 2. [P3] nightly setup bounded depth (100) could theoretically miss the midnight commit. Raised to 1000 (~52 days of volume at the ~19/day max). Rejected dotbot's suggested fetch-depth: 0 - that regresses the nightly setup job to a ~19-20 min full fetch for a once-daily schedule when a comfortably-bounded fetch is equally correct in practice.
👀 Backend reviewer (dotbot) findings addresseddotbot (deepseek-v4-pro-0813 + glm-latest) reviewed the PR and reported 2 findings. Assessment and resolution: 1️⃣ [P2] build-phase ref-checkout depth keyed on the wrong condition — fixedThe comment claimed the PR-check steps were gated on One correction to dotbot's own suggested fix, though: its proposed expression 2️⃣ [P3] nightly setup bounded depth could miss the midnight commit — hardened
Both changes committed as |
|
dotbot code review:
The incremental diff directly addresses the prior comments: the build-phase ref checkout now drives fetch-depth from inputs.run-pr-checks using truthy string literals ('0'/'1'), which works correctly in GitHub expressions, and the nightly setup checkout's bounded depth is raised to 1000 with an accurate explanation. No current caller combines ref and run-pr-checks, and the new changes introduce no live bug. Tip: comment with "/dotbot address comments" to attempt automated fixes for unresolved review threads. reviewed by dotbot · deepseek/deepseek-v4-pro-0813 · medium |
|
dotbot code review:
The prior finding is fixed: fetch-depth was raised from 100 to 1000, providing ample headroom over the observed ~19 commits/day. The build-phase fix also correctly gates depth on run-pr-checks using truthy string literals. No remaining issues. Tip: comment with "/dotbot address comments" to attempt automated fixes for unresolved review threads. reviewed by dotbot · ~z-ai/glm-latest · medium |
2 participants
Closes #37178
Problem
Release runs spend most of their wall-clock time in
git fetch, not building. From run 32759840677 (Release 26.08.24-01):Root cause: every checkout uses
fetch-depth: 0, which fetches all refs — all ~2,100+ branches plus full history (~1.1 GB pack). The build fetch log even shows it downloading unrelated branches (e.g.revert-pubsub-checkout) that have nothing to do with the release tag.Changes
cicd_comp_build-phase.yml— the ref-based checkout (release tag, LTS tag, manual-deploy ref, nightly ref) now usesfetch-depth: 1. The build only needs the ref snapshot;buildnumber-maven-plugin(scmRevision) reads HEAD only, which works fine on a shallow clone. The no-ref path used by PR checks (git diff origin/main...HEAD,git statusclean-tree check) keepsfetch-depth: 0, so PR workflows are unaffected.cicd_comp_release-prepare-phase.yml— shallow checkout of the default branch HEAD. When an explicitrelease_commitis supplied and isn't already present locally, it is fetched withgit fetch --depth=1 origin <sha>(GitHub enablesuploadpack.allowAnySHA1InWant) beforegit reset --hard.cicd_comp_test-phase.yml— the same fix, multiplied: the test phase fans out to up to 26 jobs (setup-matrix + 25 suites from.github/test-matrix.yml), each doing a fullfetch-depth: 0checkout (~19-20 min per job). Nothing in these jobs reads git history (no git commands after checkout, the maven-job action has no git usage, Sonar runs in a separate workflow), so both checkouts are nowfetch-depth: 1. This speeds up PR, merge-queue, trunk, and nightly test cycles.cicd_comp_deployment-phase.yml,cicd_6-release.yml(bump-min-sdk-version), andcicd_4-nightly.yml(setup) — completes the audit of everyfetch-depth: 0in.github/. The semantics were misread:fetch-depth: 0means unlimited history for all refs (every one of ~2,100+ branches, ~1.1 GB pack), not "no history". Deployment and the SDK bump only need the snapshot/HEAD of main (fetch-depth: 1); nightly's setup walksgit logon main back to midnight UTC, so it gets a boundedfetch-depth: 100(max observed on main is ~19 commits/day). Two spots intentionally keepfetch-depth: 0because they genuinely read history: the build-phase PR path (git diff origin/main...HEADmerge base) andai_claude-backend-reviewer.yml(agent runs git diff/log/show between base and head SHAs). The deprecatedlegacy-release_maven-release-process.ymlis left untouched.Measurements (cold clone, tag
v26.08.24-01)fetch-depth: 0)The tarball was rejected: no
.gitmeansscmRevisionsilently falls back toUNKNOWNin the shipped artifacts.Expected savings: ~38 of the ~66 min profiled in the release run.
Validation
buildnumber-maven-plugin(works shallow), PR-check steps (gated off,run-pr-checks: false), docker/npm scripts (no git usage)ref: release, LTS, java-variant, manual-deploy, CLI, nightly — all haverun-pr-checks: falseFollow-ups (not in this PR)
bump-min-sdk-versionis fixed by this PR. Correction onCheck Changed Files: it does not pay the fetch tax — for PRsdorny/paths-filteruses the GitHub REST API for changed files (~1s) and its checkout is already shallow by default (~13s); the 2m26s seen in the release run was runner-queue time (release also passeschange-detection: 'disabled', skipping those steps entirely)maven-jobaction declares aneeds-historyinput that is never consumed (vestigial)