test(sbom): build e2e fixtures straight from the base-images builders

[reyreavman]

Summary

Every SBOM e2e fixture now builds from: the published base-images builders — builder/distroless (registry.deckhouse.io/container-factory@sha256:7aac8d…) and builder/golang (@sha256:ef3bd2…) — which carry their own SBOM. The per-fixture Dockerfile.builder-base, the io.deckhouse.internal.builder label and the WERF_E2E_ALLOW_LOCAL_BUILDER_IMAGES escape hatch in the tests are gone. Language runtimes come from os-pm packages entries; yarn/pnpm/poetry/uv are bootstrapped by a preceding npm/pip entry plus packages[].manager, the workflow #318 was built for. Along the way the python-pip directive switches its default binary from pip to pip3.

Kaiten: 68939510.

What

werf

• packages: [{type: python-pip}] without manager runs python3 -m pip install --no-cache-dir -r <spec>; before it ran pip install …. The pip console script is not named consistently across images — the base-images python package ships pip3/pip3.12 only, other images ship only pip — and invoking the module through the interpreter sidesteps the script name. An explicit manager: still overrides the whole invocation. Docs (usage/build/stapel/instructions.md, en+ru) updated. The manager validator is untouched (a bare manager: pip is still rejected, as before this PR).
• VERIFIED: default python-pip on the base-images python (pip3 only) — inject/pip_simple, inject/poetry_simple, inject/uv_simple all reach Successfully installed …; and on a pip-only image (python:3.12-slim with pip3 renamed to pip in shell.beforeInstall) — Successfully installed … requests-2.32.3. python3 exists on every image checked: python:3.12-slim, python:3.12-alpine, python:3.13-slim, python3-pip on debian:bookworm-slim and alpine:3.20, container-factory python==3.12.12.
• Images checked for the pip script name: python:3.12-slim, python:3.12-alpine, python3-pip on debian:bookworm-slim, ubuntu:24.04, alpine:3.20, fedora:41 ship both pip and pip3; container-factory python ships pip3 only.

Fixtures

• No Dockerfile.builder-base, BUILDER_BASE_IMAGE, buildTrustedBuilderBase or WERF_E2E_ALLOW_LOCAL_BUILDER_IMAGES remain under test/e2e/sbom; 32 werf-giterminism.yaml that only allowed that env var are removed.
• go fixtures (gomod_replace, stage_deps_gomod, type_change/state1) build from: builder/golang; every other fixture from: builder/distroless. type_change switches the base between state0 and state1.
• pip_simple: os-pm python==3.12.12 instead of pm install in the builder. poetry, uv, pnpm, yarn_manager unchanged in shape: runtime via os-pm, tool via a tools/ npm/pip entry and manager:.
• yarn_simple removed: without the builder's npm install -g yarn it became byte-identical to yarn_manager (fixture and spec).
• no_pm_binary deletes /usr/local/bin/pm in shell.beforeInstall and still fails with pm: command not found, exit 127 (VERIFIED by hand-run build).
• Versions and hashes follow the v3.0.2 package index: yq==4.53.6 (4.48.1 is not in the index), component hashes for curl/openssl/jq/yq, and containerfactoryversion=v3.0.2 in the PURL assertions. PACKAGES_VERSION/REGISTRY still reach the from: scratch fixtures as build secrets.
• lua stays at 5.3.6 (5.4.8 in the same index is AVX-512 and dies with Illegal instruction on the runners).
• VERIFIED on CI (run 35120999662): the base-image SBOM fetch from registry.deckhouse.io succeeds; e2e_simple passes all 103 sbom specs. Locally the sha256-<digest> SBOM index of both builders still returns 404 from outside.

Why

The card asks to stop stamping third-party images with the internal builder label. The label existed only because the container-factory image had no SBOM attached; base-images v3.0.2 publishes builder/distroless and builder/golang with one, so the fixtures can use them as any user would. Keeping the label pattern would have left the e2e suite exercising a code path no production build takes.

pip → python3 -m pip: the python package in base-images installs only pip3/pip3.12, so the old default pip failed there with pip: not found. Switching the default to pip3 instead would have broken the mirror case — an image that ships only pip — with no configuration-level escape hatch, since the manager validator accepts only a path from a preceding packages entry (manager: pip and manager: /usr/bin/pip3 are both rejected, by design of #318). python3 -m pip depends on the interpreter name rather than the script name and works on both kinds of image; the generated command stays flat, so env and manager behave exactly as for every other ecosystem.

[reyreavman]

Verification (HEAD 909c3e7)

• task format, task build, task lint (0 issues), task test:unit (72 suites, passed) on the rebased branch.
• pkg/config + pkg/build/stage unit tests updated for the pip3 default and pass.
• Hand-run werf build (linux/amd64 under Docker Desktop, local registry, mock external-refs server, WERF_E2E_ALLOW_LOCAL_BUILDER_IMAGES=true to get past the base-SBOM fetch) against the reshaped fixtures: pip_simple app/packages runs pip3 install and resolves requests==2.32.3; poetry_simple and uv_simple packages stage completes; type_change/state1 go build completes on builder/golang; no_pm_binary fails in app/packages with pm: command not found, exit 127.
• Every name==version in the fixtures resolves in the v3.0.2 index (crane export …/index:v3.0.2), and the asserted SHA-256 hashes are that index's digest fields.
• NOT verified locally: the base-image SBOM fetch itself. registry.deckhouse.io/container-factory answers 404 for tag sha256-7aac8d… and sha256-ef3bd2… (the release job log shows the copy into registry-write, and dev-registry has both). Per the card owner the artifacts are in place for CI; the e2e_simple job is the check.
• Mutation: none run; assertion changes are hash/version pins whose source is the published index.

Follow-ups

• e2e_complex / e2e_extra (test/e2e/build) were red on main too in previous runs; they do not touch this diff.

[reyreavman]

CI note: phantom unit failure in ./pkg/build

The unit job on this branch fails in ~5 of 7 attempts with

Build Suite - 154/154 specs … SUCCESS!
There were failures detected in the following suites:
  build ./pkg/build

— no [FAIL], no Summarizing, no timeout. Same signature today on fix/sbom/canonical-merge (runs 35107622958, 35104411549, 35070162569) and on main's test:daily for ./pkg/sbom/externalref, so it is not this diff (which touches pkg/config and test strings only; pkg/build passes 5/5 locally).

Mechanism (traced in ginkgo internal/run.go:runParallel): a suite is marked failed from the per-proc exit status; if a proc posts suite-did-end (SUCCESS) and then exits non-zero, ginkgo prints exactly this and drops the proc output. pkg/build mixes 19 plain TestXxx with the Ginkgo TestBuild; under -p each of 15 procs also runs all plain tests and their --- FAIL is swallowed. Most timing-sensitive candidate: graph_scheduler_integration_test.go (30 ms chain vs 150 ms "slow" image, real parallel.DoTasksDynamic).

Not fixed here (out of scope). Cheap mitigation for a follow-up: run the CI unit step with --output-interceptor-mode=none or -v so the plain-test failure line surfaces, and/or move those plain tests under Ginkgo.

[reyreavman]

CI (run 35120999662, attempt 6, HEAD 48098ac — after the runner disk fix)

All jobs green: unit, lint, integration_main, integration_git, e2e_simple, e2e_complex, e2e_extra, docs. e2e_simple carries every sbom spec: 103/103 passed, including go-mod, cargo, purl-resolver and the base-image SBOM fetch from registry.deckhouse.io.

Earlier attempts of this run failed on the runner (disk exhaustion → no space left on device in cargo, syft scan of the 1 GB builder/golang stage dying) and on the pre-existing ./pkg/build phantom unit failure described above; neither reproduced after the runner fix.

[Fral738]

The description's workaround for images that provide pip but not pip3 (manager: pip) is rejected by configuration validation. A single python-pip entry with workdir: /app and manager: pip makes werf config render fail with:

invalid manager "pip" for type "python-pip": no preceding packages entry installs it

The manager validator requires the manager path to be inside a preceding non-os-pm entry's workdir; it is not an unrestricted override for an executable already on the base image's PATH. Please remove the manager: pip recommendation and replace it with a tested migration path, or preserve compatible default selection for pip-only images. The description should not present the default change as having a configuration workaround that the parser rejects.

[reyreavman]

@Fral738 confirmed — manager: pip was rejected exactly as you showed; the description promised an escape hatch the parser did not have.

Root cause is in the validator, not the description: a bare name was rejected as "resolved by the image instead of the configuration", yet the ecosystem default (pip3, yarn, npm) is resolved by the image PATH in precisely the same way. Rejecting a bare name added no guarantee over the default — it only prevented picking a differently named executable.

f7bc549: validatePackagesManagers now accepts a bare executable name (no /) as-is; paths keep the existing rule (inside the workdir of a preceding entry, pinned by its lock file). Unit entries for python-pip/manager: pip and javascript-yarn/manager: yarnpkg added to the accepting tables — both fail on the previous validator. Docs en+ru updated in both places that stated the old rule.

Live check on a genuinely pip-only image (python:3.12-slim, pip3 renamed to pip in shell.beforeInstall — note that on stock python:* images pip is a symlink to pip3, so deleting pip3 alone is not a pip-only image):

packages:
  - type: python-pip
    workdir: /app
    manager: pip

werf config render accepts it; werf build → Building stage app/packages (9.06 seconds), Successfully installed certifi-… requests-2.32.3 urllib3-2.8.0.

Also checked which real images would need this at all: python:3.12-slim, python:3.12-alpine, python3-pip on debian/ubuntu/alpine/fedora all ship both pip and pip3. PR description updated accordingly.

[reyreavman]

Update: f7bc549 (accepting a bare manager name) is reverted in b47f5d8 — loosening the #318 validator is not the right place to solve this, and the previous comment's reasoning is withdrawn.

The default itself is changed instead, 6f8569d: python-pip without manager now runs python3 -m pip install --no-cache-dir -r <spec>. The pip console script is what is named inconsistently across images (pip3/pip3.12 only on the base-images python, pip only on some others); the interpreter name is not. So there is no pip-only vs pip3-only case to escape from, and the validator stays exactly as it is.

Verified:

• base-images python (pip3 only): inject/pip_simple, inject/poetry_simple, inject/uv_simple reach Successfully installed …;
• pip-only image (python:3.12-slim, pip3 renamed to pip): Successfully installed … requests-2.32.3, no manager needed;
• python3 present with a working pip module on python:3.12-slim, python:3.12-alpine, python:3.13-slim, python3-pip on debian:bookworm-slim / alpine:3.20, container-factory python==3.12.12.

Docs en+ru and the PR description updated; the manager: pip recommendation is gone.