test(sbom): slim down the SBOM e2e suite - #348
Draft
reyreavman wants to merge 2 commits into
Draft
reyreavman wants to merge 2 commits into
reyreavman wants to merge 2 commits into
Conversation
…ries Every table in the SBOM e2e suite carried four backend entries: two active (vanilla-docker, buildkit-docker) and two XEntry ones for Buildah that ran nowhere, so CI reported 180 specs where 113 existed. All fixtures in the suite are Stapel and DOCKER_BUILDKIT only reaches the Dockerfile build path, so the buildkit entry executed the same code as the vanilla one. Tables left with a single entry become plain It blocks, the multi-platform specs gated on Buildah alone are gone, and their registry helpers that other specs still use moved to helpers_test.go. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
The sbom validate specs only run the CLI against local CycloneDX fixtures, yet they lived in the SBOM suite and so demanded WERF_TEST_K8S_DOCKER_REGISTRY, buildah on linux and a werf host purge after every one of the 30 specs. They move to their own suite, which requires docker alone — the validation itself runs in the sbom-checker container — so the specs now run against a plain checkout with no registry set up. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
reyreavman
force-pushed
the
test/sbom/slim-down-e2e-suite
branch
from
September 23, 2026 06:46
0a20146 to
3b20186
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The SBOM e2e suite reported 180 specs where 113 existed, and ran every scenario twice on two container backends that execute the same code. It now declares 44 build-driven specs, plus 30 validate specs in a separate suite that needs no container registry.
What
XEntryfornative-chroot/native-rootlessis gone (67 of them across 22 files). They were Pending in every CI job, so no coverage changes.buildkit-dockerentry is gone (39 of them). All fixtures undertest/e2e/sbom/_fixturesare Stapel;DOCKER_BUILDKITis read only by the docker CLIBuildKitEnabled()and only affects the Dockerfile build path, which a Stapel image on the docker backend never takes —DockerServerBackend.HasStapelBuildSupport()returns false andBuildStapelStagepanics there. Both entries drove the identical code path.setupEnvOptions/sbomTestOptionsand the backend switch insetupSbomBuildEnvare gone; the helper now pinsWERF_BUILDAH_MODE=dockerand unsetsDOCKER_BUILDKIT.Itblocks. The two tables that keep a real axis (--ispras-formatcontainer/oss) stayDescribeTable.multiplatform_test.gois removed: both of its tables consisted exclusively of BuildahXEntrys. The registry helperssigning_multiplatform_test.gostill uses moved tohelpers_test.go;injectFactoryPmFiles,mustExtractCycloneDXBOMandcontainerFactoryImageRefhad no remaining callers and are deleted. Per-platform SBOM artifacts, their in-toto subjects and the per-platform cache hit stay covered bysigning_multiplatform_test.go.validate_test.goand its 22 JSON fixtures move totest/e2e/sbom-validate. That suite requires onlydocker—sbom validateruns the sbom-checker container — so it no longer demandsWERF_TEST_K8S_DOCKER_REGISTRY,buildahon linux, or awerf host purgeafter each of its 30 specs. Labels are unchanged, so CI keeps running them ine2e_simple.test/e2e/sbom-validatepasses 30/30 withWERF_TEST_K8S_DOCKER_REGISTRYunset.test/e2e/sbomand the new suite changes: no workflow, no Taskfile, no sharedtest/pkghelper.Why
The backend matrix was inherited from the Dockerfile-based build suites, where
DOCKER_BUILDKITdoes select a different builder. In a Stapel-only suite it bought a second, identical run of every scenario, while the one backend that genuinely differs — Buildah — was disabled in every entry. Deleting the dead axis rather than keeping a token representative pair keeps the suite honest about what it exercises: re-enabling Buildah is a separate change that has to make SBOM generation work there first.