Skip to content

Feat/web UI sboms - #1620

Draft
dervoeti wants to merge 8 commits into
mainfrom
feat/web-ui-sboms
Draft

Feat/web UI sboms#1620
dervoeti wants to merge 8 commits into
mainfrom
feat/web-ui-sboms

Conversation

@dervoeti

Copy link
Copy Markdown
Member

Description

Follow-up to #1600, which covered the frontends that ship an npm build. This covers the remaining JavaScript in our products: the HBase, HDFS, Spark and Trino web UIs. None of it appeared in any SBOM, because the CycloneDX Maven plugin only sees Java dependencies.

Three different mechanisms are needed, because the JavaScript arrives three different ways.

HBase unpacks its web UI assets from webjars during the build, the CycloneDX Maven plugin does not pick them up. hbase_webapps_deps.py derives the coordinates from the poms and writes a package.json, which cdxgen then turns into the SBOM. A version bump or an added webjar is picked up automatically, and the script fails the build if the pom layout changes or a version cannot be resolved.

Hadoop and Spark vendor pre-built, usually minified files directly into their source trees. Those carry no package manifest and no lockfile, I did not find a tool that would identify a minified bundle reliably. They are therefore recorded by hand in a manifest per product version, which shared/sbom/vendored_js.py turns into a CycloneDX SBOM. I chose JSON manifests over writing the SBOM directly so we don't have to update all of them if we for example switch from CycloneDX 1.5 to 1.7. These are independent records of what we detected manually, not SBOMs.
Every entry pins the SHA-256 of the file it describes, so a product upgrade that changes, adds or removes a file fails the build instead of silently shipping a wrong SBOM. Recording a version is a one-time cost per file.
shared/sbom/identify_js.py is a helper script for creating these files. Its inspect subcommand lists every .js file below the given directories with its SHA-256 and any version string it states, and identify hashes every file of every published release of a candidate npm package to find a byte-identical match, which is proof rather than inference. File names can lie, for example Hadoop's d3-v4.1.1.min.js really is d3 4.1.0.

Trino gets three SBOMs, because its web UI is three separate things: the npm project behind the current UI, the npm project behind the preview UI, and the pre-built libraries under webapp/vendor. The two npm projects are handled by cdxgen, the pre-built binaries by the same tooling used for Hadoop / Spark.

The helper scripts and the vendored manifests were generated by Claude, I reviewed the helper scripts and iterated with Claude over the general concepts and we landed at this solution. I tested some builds and inspected the SBOMs, looks good.

Definition of Done Checklist

  • Changes are OpenShift compatible
  • All added packages (via microdnf or otherwise) have a comment on why they are added
  • Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
  • All packages should have (if available) signatures/hashes verified
  • Add an entry to the CHANGELOG.md file
  • Integration tests ran successfully

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant