MINIFICPP-2892 Tensor extensions based on sonos/tract

[martinzink]

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• Has your PR been rebased against the latest commit within the target branch (typically main)?

• Is your initial contribution a single, squashed commit?

For code changes:

• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE file?
• If applicable, have you updated the NOTICE file?

For documentation related changes:

• Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

This PR introduces a new minifi_tensor Rust extension (based on sonos/tract) that adds image-to-tensor preprocessing and Tract-based inference processors/services, and updates the build packaging to support it.

Changes:

• Added minifi_tensor extension with a Tract-backed controller service and processors for classification, detection, tensor preprocessing, and post-processing.
• Enhanced build tooling (Dockerfiles + CI build script) to build with all features and leverage GitHub Actions buildx cache.
• Updated extension registration macro to support #[cfg(...)] / attributes on declared processors/controllers.

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
minifi_rust/minifi_rs_behave/linux_build.sh	Enables buildx GHA caching for faster CI builds.
minifi_rust/minifi_rs_behave/Dockerfile.debian	Updates Debian base and builds Rust artifacts with --all-features.
minifi_rust/minifi_rs_behave/Dockerfile.alpine	Builds Rust artifacts with --all-features.
minifi_rust/minifi_rs_behave/Cargo.toml	Removes playground dependency from behave crate.
minifi_rust/minifi_native/src/lib.rs	Extends declare_minifi_extension! to allow optional attributes per processor/controller.
minifi_rust/extensions/minifi_tensor/src/utils/tensor_helpers.rs	Adds tensor deserialization helpers and image loading helpers.
minifi_rust/extensions/minifi_tensor/src/utils/score_activation.rs	Adds ScoreActivation enum for classification/detection post-processing.
minifi_rust/extensions/minifi_tensor/src/utils/per_channel_f32.rs	Adds per-channel float parsing for mean/std normalization properties.
minifi_rust/extensions/minifi_tensor/src/utils/mod.rs	Wires up the new utils module tree.
minifi_rust/extensions/minifi_tensor/src/utils/dimensions.rs	Adds helper struct for image/original/target dimensions.
minifi_rust/extensions/minifi_tensor/src/utils/bounding_box.rs	Adds bounding-box model, NMS/IoU, drawing, and property parsing.
minifi_rust/extensions/minifi_tensor/src/services/tract_model_service/service_definition.rs	Defines properties and definition for TractModelService.
minifi_rust/extensions/minifi_tensor/src/services/tract_model_service.rs	Implements Tract model loading and inference execution.
minifi_rust/extensions/minifi_tensor/src/services/mod.rs	Wires up the new services module tree.
minifi_rust/extensions/minifi_tensor/src/processors/mod.rs	Wires up the new processor modules.
minifi_rust/extensions/minifi_tensor/src/processors/draw_bounding_box.rs	Adds processor to draw bounding boxes and output an annotated PNG.
minifi_rust/extensions/minifi_tensor/src/processors/detect_object/detect_object_def.rs	Defines DetectObject processor properties/relationships.
minifi_rust/extensions/minifi_tensor/src/processors/detect_object.rs	Implements DetectObject as a composed processor pipeline.
minifi_rust/extensions/minifi_tensor/src/processors/classify_image/classify_object_def.rs	Defines ClassifyImage processor properties/relationships.
minifi_rust/extensions/minifi_tensor/src/processors/classify_image.rs	Implements ClassifyImage as a composed processor pipeline.
minifi_rust/extensions/minifi_tensor/src/low_level_processors/mod.rs	Adds low-level processor modules (some behind a feature flag).
minifi_rust/extensions/minifi_tensor/src/low_level_processors/invoke_tract_model/invoke_tract_model_def.rs	Defines InvokeTractModel processor properties/relationships.
minifi_rust/extensions/minifi_tensor/src/low_level_processors/invoke_tract_model.rs	Implements low-level InvokeTractModel processor.
minifi_rust/extensions/minifi_tensor/src/low_level_processors/image_to_tensor/image_to_tensor_def.rs	Defines ImageToTensor processor properties/relationships.
minifi_rust/extensions/minifi_tensor/src/low_level_processors/image_to_tensor.rs	Implements image preprocessing into model-ready tensors.
minifi_rust/extensions/minifi_tensor/src/low_level_processors/filter_bounding_boxes/filter_bounding_boxes_def.rs	Defines FilterBoundingBoxes processor properties/relationships.
minifi_rust/extensions/minifi_tensor/src/low_level_processors/filter_bounding_boxes.rs	Implements detection post-processing (activation, decode, NMS, output).
minifi_rust/extensions/minifi_tensor/src/low_level_processors/classify_output/classify_output_def.rs	Defines ClassifyOutput processor properties/relationships.
minifi_rust/extensions/minifi_tensor/src/low_level_processors/classify_output.rs	Implements classification post-processing (activation, Top-K, labels).
minifi_rust/extensions/minifi_tensor/src/lib.rs	Registers new processors and controller service in the extension.
minifi_rust/extensions/minifi_tensor/minifi_tensor.md	Adds user-facing documentation for processors/services.
minifi_rust/extensions/minifi_tensor/features/steps/steps.py	Wires behave step imports for the extension’s feature tests.
minifi_rust/extensions/minifi_tensor/features/resources/.gitignore	Prevents committed test-download artifacts.
minifi_rust/extensions/minifi_tensor/features/image_to_tensor.feature	Adds behave tests for ImageToTensor scenarios.
minifi_rust/extensions/minifi_tensor/features/environment.py	Adds behave hook to download models/images/labels on first run.
minifi_rust/extensions/minifi_tensor/features/detection.feature	Adds behave face-detection scenarios (pipeline + DetectObject).
minifi_rust/extensions/minifi_tensor/features/classification.feature	Adds behave classification scenarios (pipeline + ClassifyImage).
minifi_rust/extensions/minifi_tensor/features/basic.feature	Adds behave smoke test for extension registration.
minifi_rust/extensions/minifi_tensor/Cargo.toml	New crate manifest for the minifi_tensor extension.
minifi_rust/extensions/minifi_rs_playground/minifi_rs_playground.md	Updates playground docs (adds dummy controller service row).
.github/references/ubuntu_22_04_clang_arm_manifest.json	Updates component manifest reference to include minifi_tensor.

Suppressed comments (3)

minifi_rust/extensions/minifi_tensor/src/utils/tensor_helpers.rs:1

• deserialize_tensors can panic on &flow_file_contents[cursor..cursor + tensor_len] if attributes are inconsistent with the payload length (truncated content or malformed tensor.{i}.bytes). This should be validated and return a MinifiError instead of panicking: check cursor + tensor_len <= flow_file_contents.len() before slicing (and consider also rejecting extra trailing bytes after the last tensor if that’s not allowed).
  minifi_rust/extensions/minifi_tensor/src/utils/tensor_helpers.rs:1
• deserialize_tensors introduces new parsing logic (length/shape/dtype attributes + payload slicing) but has no unit tests in this module, while other helpers here are tested. Adding tests for: (1) valid multi-tensor payload, (2) truncated payload (should error, not panic), and (3) invalid dtype/shape strings would prevent regressions.
  minifi_rust/extensions/minifi_tensor/src/utils/bounding_box.rs:1
• draw_onto casts potentially user-supplied normalized coordinates (via JSON in DrawBoundingBox) directly to u32. Negative values or NaNs will wrap/produce unexpected large coordinates and can lead to invalid rectangles (and as i32 conversions inside drawing) with unpredictable behavior. Clamp/sanitize x_min/y_min/x_max/y_max to finite values and to [0.0, 1.0] (and ensure min <= max) before converting to pixel coordinates.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 42 out of 42 changed files in this pull request and generated 5 comments.

Suppressed comments (3)

minifi_rust/extensions/minifi_tensor/src/utils/tensor_helpers.rs:1

• deserialize_tensors can panic on truncated/malformed payloads because it slices flow_file_contents[cursor..cursor + tensor_len] without verifying bounds. Please add an explicit check that cursor + tensor_len <= flow_file_contents.len() (and ideally that the final cursor equals the payload length, or at least does not exceed it) and return a MinifiError instead of panicking.
  minifi_rust/extensions/minifi_tensor/src/utils/bounding_box.rs:1
• IoU can divide by zero if either box has zero area (or if both areas cancel such that the union is 0), yielding NaN/inf and destabilizing NMS. Add an explicit guard for union_area <= 0.0 (or non-finite) and return 0.0 in that case.
  minifi_rust/minifi_rs_behave/linux_build.sh:1
• This uses bash arrays (CACHE_ARGS=(), CACHE_ARGS+=(...), and later ${CACHE_ARGS[@]}). If linux_build.sh is executed under sh (or a non-bash shell), it will fail. Ensure the script’s shebang is bash (e.g. #!/usr/bin/env bash) and that CI invokes it accordingly, or rework CACHE_ARGS to be POSIX-shell compatible.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[martinzink]

Couldnt allow the actions yet because it needs to be enabled in apache org, it still needs this actions

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - name: Expose GitHub Actions cache to buildx
        uses: crazy-max/ghaction-github-runtime@v3