fix(eval): surface actionable message on target output key mismatch

[Chibionos]

Milo hit this in the FUSION-workshop coding-agent authoring path (MST-14878): an LLM judge evaluator had targetOutputKey: reason, but the data point's expectedOutput only had decision and route. Normal authoring miss. The run dumped a raw Python traceback ending in KeyError: 'reason' instead of a readable eval error.

output_evaluator.py already wraps that KeyError in a UiPathEvaluationError with TARGET_OUTPUT_KEY_NOT_FOUND, but str(exc) only returned detail, and detail was f"Error: {e}" plus the full traceback appended. track_evaluation_metrics writes str(e) into the evaluator's details field, so the traceback is exactly what landed in the run results.

I changed str(exc) to return f"{title}: {detail}" and left the traceback on error_info.detail for logs/support. The four TARGET_OUTPUT_KEY_NOT_FOUND messages now name the missing key(s) and list the keys that were actually present, instead of just echoing str(KeyError).

What to look at

__str__ lives on the shared UiPathEvaluationError type, not just this code path. Every evaluation error that used to render as a bare detail (often traceback and all) now renders as title: detail. I think that's the right default everywhere, but it's a behavior change beyond the one evaluator that reported this.

Testing

• pytest tests/evaluators/ — full suite green
• Extended test_scalar_key_missing_in_actual_raises and test_scalar_key_missing_in_expected_raises to assert the surfaced message names the missing key and the available keys, and contains no Traceback
• ruff check ., ruff format --check ., mypy on both changed modules — clean

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[Copilot]

🟢 Approval recommended

The changes are targeted, consistent with existing error handling patterns, and include test assertions that verify the intended user-facing message improvements.

Pull request overview

This PR improves the UiPath Python SDK evaluation UX by making UiPathEvaluationError render as a concise, actionable message (title + detail) instead of often surfacing raw tracebacks in evaluator result “details”, and enhances TARGET_OUTPUT_KEY_NOT_FOUND errors to explicitly describe missing target output keys and available keys.

Changes:

• Adjusted UiPathEvaluationError construction so str(exc) is "{title}: {detail}", while preserving tracebacks in error_info.detail for support/logging.
• Added a helper in output_evaluator.py to generate actionable “missing key(s) + available keys” detail messages for target output key lookup failures.
• Extended evaluator tests to assert the surfaced details include missing/available keys and do not include “Traceback”.

File summaries

File	Description
packages/uipath/src/uipath/eval/models/models.py	Changes UiPathEvaluationError string rendering to include title + detail while keeping traceback in structured error info.
packages/uipath/src/uipath/eval/evaluators/output_evaluator.py	Produces clearer TARGET_OUTPUT_KEY_NOT_FOUND details including requested keys and available top-level keys.
packages/uipath/tests/evaluators/test_evaluator_methods.py	Adds assertions validating key names appear in error details and that tracebacks are not surfaced.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud