fix(workflows): reject a condition that has no {{ }} block - #4182
fix(workflows): reject a condition that has no {{ }} block#4182ntdatt812 wants to merge 2 commits into
Conversation
`evaluate_condition` resolves its argument through `evaluate_expression`,
which only substitutes `{{ ... }}` blocks. A string with no such block
comes back unchanged and — unless it reads `true`/`false` — is then
coerced by `bool()`. So a condition authored without the braces is never
evaluated at all:
evaluate_condition("inputs.count > 100", ctx) -> True
evaluate_condition("{{ inputs.count > 100 }}", ctx) -> False
with `inputs.count == 5` in both cases. An `if` step always takes `then`,
and a `while`/`do-while` step always runs to `max_iterations` — ten agent
invocations for a loop the author expected to stop.
This is the same silent-truthiness authoring mistake the three step
validators already reject for a list/dict/number condition, and it is
easier to make: GitHub Actions accepts a bare expression in `if:`, so the
brace-less form is a habit to bring here.
Adds `condition_is_never_evaluated()` and wires it into the `if`,
`while` and `do-while` validators, so the mistake surfaces at validation
with the corrected form spelled out. Boolean literals, real bools, empty
strings and any string containing `{{` stay valid — runtime behaviour is
unchanged.
There was a problem hiding this comment.
Pull request overview
Adds validation to reject brace-less workflow conditions that would otherwise always evaluate as true.
Changes:
- Adds a shared condition-validation helper.
- Integrates validation into
if,while, anddo-whilesteps. - Adds runtime and validator regression tests.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/workflows/expressions.py |
Adds condition detection helper. |
src/specify_cli/workflows/steps/if_then/__init__.py |
Validates if conditions. |
src/specify_cli/workflows/steps/while_loop/__init__.py |
Validates while conditions. |
src/specify_cli/workflows/steps/do_while/__init__.py |
Validates do-while conditions. |
tests/unit/test_condition_expression_block.py |
Adds regression coverage. |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 5/5 changed files
- Comments generated: 4
- Review effort level: Balanced
| stripped = condition.strip() | ||
| if not stripped or stripped.lower() in ("true", "false"): | ||
| return False | ||
| return "{{" not in stripped |
| f"If step {config.get('id', '?')!r}: 'condition' " | ||
| f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never " | ||
| "evaluated and is always true. Wrap the expression: " | ||
| '"{{ ' + str(config["condition"]).strip() + ' }}".' |
| f"While step {config.get('id', '?')!r}: 'condition' " | ||
| f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never " | ||
| "evaluated and is always true. Wrap the expression: " | ||
| '"{{ ' + str(config["condition"]).strip() + ' }}".' |
| f"Do-while step {config.get('id', '?')!r}: 'condition' " | ||
| f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never " | ||
| "evaluated and is always true. Wrap the expression: " | ||
| '"{{ ' + str(config["condition"]).strip() + ' }}".' |
Two gaps in the condition validator, both raised in review.
An opening `{{` with no `}}` after it is never substituted either:
_interpolate_expressions takes its `raw_close == -1` branch and appends
the tail verbatim. So `condition: "{{ inputs.count > 100"` -- and the
reversed `"}} inputs.count > 100 {{"`, whose only `{{` is last -- come
back unchanged and are coerced to true exactly like a brace-less string.
The helper now looks for a complete block rather than an opening one.
The suggested correction was interpolated into a double-quoted scalar,
so a condition containing a double quote produced YAML that does not
parse: `condition: "{{ inputs.name == "zzz" }}"` raises a ParserError.
format_condition_correction() now picks the quoting from the content and
drops a stray delimiter instead of nesting a second one, so the message
stays paste-ready. All three validators share it.
Tests: 30 more cases -- the incomplete forms, and a YAML round trip over
conditions holding single quotes, double quotes, both, and backslashes,
asserting each correction loads back exactly and is not re-flagged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Both review points were correct — thanks. Fixed in c0c291c. 1. Unterminated / reversed delimiters slipped through Confirmed against the interpolator rather than assumed.
The helper now requires a complete block: it finds the first 2. The correction was not valid YAML Reproduced exactly as described — Added
Tests — the file goes from 40 to 70 cases. The new ones cover the incomplete forms, and run every correction through
|
| # An opening ``{{`` with no ``}}`` anywhere after it is never substituted | ||
| # either: ``_interpolate_expressions`` takes its ``raw_close == -1`` branch | ||
| # and appends the tail verbatim. So ``{{ inputs.count > 100`` -- and the | ||
| # reversed ``}} inputs.count > 100 {{``, whose only ``{{`` is last -- come | ||
| # back unchanged and are just as silently true as a brace-less string. | ||
| return stripped.find("}}", open_at + 2) == -1 |
| if '"' not in wrapped and "\\" not in wrapped: | ||
| return '"' + wrapped + '"' | ||
| if "'" not in wrapped: | ||
| return "'" + wrapped + "'" | ||
| return '"' + wrapped.replace("\\", "\\\\").replace('"', '\\"') + '"' |
mnriem
left a comment
There was a problem hiding this comment.
Please address Copilot feedback
Defect
evaluate_conditionresolves its argument throughevaluate_expression, which only substitutes{{ ... }}blocks. A string with no such block comes back unchanged, and — unless it readstrue/false— is then coerced bybool(). So a condition authored without the braces is never evaluated at all.With
inputs.count == 5:Every brace-less condition is true, whatever it says. An
ifstep always takesthen; awhile/do-whilestep never terminates on its condition and runs tomax_iterations— ten agent invocations for a loop the author expected to stop after one.Nothing reports it. The workflow validates, runs, and takes the wrong branch silently.
Why this is worth a validation error
The three step validators already reject a list/dict/number condition, and the comment there states the reason exactly:
A brace-less string is the same failure mode, and a likelier mistake: GitHub Actions accepts a bare expression in
if:(if: github.event_name == 'push'), so an author arriving from Actions writes the brace-less form by habit — and unlike[1, 2],condition: inputs.count > 100looks completely correct on the page.Fix
condition_is_never_evaluated()inexpressions.py, wired into theif,whileanddo-whilevalidators. The error names the problem and hands back the corrected form:Validation only — no runtime behaviour changes. Still valid, and covered by tests:
"{{ ... }}"in any position,"true"/"false"in any case, realbools, empty and whitespace strings, and every non-string type (already handled by the branch above).Tests
New
tests/unit/test_condition_expression_block.py, 40 cases:FalsevsTrue— so the defect stays documented even if the validator changesThe same 22 pre-existing failures in both runs — all
symlink_toon Windows without the privilege (OSError: [WinError 1314]), unrelated to this change. Everything added is the +40 new tests.