Skip to content

fix(workflows): reject a condition that has no {{ }} block - #4182

Open
ntdatt812 wants to merge 2 commits into
github:mainfrom
ntdatt812:fix/condition-without-expression-block
Open

fix(workflows): reject a condition that has no {{ }} block#4182
ntdatt812 wants to merge 2 commits into
github:mainfrom
ntdatt812:fix/condition-without-expression-block

Conversation

@ntdatt812

Copy link
Copy Markdown

Defect

evaluate_condition resolves its argument through evaluate_expression, which only substitutes {{ ... }} blocks. A string with no such block comes back unchanged, and — unless it reads true/false — is then coerced by bool(). So a condition authored without the braces is never evaluated at all.

With inputs.count == 5:

evaluate_condition("{{ inputs.count > 100 }}", ctx)   # False  — correct
evaluate_condition("inputs.count > 100", ctx)         # True   — never evaluated
evaluate_condition("inputs.name == 'zzz'", ctx)       # True
evaluate_condition("inputs.count < 3", ctx)           # True

Every brace-less condition is true, whatever it says. An if step always takes then; a while/do-while step never terminates on its condition and runs to max_iterations — ten agent invocations for a loop the author expected to stop after one.

Nothing reports it. The workflow validates, runs, and takes the wrong branch silently.

Why this is worth a validation error

The three step validators already reject a list/dict/number condition, and the comment there states the reason exactly:

a list/dict/number condition silently resolves to its truthiness (e.g. condition: [1, 2] is always True) with no error, branching wrongly on an authoring mistake. Reject those at validation […]

A brace-less string is the same failure mode, and a likelier mistake: GitHub Actions accepts a bare expression in if: (if: github.event_name == 'push'), so an author arriving from Actions writes the brace-less form by habit — and unlike [1, 2], condition: inputs.count > 100 looks completely correct on the page.

Fix

condition_is_never_evaluated() in expressions.py, wired into the if, while and do-while validators. The error names the problem and hands back the corrected form:

If step 's1': 'condition' 'inputs.count > 100' has no '{{ }}' block, so it is
never evaluated and is always true. Wrap the expression: "{{ inputs.count > 100 }}".

Validation only — no runtime behaviour changes. Still valid, and covered by tests: "{{ ... }}" in any position, "true"/"false" in any case, real bools, empty and whitespace strings, and every non-string type (already handled by the branch above).

Tests

New tests/unit/test_condition_expression_block.py, 40 cases:

  • the runtime behaviour is pinned first — same expression with and without braces, asserting False vs True — so the defect stays documented even if the validator changes
  • each of the three step validators rejects the brace-less form and echoes the corrected expression
  • no false positives, parametrised across all three step types
  • the helper itself across strings, bools, containers and numbers
tests/unit + tests/test_workflows.py

before:  22 failed, 1083 passed, 9 skipped
after:   22 failed, 1123 passed, 9 skipped

The same 22 pre-existing failures in both runs — all symlink_to on Windows without the privilege (OSError: [WinError 1314]), unrelated to this change. Everything added is the +40 new tests.

`evaluate_condition` resolves its argument through `evaluate_expression`,
which only substitutes `{{ ... }}` blocks. A string with no such block
comes back unchanged and — unless it reads `true`/`false` — is then
coerced by `bool()`. So a condition authored without the braces is never
evaluated at all:

    evaluate_condition("inputs.count > 100", ctx)      -> True
    evaluate_condition("{{ inputs.count > 100 }}", ctx) -> False

with `inputs.count == 5` in both cases. An `if` step always takes `then`,
and a `while`/`do-while` step always runs to `max_iterations` — ten agent
invocations for a loop the author expected to stop.

This is the same silent-truthiness authoring mistake the three step
validators already reject for a list/dict/number condition, and it is
easier to make: GitHub Actions accepts a bare expression in `if:`, so the
brace-less form is a habit to bring here.

Adds `condition_is_never_evaluated()` and wires it into the `if`,
`while` and `do-while` validators, so the mistake surfaces at validation
with the corrected form spelled out. Boolean literals, real bools, empty
strings and any string containing `{{` stay valid — runtime behaviour is
unchanged.
@ntdatt812
ntdatt812 requested a review from mnriem as a code owner August 18, 2026 10:04
@mnriem
mnriem requested a balanced review from Copilot August 18, 2026 12:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds validation to reject brace-less workflow conditions that would otherwise always evaluate as true.

Changes:

  • Adds a shared condition-validation helper.
  • Integrates validation into if, while, and do-while steps.
  • Adds runtime and validator regression tests.
Show a summary per file
File Description
src/specify_cli/workflows/expressions.py Adds condition detection helper.
src/specify_cli/workflows/steps/if_then/__init__.py Validates if conditions.
src/specify_cli/workflows/steps/while_loop/__init__.py Validates while conditions.
src/specify_cli/workflows/steps/do_while/__init__.py Validates do-while conditions.
tests/unit/test_condition_expression_block.py Adds regression coverage.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 5/5 changed files
  • Comments generated: 4
  • Review effort level: Balanced

stripped = condition.strip()
if not stripped or stripped.lower() in ("true", "false"):
return False
return "{{" not in stripped
f"If step {config.get('id', '?')!r}: 'condition' "
f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never "
"evaluated and is always true. Wrap the expression: "
'"{{ ' + str(config["condition"]).strip() + ' }}".'
f"While step {config.get('id', '?')!r}: 'condition' "
f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never "
"evaluated and is always true. Wrap the expression: "
'"{{ ' + str(config["condition"]).strip() + ' }}".'
f"Do-while step {config.get('id', '?')!r}: 'condition' "
f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never "
"evaluated and is always true. Wrap the expression: "
'"{{ ' + str(config["condition"]).strip() + ' }}".'
Two gaps in the condition validator, both raised in review.

An opening `{{` with no `}}` after it is never substituted either:
_interpolate_expressions takes its `raw_close == -1` branch and appends
the tail verbatim. So `condition: "{{ inputs.count > 100"` -- and the
reversed `"}} inputs.count > 100 {{"`, whose only `{{` is last -- come
back unchanged and are coerced to true exactly like a brace-less string.
The helper now looks for a complete block rather than an opening one.

The suggested correction was interpolated into a double-quoted scalar,
so a condition containing a double quote produced YAML that does not
parse: `condition: "{{ inputs.name == "zzz" }}"` raises a ParserError.
format_condition_correction() now picks the quoting from the content and
drops a stray delimiter instead of nesting a second one, so the message
stays paste-ready. All three validators share it.

Tests: 30 more cases -- the incomplete forms, and a YAML round trip over
conditions holding single quotes, double quotes, both, and backslashes,
asserting each correction loads back exactly and is not re-flagged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ntdatt812

Copy link
Copy Markdown
Author

Both review points were correct — thanks. Fixed in c0c291c.

1. Unterminated / reversed delimiters slipped through

Confirmed against the interpolator rather than assumed. _interpolate_expressions() takes its raw_close == -1 branch and appends the tail verbatim, so nothing is substituted:

condition evaluate_condition old helper new helper
inputs.count > 100 True flags flags
{{ inputs.count > 100 True misses flags
}} inputs.count > 100 {{ True misses flags
{{ inputs.count > 100 }} False

The helper now requires a complete block: it finds the first {{ and checks a }} follows it. {{ a }} {{ b }} and {{ inputs.text | default('}}') }} stay unflagged.

2. The correction was not valid YAML

Reproduced exactly as described — condition: "{{ inputs.name == "zzz" }}" raises yaml.parser.ParserError. A remediation the author cannot paste is not a remediation.

Added format_condition_correction() in expressions.py, used by all three validators as suggested. It picks quoting from the content — double by default, single when the expression holds a double quote, double with backslash escapes when it holds both — and drops a stray delimiter instead of nesting a second one, so {{ inputs.count > 100 corrects to "{{ inputs.count > 100 }}" rather than "{{ {{ ... }} }}".

condition emitted correction
inputs.name == "zzz" '{{ inputs.name == "zzz" }}'
inputs.name == 'zzz' "{{ inputs.name == 'zzz' }}"
inputs.a == "x" and inputs.b == 'y' "{{ inputs.a == \"x\" and inputs.b == 'y' }}"
inputs.path == 'C:\tmp' "{{ inputs.path == 'C:\tmp' }}"

Tests — the file goes from 40 to 70 cases. The new ones cover the incomplete forms, and run every correction through yaml.safe_load asserting it loads back to exactly {{ <expr> }} and that the corrected form is not itself re-flagged — the invariant that would have caught both of these.

uvx ruff@0.15.0 check src tests clean; pytest tests/unit 250 passed (the 2 symlink failures are a local Windows privilege limitation, green on CI).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 5/5 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment on lines +722 to +727
# An opening ``{{`` with no ``}}`` anywhere after it is never substituted
# either: ``_interpolate_expressions`` takes its ``raw_close == -1`` branch
# and appends the tail verbatim. So ``{{ inputs.count > 100`` -- and the
# reversed ``}} inputs.count > 100 {{``, whose only ``{{`` is last -- come
# back unchanged and are just as silently true as a brace-less string.
return stripped.find("}}", open_at + 2) == -1
Comment on lines +748 to +752
if '"' not in wrapped and "\\" not in wrapped:
return '"' + wrapped + '"'
if "'" not in wrapped:
return "'" + wrapped + "'"
return '"' + wrapped.replace("\\", "\\\\").replace('"', '\\"') + '"'

@mnriem mnriem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address Copilot feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants