diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 51ae363e2..7e3968e61 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -593,3 +593,18 @@ extracting `OffsetExpression`/`LimitExpression`. | `DESCRIBE PAGE` output that **used to parse stops parsing** after a fix that restores a WHERE clause — reported against stock `Administration.Account_New`, which every project from the Mendix template carries | Two older gaps the restored clause collided with. The constraint was emitted **raw** (`expr += " where " + xpath`), so a stored XPath carrying an inner predicate — `System.grantableRoles[reversed()]/…` — ended the MDL `expression` parse at the `[`. The grammar takes either a bare expression or a bracketed `xpathConstraint`, and only the second is safe for arbitrary XPath. The same six lines also stripped the outer brackets by testing the first and last byte, turning `[a][b]` into `a][b`, and guarded on `strings.Trim(s, "")` — an empty cutset trims nothing, so a whitespace-only constraint emitted a bare `where` | `mdl/executor/cmd_pages_describe_datasource.go` (`xpathConstraintClause`) | **Emit the bracketed production always, and stop deciding.** The bare form reads better but parses only for simple comparisons, and "is this XPath also a valid MDL expression" is not a question an emitter can answer — a bracketed group is accepted for every constraint. Reuse `visitor.SplitXPathPredicateGroups` (the quote- and nesting-aware splitter written for #772) rather than keeping a second, simpler copy: the naive strip is exactly the bug that helper exists to prevent. **Measure the round trip as a SET, not a count.** The claim this regression contradicted came from counting failing pages before and after, seeing 10 both times, and concluding nothing had changed — but an equal count hides a page flipping each way, and here a template page flipped from parse to fail while the total held. `join` the pass-sets: `join <(grep PASS before) <(grep FAIL after)` names the regressions directly. Control: revert the bracketing and `Account_New` stops parsing again. Reported in mxcli-formula1 §57.1/§57.3 | | A widget datasource bound to a microflow **loses its arguments** through `describe`: `microflow Mod.DS(Race: $Race)` describes as `microflow Mod.DS`, and replaying that description fails the build with **CE1571** "No argument has been selected for parameter 'Race' and no default is available" — the very error #835 fixed on the write side. `check --references` calls the argument-less form valid, so nothing catches it before mxbuild | The write side stores the mappings correctly; **no describe path ever read them**. Not confined to pluggable widgets — a plain `dataview` loses them too. Older than #941, which merely printed the whole line wrong so the loss hid behind a larger bug | `mdl/executor/cmd_pages_describe_datasource.go` (`flowSourceArgs`, and the `case "microflow", "nanoflow"` renderer) | Storage puts the bindings in `Settings.ParameterMappings`, each `{Parameter, Expression}` behind typed-array marker 3, with **Parameter qualified by the flow** (`Mod.DS_Filtered.Term`) while MDL names the parameter alone — the flow is already left of the parentheses. **Strip that prefix by matching the flow's own qualified name, not by cutting at the last dot**: a document storing the name bare would otherwise lose its first segment. The value is in `Expression` for both a variable ref and a literal; `Variable` is the older spelling and is honoured when present. A parameterless flow must render **without** parentheses — the grammar makes the list optional and empty parens would churn every existing description. **Get the BSON shape from a real unit, not from the writers**: the two serializers here disagree in spelling (gen's `SetParameterQualifiedName` vs legacy's `Parameter`), and only a dump settles which key is on disk. Verify by replaying a description and counting CE1571 (4 → 0 on a two-widget page), not by eyeballing the text. Reported in mxcli-formula1 §57.2; same family as §39, where `SHOW_PAGE` drops `(Race: $currentObject)` | | `use building block X (datasource: …)` starts failing with "datasource override supplied, but the block has no datasource widget to rebind" — on both engines, for a stock Atlas block, immediately after a fix that made DESCRIBE stop emitting malformed datasources | The rebind expands the block by **re-rendering it to MDL through the describe emitter and re-parsing that text**, then looked for a widget already carrying a `DataSource` property. A reusable block is a *template*: its datasource is unbound. That used to render as the malformed `DataSource: database from ,` — the exact output #941 fixed — and the re-parse turned it into a `DataSource` property, so the rebind found its target. Emitting nothing for an empty datasource, which is correct, left it with nothing to match | `mdl/executor/cmd_pages_builder_v3.go` (`isDataSourceWidget`, and the `DataSourceOverride` branch) | Match the target by **widget type — what can carry a datasource, not what does**. The sibling `ActionOverride` branch already did exactly this, and its comment says why: Atlas blocks "ship placeholder buttons with no action". Same fact about the same blocks; the datasource branch just hadn't learned it. **The generalisable shape**: a consumer that re-parses another component's *rendered output* silently depends on that output's bugs, so fixing a renderer can break something that never reads the model. Grep for callers of the emitter (`outputWidgetMDLV3` here) before changing what it emits — unit tests on the emitter cannot see this, and the failure surfaced only in the integration suite. Control: restore the property-based predicate and the stock-block rebind fails again. Tests `mdl/executor/cmd_pages_builder_bb_rebind_test.go`; CI repro `mdl-examples/doctype-tests/15c-fragment-bindings-examples.mdl` | +| A microflow that reads `if Module.SomeRule(param = $x) then` is written with **no Condition at all** — `mx check` reports **CE0080** "The 'Condition' property is required." naming the Decision, whose caption still shows the call. `DESCRIBE` renders it as `if true then`, so the round trip looks like valid MDL, and re-applying an identical `CREATE OR MODIFY` over a Studio Pro-authored microflow turns a clean project into a broken one | Two defects, one behind the other. (a) `splitConditionToGen` handled only `ExpressionSplitCondition` and fell through to `default: return nil`; the caller then skipped `SetSplitCondition` entirely. The READ side had been implemented for #723, and that asymmetry is what made it silent. (b) `modelsdk/gen` binds `Microflows$RuleCall.Rule` to BSON key `Rule`, where Mendix stores **`Microflow`** (rules share the microflow namespace) — so fixing (a) alone writes a reference Mendix never reads and produces the same CE0080 for a different reason | `mdl/backend/modelsdk/microflow_write.go` (`splitConditionToGen`, new `ruleCallToGen`, the `Microflows$RuleCall` marker registration); `modelsdk/gen/microflows/types.go` (`initRuleCall` + `RuleCall.InitFromRaw` — STORAGE-NAME OVERRIDE on both sides); strike the row off `modelsdk/gen/keyaudit_test.go`; tests `mdl/backend/modelsdk/microflow_rule_split_write_test.go`; repro `mdl-examples/bug-tests/939-rule-split-condition.mdl` | **"Works on Mendix 10, fails on 11" was an ENGINE difference, not a version one** — same script, same project, `MXCLI_ENGINE=legacy` is 0 errors and the default (modelsdk) is CE0080. The microflow write path branches only on `major >= 10` / `<= 9`, so 10.24 and 11.12 take identical code; check the engine before believing a version story. **A round-trip test cannot see a wrong storage key** — it passes whichever key is used as long as both sides agree, which is exactly the shape of (b) — so assert the KEY on the encoded document, in a separate test with its own control. **Reconstruct the missing document type when the fixture does not exist**: MDL cannot author rules and no module in a blank Mendix 11 app ships one, so the repro rewrote an mxcli-created microflow's unit as `Microflows$Rule` (keeping only the ten properties `initRule` declares) — `mx check` 0 errors on the synthetic rule is the control that it is a valid stand-in. Verify the fix against the legacy engine's BSON, not just against a green build: the two documents are identical modulo key order. Issue mendixlabs/mxcli#939 | +| `mxcli check -p` reports **"Unexpected token after expression — … possible missing space between keywords"** with an EMPTY location and a hint about glued keywords like `emptyor`, on a script that parses fine without `-p` and executes correctly. Related: `declare $b Boolean = Mod.Rule(...)` passes check and is **CE0117** "Error(s) in expression" on mxbuild, and `show callers of` a rule finds nothing | Three separate gaps around the one construct MDL spells as `Module.Name(...)`. (a) `mdl/exprcheck` does not model a qualified call, so the trailing `(` was left on the stream and reported as leftover — it fired on the VALID decision form as much as the invalid ones, so it carried no signal and named a typo that was not there. (b) A Mendix expression has no user-callable functions at all — its library is built-in and unqualified — so a qualified call in a VALUE position is CE0117 whether it names a rule, a microflow or a Java action, and both engines wrote the literal text. (c) A rule is not an activity: Mendix evaluates one only as a decision's condition, and the reference extractor walked only action activities, so a rule called from a decision had no callers | `mdl/exprcheck/parser.go` (`parseQualifiedCall`, reached from `parseIdentLed`) + `ast.go` (`CallExpr.Qualified`) + `unknown_funcs.go` (skip qualified); `mdl/executor/validate_qualified_call.go` (MDL066) wired from `validate_microflow.go`; `mdl/executor/cmd_microflows_builder.go` (`buildSplitCondition` refuses a non-rule); `mdl/catalog/builder_references.go` (`collectRuleCalls`); tests `mdl/exprcheck/qualified_call_test.go`, `mdl/executor/validate_qualified_call_test.go`, `mdl/catalog/builder_rule_refs_test.go` | **A diagnostic that fires on the correct form too is worse than none** — it trains the author to ignore the checker, and here it also masked the real defect in the same script. **Exempt only the position that is actually legal**: the decision condition is the single place a bare qualified call is valid MDL (mxcli stores it as a `RuleSplitCondition`), so MDL066 skips `if` and nothing else — a `while` condition has no rule-split form and is flagged. **Split the check by what it needs**: whether a qualified call is illegal *here* is project-less (no built-in Mendix expression function has a dot in its name), while whether the name in a decision is a RULE needs the backend — so the first is a linter rule and the second is a flow-builder refusal, each with its own control. **Do not let the new node reach the unknown-function checker**: `UnknownFunctionCalls` feeds MDL044's did-you-mean, and a qualified name has no near built-in, so it would emit nonsense on the legal form; `CallExpr.Qualified` marks it and the walker skips it. Not fixed: rules are still not catalog OBJECTS, so a microflow called only from inside a rule's body is reported dead by QUAL004 / GRAPH_DEAD_ASSETS — adding the object type without also indexing rule bodies would report every rule itself as dead, so it needs both halves at once. Issue mendixlabs/mxcli#939 | +| `mxcli test`: (a) a `.test.mdl` file whose first test is preceded by a **file-level `/** … */` header** reports one test fewer — the first one vanishes with no pass, no fail and no error, and every later test renumbers over it; the workaround (a `--` header) re-triggers it the moment that header's **prose spells out `/**` and `*/`**. (b) `@expect count($Var) = N` on a bare `retrieve` block reported **PASS unconditionally** on v0.18.0, for any N, against an empty table | Two independent "substring instead of structure" defects in `cmd/mxcli/testrunner`. (a) `extractDocAndBody` took the FIRST `/**`…`*/` in a `/`-separated chunk as the test's doc comment. A header is not separated from the test below it by a `/`, so both are one chunk: the header became the doc, carried no `@test`, and the chunk — real test included — was skipped. The delimiters were found by `strings.Index` with no notion of comment or string state, which is why writing them inside a `--` comment recreated the fusion. Test IDs were numbered by **chunk index**, so a skipped chunk left a gap (`test_2` as a file's only test) and the line number came from searching the whole file for the chunk's first 20 bytes (wrong chunk when two start alike; **panics** on a chunk shorter than that). (b) `count()` is not a Mendix expression *function* — counting a list is an Aggregate list **activity** — so `ParseExpect` could not compile it. Pre-0bf9382 that dropped the annotation, and a test with no assertions passes as long as its body does not throw | `cmd/mxcli/testrunner/parser.go` (`splitTestBlocks`, `extractDocAndBody`, `scanDocComments`, `skipMDLString`), `expect.go` (`expectAggregate`, `parseAggregate`, `collectAggregates`), `generator_endpoint.go` (`writeExpectAggregates`), `generator.go` (`renameExpect`) | Scan the chunk (whitespace / `--` / `/* */` / `'…'`) instead of searching it: the doc is the **last** comment of the leading run, the body starts after it, and the line travels with the chunk from `splitTestBlocks`. Number tests, not chunks. **Two `@test` docs in one chunk is refused**, not resolved — either resolution runs one and drops the other, which is the bug. For (b), hoist the aggregate into the activity the author would write (`$mxtest_count_X = COUNT($X);`) ahead of the decision, and refuse `sum`/`average`/`minimum`/`maximum` (they need an attribute an assertion cannot supply) with the helper-microflow workaround named. In the monolithic runner the generated variable must be renamed per test like the body's own, or two tests counting same-named lists declare it twice. **Verify (b) at the Mendix layer, not the parser's**: `mxcli check` accepts plenty that mxbuild rejects — the generated flows were exec'd into a real 11.13.0 project and `mx check` run with a pristine-copy baseline (1 pre-existing error either side). **Controls**: pre-fix binary on the repro file finds 2 of 4 tests; stubbing `isListAggregate` to false restores "count() is not a Mendix expression function". Repro `mdl-examples/bug-tests/927-test-mdl-leading-comment-and-count.test.mdl`. Issue #927 | +| A `.test.mdl` test gains an assertion, a cleanup strategy or an expected error **nobody wrote**, because a sentence in its doc comment quotes the annotation — writing "`@expect $x = 1` in a sentence" gives the test that assertion, and "@cleanup none would apply here" changes its cleanup strategy. The invented assertion usually fails to compile, so the test reports an ERROR whose message quotes prose | The annotation patterns in `parseAnnotations` were unanchored (`@expect\s+(.+)`), so they matched anywhere in a line rather than at its start. An annotation is a javadoc **tag**, and a tag opens its line | `cmd/mxcli/testrunner/parser.go` (the six patterns in the `var (…)` block) | Anchor every pattern with `^`. The leading `*` and its indentation are already stripped before they run, so the real spellings — at any javadoc indentation, and the one-line `/** @test x */` form — are unaffected; keep a control asserting that, or a fix that simply stops reading annotations passes. Found while writing the #927 repro file: its header explained the bug, and explaining it created an assertion that then errored against a test nobody had written. **Generalisable**: the sibling of #927's own root cause — a delimiter means nothing outside the structure it delimits, whether that structure is a comment or a line | +| A test carrying both `@throws` and `@expect` reports making two assertions and makes one: the `@expect` is never evaluated, silently. It cannot fail, whatever it claims | `@throws` selects a different generated shape in **both** generators (`writeThrowsFlowBody` / `writeThrowsTestBlock`, where the verdict starts as a failure and only the error handler clears it) and neither emits the `@expect` checks — but `AssertionCount` still counted them. Same silent-absence class as the dropped `@expect` and the unimplemented `@verify`, in the one combination where the assertion cannot be made to work at all: the body was expected not to produce a result to assert on | `cmd/mxcli/testrunner/parser.go` (`checkThrowsExpect`, called from `parseAnnotations` beside `checkVerifyCleanup`) | Move the expects to `AssertionErrors` and clear them, so the test is an ERROR and gets no microflow — the annotation pipeline's fail-closed rule. Keep the `@throws` itself: it is the assertion that still stands. **Prove it end to end, not on the struct**: the visible symptom is `AssertionCount` and the generated MDL, so the test parses a file, checks the count, and greps the generated flow for the microflow's absence. Sibling of the `@verify`-under-rollback refusal, which is the precedent for the message shape (name the annotation, name the one-line change) | +| `ALTER WORKFLOW` over `--mcp` inserts a `CALL MICROFLOW` and Studio Pro shows **CE0495 "Duplicate name"** on the workflow, while `mxcli`'s own read-back looks fine and the statement reported success | A `CALL MICROFLOW` activity is named after its target microflow (`buildCallMicroflowTask`: `task.Name = n.Microflow.Name`), so two calls to the same microflow collide before anything inspects them. The file backends deduplicate in `wfmutator`; the MCP backend had no equivalent and sent the derived name verbatim. Two separate gaps, both needed: PED **accepts** the duplicate with `SUCCESS` and does **not** auto-rename (measured live), and `mcpWorkflowMutator.Save()` early-returned when only activity ops had run, so `ped_check_errors` — the only thing that sees a consistency error — was never called. `CREATE`/`CREATE OR REPLACE` are **not** affected: the executor dedups at `cmd_workflows_write.go` for every backend | `mdl/backend/wfnames/` (shared policy); `mdl/backend/mcp/workflow.go` (`resolve`/`searchActivities` collect names on the walk, `Save`/`checkSettled`); `mdl/backend/wfmutator/mutator.go` (`serializeAndDedup`, `buildSubFlowBson`) | Deduplicate **before the add** — PED refuses `set` on an activity's `/name` (`Element type does not support renaming`), so there is no repair after the fact. Collect the taken-set on the walk `resolve` already does, or the fix costs a PED round-trip per level. **Use a taken-set, not a seen-count**: counting re-collides the moment the workflow already contains the suffixed name (`A, A, A_2` counts its way to `A, A_2, A_2`). Uniqueness is workflow-**wide**, so recurse into the inserted activity's own outcome and boundary-event sub-flows. **`ped_update_document` reporting `SUCCESS` proves nothing about consistency** — it reports op-level failures only; `ped_check_errors` is the only signal, and it reads Studio Pro's **background** error list, which lags the write in BOTH directions (26 samples at a 20ms poll interval: error becomes visible 77-115ms after the write, stops being shown 75-128ms after the fix; one symmetric debounce, no growth under load). Asked too early it answers "No errors found." for a document just broken **and** still reports an error a just-applied op has already cleared — measured against an immediate check as the control, 6 trials each: immediate missed the real error 6/6 and falsely failed on a cleared transient 6/6, settled 0/6 on both. `Backend.pedCheckDocument` owns the pacing for every call site (delay, then re-ask while clean) so a new write path cannot forget it. **Poll at a fine interval when measuring a debounce** — a first pass at 50-100ms reported 170-350ms and "grows under load", both pure granularity artifacts. Issue #945 | +| A `SHOW_PAGE` widget action **silently ignores an argument that is not the context object**: `action: show_page Mod.Detail(Car: $Other)` inside a data view bound to `$Car` opens the page with **`$Car`**. Nothing reports it — `mx check` is **0 errors**, the build is green, and `describe page` prints `(Car: $currentObject)`, so the description looks like a *diagnosis* of a lost mapping rather than an accurate report of a model that never held one | mxcli stores a widget show-page action with an **empty** `ParameterMappings` array and lets Mendix infer the argument from the enclosing widget's context object. That part is deliberate and twice-confirmed — an explicit `Forms$PageParameterMapping` whose `Argument` is `$currentObject` is rejected as **CE0115** "parameters do not match" (#296, re-confirmed on mxbuild 11.12.1 for §56). The missing half was the case where the author names something *else*: the builder built `PageClientParameterMapping` objects and both engines dropped them on the floor (`formSettingsToGen` takes only the page name; the legacy writer hardcodes `bson.A{int32(2)}`) | `mdl/executor/cmd_pages_showpage_args.go` (new: `pageArgumentBindsContextObject`, `contextVarFor`, `validateShowPageArguments`); the guard in `cmd_pages_builder_v3.go`'s `case "showPage"`; `contextVarName` threaded through `cmd_pages_builder.go`, `cmd_pages_builder_v3_widgets.go`, `widget_engine.go`, `validate_widgets.go` | **Refuse, don't author.** The instinct is to write the mapping the user asked for, but #296 already established that Mendix rejects an explicit one — so the only honest options are "bind the context object" or "say no", and silently doing the first while the author wrote the second is what caused the bug. **To refuse without false positives you need the context *variable*, not its entity.** `pb.entityContext` carries only `Module.Entity`, and both `$Car` and `$Other` have it — a type check cannot tell them apart. Track the name the data source gives the context object (`contextVarFor`: a `parameter` source names it, a database/association/microflow source leaves it `$currentObject`-only) and allow the argument when it matches either spelling. That keeps the **documented** form working — `create-page.md`'s `show_page Mod.EditPage(Product: $Product)` inside a data view on `$Product` is the arg-equals-context case — while catching only arguments that are provably discarded. **`mx check` is worthless as a control here** and that is the whole point of the bug: an inferred mapping is a *valid* mapping, so mxbuild reports 0 errors on the corrupted page. The control is the guard itself — stub `pageArgumentBindsContextObject` to `return true` and the probe writes silently again, describing as `$currentObject`. Mirrored at check time as **MDL-PAGEARG01** so the LSP and `mxcli check` catch it with no project, the same pairing as MDL-WIDGET09. **"No context variable" and "context unknown" are different states, and conflating them is a false positive that bites immediately**: `ALTER PAGE … SET Action = SHOW_PAGE P(Car: $Car) ON btnGo` builds the action against a *stored* page the pass never traverses, so `contextVarName` is empty for a reason that has nothing to do with the argument — the first cut refused that statement, which is correct code. A separate `contextKnown` flag, set only on entering a data-bound widget, keeps the guard to what it can prove; ALTER PAGE behaves exactly as it did before. Caught by trying the ALTER form by hand, not by any test in the suite. Found while verifying §39, whose *reported* half (DESCRIBE omitting the inferred mapping entirely) was already fixed by `4551a4e` | +| A workflow write is never elided: re-running an unchanged `ALTER WORKFLOW` (or any workflow rewrite) produces a different `.mpr` every time, so Studio Pro shows a version-control change on every run even though nothing was edited | Every `Workflows$*` element — activities, outcomes, boundary events, and the workflow root — carries a `PersistentId`, and **both** engines mint a fresh GUID for it on every write (`addFreshPersistentID` in modelsdk, a literal `idToBsonBinary(generateUUID())` in legacy) while **neither** reads the stored value back. A rebuilt document therefore never equals the stored one, so `canon.Reconcile`'s no-op elision could not fire for a workflow | `modelsdk/canon/persistentid.go` (`CarryPersistentIDs`, wired into `Reconcile` in `identity.go`); tests `modelsdk/canon/persistentid_test.go`; blind-spot note in `mdl/backend/modelsdk/identity_drift_test.go` | Carry it in **canon**, not in the writers: one place covers both engines and every nested type, and the writers keep minting fresh for genuinely new elements (which is correct — the carry only applies where a stored counterpart exists). `canon.identityFields`/`CarryIdentity` are **not** the right home — they only reach **top-level** properties of the document root, and `PersistentId` sits on elements arbitrarily deep in the flow tree; reuse `TransplantIDs`' structural pairing instead, and keep the traversal identical to `pairer`'s or one element's `$ID` and `PersistentId` could come from two different stored elements. **`TestFreshGUIDFieldsHaveAnIdentityDecision` does not catch this class**: it sees only properties registered via the codec's `FreshGUIDFields`, and a writer that mints a GUID by hand is invisible to it. Verify by **byte-comparing the `.mpr` across repeated no-op runs on both engines** (`MXCLI_ENGINE=legacy`), not by reading the code — and keep a control that a real edit still writes, or the carry has become a way of losing writes. Issue #949 | +| `DROP ATTRIBUTE` succeeds, but Studio Pro / `mx check` reports **CE1613** "The selected attribute 'Mod.Ent.Attr' no longer exists." at *Validation rule of entity 'Mod.Ent'*. The executor even prints `Removed 1 validation rule(s)` | **Two independent bugs on the same statement, and each hides the other.** (1) The executor's cleanup compared `vr.AttributeID` against the dropped attribute's **element ID**, but a validation rule and a MemberAccess reference their attribute by **BY_NAME qualified name** (`Mod.Ent.Attr`) — both arrive in the same `model.ID` field, so the comparison matched nothing and every rule was kept. Both engines feed the qualified name (`sdk/mpr/parser_domainmodel.go` stores the string verbatim; `mdl/backend/modelsdk/domainmodel.go` says so in a comment). (2) With (1) fixed the rule was removed in memory and **still not written**: `entityToGen` only appends to a child list when it is non-empty, so a list emptied by the update stays *clean* and the codec passes the **stored raw bytes** through — the removal silently does not happen | `mdl/executor/cmd_entities.go` (`AlterEntityDropAttribute` cleanup) and `mdl/backend/modelsdk/domainmodel_alter.go` (`UpdateEntity`, the emptied-child-list loop) | Match **both reference forms** (element ID *or* qualified name built as `Module.Entity.Attr`, exactly what `serializeValidationRule` writes), and dirty **every** emptied child list, not one at a time. The list bug already had a precedent — Indexes, ledger #39 — so `ValidationRules`, `Attributes`, `AccessRules` and `EventHandlers` are all covered in one loop rather than rediscovered one error code at a time. **The trap is that it only bites on the LAST member**: removing one of two rules dirties the list and works fine, so any two-member fixture passes against the broken writer — the `Attributes` case is worse than a dangling reference (dropping an entity's *only* attribute reported success and wrote nothing at all) and had gone unnoticed for the same reason. Index references really are element IDs (`AttributePointer`), which is why that one path always worked and makes the positive control. Verify with `mx check`, not the executor's own output: it reported `Removed 1 validation rule(s)` while CE1613 persisted. Repro `mdl-examples/bug-tests/drop-attribute-orphaned-validation-rule.mdl`. Reported in the banking-app feedback | +| `ALTER WORKFLOW … REPLACE ACTIVITY X WITH ` silently renames it to `X_2`, and re-running the same script compounds it — `X_2_2`, `X_2_2_2`, … A no-op replace (nothing changed) renames too. Invisible to `mxcli check`, `exec` output, and — for `CALL MICROFLOW` steps, whose DESCRIBE renders only the target microflow — to `DESCRIBE` as well | The dedup helper was written for INSERT and reused by REPLACE. `collectAllActivityNames()` walks the still-unmodified tree, so the activity about to be spliced out is still in the name pool when the replacement's name is checked against it — a same-name replace therefore always "collides" with itself. The MCP backend had the same shape, propagated deliberately in PR #204 on a **guess** that the file backends' choice was intentional | `mdl/backend/wfmutator/mutator.go` (`serializeAndDedup` gains `freeName`, `ReplaceActivity` passes the outgoing name); `mdl/backend/mcp/workflow.go` (`activityRefMatch.name` / `wfLocation.name`, `delete(loc.taken, loc.name)`); tests `mdl/backend/wfmutator/replace_samename_test.go`, `mdl/backend/mcp/workflow_dedup_test.go` | Free **only the outgoing activity's name**, and read it off the **resolved element** — the reference may be a caption, not the name, so using `activityRef` frees the wrong key (or none). Keep a control proving a replacement colliding with a *surviving* activity still dedupes, or the fix over-corrects into CE0495. **Check the severity claim before believing it**: the report argued renames detach in-flight `System.WorkflowUserTask` records, but Mendix's [workflow-versioning docs](https://docs.mendix.com/refguide/workflow-versioning/) list "changing names, captions, and titles" as explicitly **non-conflicting** — the engine matches structurally, not by name. The real harm is unbounded accretion and the ADR-0008 idempotence break. **The fix only half-closes that**: a no-op replace still rewrites the `.mpr` because `addFreshPersistentID` re-mints `PersistentId` on every activity write in both engines and neither reads it back (measured: untouched siblings keep theirs, the replaced one does not). Issue #944 | +| `SHOW PROJECT SECURITY` prints no **Guest User Role** on the default engine while `MXCLI_ENGINE=legacy` prints it for the same project — and the documented Starlark fields `anonymous_user_role` / `role.is_anonymous` resolve to `""` / never, so a lint rule asking "what can anonymous visitors read" matches nothing | `modelsdk/gen` bound `Security$ProjectSecurity.GuestUserRoleName`; every Studio Pro document stores **`GuestUserRole`** (likewise `AdminUserRoleName` → `AdminUserRole`). Both were on the known-mismatch ledger, unfixed. The legacy parser reads the right key, which is why the engines disagreed | `modelsdk/gen/security/types.go` (`initProjectSecurity`), ledger rows struck from `modelsdk/gen/keyaudit_test.go` | **A `property.Primitive` decodes AND encodes under one bound name** (`Get()` passes `p.name` to the decode func; the encoder reads the same), so unlike the pages precedents this needs **one** literal, not two — check the property kind before hunting for a second site in `InitFromRaw`. **The read bug was the quiet half; the write bug was fatal.** gen preserved the stored `GuestUserRole` as an unknown-key passthrough, so existing writes lost nothing — but calling `SetGuestUserRoleName` would have added a *second* key beside it, which Studio Pro refuses to open (`Sequence contains no matching element` at `MprProperty.cs`) while mxbuild tolerates it. Measure the encode side before concluding a passthrough makes the key safe. Control for the whole fix: rewrite the stored role under the pre-fix key and `mx check` reports **CE0133** (6 errors vs 5) — i.e. without the override the guest-access feature would have shipped writing a project that cannot build. Tests `modelsdk/gen/security/storagename_projectsecurity_test.go`. Issue #924 | +| An app needing **anonymous access** cannot be built unattended: `SHOW PROJECT SECURITY` reads `Guest Access` but no MDL statement writes it, so every headless run ends with a manual step in Studio Pro | `ALTER PROJECT SECURITY` had exactly two forms (`LEVEL`, `DEMO USERS`). Nothing was missing underneath — `SetEnableGuestAccess` already existed in gen — it had simply never been wired through grammar → AST → visitor → executor → backend | `mdl/grammar/domains/MDLSecurity.g4`, `mdl/ast/ast_security.go`, `mdl/visitor/visitor_security.go`, `mdl/executor/cmd_security_write.go` (`applyGuestAccess`), `mdl/backend/security.go` + both engines | **`DEMO USERS ON\|OFF` is the whole template** for any project-security flag: same AST node, same unit, same write choke point. Copy it rather than designing. **What mxbuild does and does not enforce decides the syntax, and only measurement tells you which:** guest access on with an empty role is **CE0133** (measured 6 errors vs 5), so `ROLE` cannot be optional in effect — but guest access on with a role that **does not exist** builds with the *same* count as a valid one, so mxcli must validate the reference itself or a typo yields a silently broken public site. That asymmetry is why `ROLE` is optional in the *grammar* (a stored role satisfies CE0133, so re-enabling need not retype it) and required by the *executor*. **`OFF` keeps the stored role** — guest-off-with-role is valid Mendix, and clearing it would lose the operator's choice on a toggle. **No check-time (no-project) rule is possible for the bare `ON` case**: whether it is legal depends on what the project stores, which `mxcli check` cannot see — so unlike most refusals this one has no `validate_security.go` counterpart, deliberately. Example `mdl-examples/doctype-tests/30-guest-access.mdl`; tests `mdl/executor/cmd_security_guest_access_test.go`. Issue #924 | +| `DELETE_BEHAVIOR PREVENT` (or `DELETE_IF_NO_REFERENCES`, or `DELETE_AND_REFERENCES`) reports `Modified association` and stores `DeleteMeButKeepReferences`, destroying whatever the association had — and `DESCRIBE ASSOCIATION` emits `delete_behavior DELETE_CASCADE`, which the parser then rejects | `buildDeleteBehavior` read only `db.CASCADE()`; the other four of the rule's five tokens fell through to the zero value `ast.DeleteKeepReferences`. A **legal** behaviour, so nothing below could tell it had been substituted. Underneath it, `ALTER ASSOCIATION SET` built the stored value as `DeleteBehaviorType(s.DeleteBehavior.String())`, and `String()` spells prevent `"DeleteIfNoReferences"` where Mendix writes `"DeleteMeIfNoReferences"` — so fixing the visitor alone put an **out-of-domain enum** on disk | `mdl/visitor/visitor_helpers.go` (`buildDeleteBehavior`), then `mdl/executor/cmd_associations.go` (`storageDeleteBehavior`) | Read **every** token the grammar rule admits — a `default:` arm returning a plausible value turns a parse gap into silent data loss. Route all storage conversions through one function; a `String()` returning storage-ish names is a trap, not an encoder. Emit from DESCRIBE only spellings the lexer has (`DELETE_AND_REFERENCES`, not `DELETE_CASCADE`) and prove it by feeding the output back through `visitor.Build`. **`mx check` is not the oracle here**: measured on mxbuild 11.6.6, a project carrying `DeleteIfNoReferences` builds with 0 errors exactly like a correct one — only Studio Pro refuses it, so assert the stored value is one of Mendix's three, not merely the one you asked for (upstream #901) | +| `CREATE OR REPLACE WORKFLOW` silently deletes a boundary event (timer + its whole handler flow) or an event sub-process that the script does not restate — `exec` reports "Created workflow" and exit 0, and `mx check` afterwards reports 0 errors because the result is a valid workflow that simply no longer does what it did. `DESCRIBE WORKFLOW` does not show the construct either, so nothing reveals the loss | Two separate defects. (1) The **default (modelsdk) engine's workflow reader had no boundary-event support at all** — `grep BoundaryEvent mdl/backend/modelsdk/workflow_read.go` → 0, while `sdk/mpr/parser_workflow.go` → 17. Both engines *write* them, so a boundary event mxcli itself had just written read back as absent. (2) A rewrite rebuilds from the statement, so anything unstated is dropped, and no guard existed — unlike the queued-call and validation-rule paths | `mdl/backend/modelsdk/workflow_read.go` (`boundaryEventsFromGen`, wired into the 6 gen types with `BoundaryEventsItems`); `mdl/executor/validate_workflow_rewrite.go` (`checkNoDroppedWorkflowConstructs`, called from `cmd_workflows_write.go`); `mdl/executor/cmd_workflows.go` (clause order) | **Read the stored side from the RAW unit, not the semantic model** — the reader is what was blind, and a guard sharing its blind spot cannot see what it protects; `GetRawUnit` also covers constructs no engine models yet (event sub-processes exist in `modelsdk/gen` but have no semantic type, reader or writer anywhere). Match the `$Type` by **substring** so all three timer variants and any later one are caught. Authorable constructs get the queue treatment (restate them and the rewrite proceeds); unauthorable ones refuse outright. **Fixing a reader can expose a describer bug**: with boundary events finally readable, `describe → exec` stopped parsing, because the call-microflow describer emitted `boundary event` BEFORE `outcomes` and the grammar requires the reverse — pre-existing on legacy, invisible while the default engine emitted neither. Always re-run a describe→re-exec round trip after widening a reader. Issue #948 | +| A workflow that calls a microflow existing nowhere passes `mxcli check --references` with "All references valid" and is written by `exec` with exit 0; only native `mx check` catches it (**CE1613** "The selected microflow ... no longer exists"). The identical mistake inside a plain microflow body IS caught | `validateWithContext`'s `CreateWorkflowStmt` case called only `validateWorkflowParameterMappings`, whose own comment defers a target that is not in the project to "the missing-reference check" — **which was never written**. `validateFlowBodyReferences` (which does catch it) is wired only to `CreateMicroflowStmt`/`CreateNanoflowStmt`. `AlterWorkflowStmt` had **no case at all** and fell to `default: skip validation` | `mdl/executor/validate_workflow_refs.go` (`validateWorkflowReferences`, `validateWorkflowStatementRefs`, `validateAlterWorkflowRefs`); `mdl/executor/validate.go` (both switch cases, `scriptContext.workflows`, `collectDefinitions`/`collectSingle`/`allNames`); `mdl/executor/helpers.go` (`buildWorkflowQualifiedNames`); exec-side guards in `cmd_workflows_write.go` + `cmd_alter_workflow.go` | **The gap is never one reference kind** — probe them all before fixing. Here *every* reference in a workflow was unvalidated: call microflow, call workflow, user task page, targeting microflow, context entity, and the workflow's own module (checked for a microflow, not for a workflow — and `exec` auto-creates a module, so `check` was the only guard against a typo silently making one). **`check` and `exec` run different passes**: `check --references` runs `validateProgram`, `exec` does not, so a validator wired only into the switch leaves exec writing the broken model — the guard must ALSO be called from the statement handler, before `findOrCreateModule` (same shape as #833). Pass `nil` for the scriptContext there: exec applies statements one at a time, so an earlier statement's output is already in the project. **The real risk is false positives** — verify by diffing error counts against a baseline binary over every workflow script in `mdl-examples/`, with the referenced modules actually present, and exempt `System.*` (`isBuiltinModuleEntity`) or every built-in target is reported missing. Issue #943 | diff --git a/.claude/skills/mendix/README.md b/.claude/skills/mendix/README.md index 140f1eabc..3d3972d9b 100644 --- a/.claude/skills/mendix/README.md +++ b/.claude/skills/mendix/README.md @@ -20,6 +20,7 @@ Detailed syntax for each MDL document type: | [mdl-entities.md](mdl-entities.md) | Entity, attribute, association syntax | Creating domain models | | [write-microflows.md](write-microflows.md) | Microflow syntax reference | Writing microflow logic | | [write-nanoflows.md](write-nanoflows.md) | Nanoflow syntax reference | Writing client-side nanoflow logic | +| [write-rules.md](write-rules.md) | Rule syntax reference | Writing reusable decision logic a decision calls | | [write-oql-queries.md](write-oql-queries.md) | OQL query syntax | Creating VIEW entities | | [create-page.md](create-page.md) | Page and widget syntax | Creating pages | | [fragments.md](fragments.md) | Fragment (reusable widget group) syntax | Reusing widget patterns across pages | @@ -46,6 +47,7 @@ External system integration: | [demo-data.md](demo-data.md) | Demo data & IMPORT | Seeding data, `import from` bulk import from external DB | | [rest-client.md](rest-client.md) | REST API consumption | Calling external REST APIs via consumed REST client documents | | [rest-call-from-json.md](rest-call-from-json.md) | REST CALL end-to-end | JSON Structure → Entities → Import Mapping → REST CALL microflow | +| [mock-rest-apis.md](mock-rest-apis.md) | Mock a REST dependency | Building or debugging a REST integration without the live API; forcing 404/500; running offline or in CI | | [json-structures-and-mappings.md](json-structures-and-mappings.md) | JSON structures & mappings | CREATE/DESCRIBE JSON structures, import/export mappings, domain model patterns | | [java-actions.md](java-actions.md) | Custom Java actions | Extending with Java code | | [download-marketplace-content.md](download-marketplace-content.md) | Marketplace download & install | Adding a marketplace module/widget; downloading a `.mpk`; module-update caveat | diff --git a/.claude/skills/mendix/create-page.md b/.claude/skills/mendix/create-page.md index 967f8f8e2..34870f292 100644 --- a/.claude/skills/mendix/create-page.md +++ b/.claude/skills/mendix/create-page.md @@ -319,6 +319,13 @@ describe icon collection Atlas_Core.Atlas_Filled -- every icon + its reference - `action: show_page Module.PageName(Param: $value)` - Navigate with parameters - `action: show_page Module.PageName($Param = $value)` - Also accepted (microflow-style) - `action: create_object Module.Entity then show_page Module.PageName` - Create and navigate +- **A `show_page` argument must be the context object.** Mendix takes the page + argument from the enclosing data widget, so the only spellings that mean + anything are `$currentObject` or the name of the variable that widget is bound + to (`datasource: $Customer` → `(Customer: $Customer)` is fine). Naming any other + variable is refused as **MDL-PAGEARG01** — it used to be accepted and silently + opened the page with the context object anyway. To open a page with something + else, call a microflow that shows it. **Button Styles:** `default`, `primary`, `success`, `info`, `warning`, `danger`, `inverse` - Case-insensitive (`primary` and `Primary` both work). diff --git a/.claude/skills/mendix/manage-security.md b/.claude/skills/mendix/manage-security.md index 1662e37fe..6e111a6ac 100644 --- a/.claude/skills/mendix/manage-security.md +++ b/.claude/skills/mendix/manage-security.md @@ -262,6 +262,43 @@ alter project security demo users on; alter project security demo users off; ``` +### Guest (Anonymous) Access + +Anonymous access is what makes part of an app public — a product catalogue anyone +can browse without signing in. It is one flag plus a user role, and the role is +the important half: **whatever that role can read is the app's public surface.** + +```sql +-- The role anonymous visitors are given. System.User is what lets an +-- unauthenticated session exist at all. +create user role Anonymous (Shop.Viewer, System.User); + +alter project security guest access on role Anonymous; + +-- Now grant exactly what should be public — and nothing else. +grant Anonymous on Shop.Product (read *); + +-- Re-enabling later does not need the role retyped; the stored one is used. +alter project security guest access off; +alter project security guest access on; +``` + +Three things worth knowing: + +- **The role is mandatory.** Mendix fails the build with **CE0133** ("No user role + for anonymous users selected even though the feature anonymous users is + enabled") when access is on with no role. `guest access on` is refused unless a + role is given or one is already stored. +- **Mendix does not check the role exists**, so mxcli does. A misspelled role + would otherwise build with zero errors and leave anonymous visitors with no + access at all — a broken public site that passes every check. +- **`off` keeps the stored role**, so toggling access while testing does not lose + it. Guest access off with a role set is valid Mendix. + +Review anonymous entity access the way lint rule **SEC004** asks you to: any +unconstrained `read *` granted to the anonymous role is readable by the whole +internet (DIVD-2022-00019). Add an XPath constraint or do not grant it. + ### Demo Users ```sql diff --git a/.claude/skills/mendix/mock-rest-apis.md b/.claude/skills/mendix/mock-rest-apis.md new file mode 100644 index 000000000..0f99d25ec --- /dev/null +++ b/.claude/skills/mendix/mock-rest-apis.md @@ -0,0 +1,194 @@ +# Mock REST APIs Skill + +Use this skill when a REST integration needs an endpoint you control instead of a +live third-party API — while building it, while reproducing a bug, or while +verifying the app in a browser or a test run. + +Developing against the real API means network, rate limits, credentials, and a +payload that can change under you. None of that is where Mendix integration +defects live: those are in the mapping, the entity types, the error handler, and +the BSON. A mock removes the variables that are not the bug. + +## When to Use This Skill + +- Building a REST client or `REST CALL` microflow before (or without) real credentials +- Reproducing a payload-shaped bug **deterministically** — a shape small enough to read, that behaves the same on every run +- Exercising error paths: 404, 500, a timeout, a 401 from missing auth +- Verifying the app (`test-app.md`) or running a suite (`test-microflows.md`) offline or in CI +- Redirecting the outbound calls of an app whose model you must not edit + +## Two separate problems + +Almost every wasted hour here comes from conflating them: + +| Problem | Answer | +|---|---| +| **Something must answer the request** | A mock server: Prism (from a contract), WireMock, mitmproxy | +| **The app must send the request there** | A constant, a `BaseUrl`, or a forward proxy — see below | + +A mock server is **not** an interceptor. Prism serves one contract at one port +and answers only clients that address it. Asking it to "catch all calls the app +already makes" is a category error — that is the forward-proxy job, further down. + +## 1. Point the app at the mock + +Three routes, cheapest first. Pick by how the URL is built. + +### The URL is built in the microflow — use a constant, change nothing per run + +A `REST CALL` URL is an **expression**, so it can be assembled from a constant +(`@Module.Constant` is Mendix's constant reference — `$Name` is a *variable*): + +```sql +create constant MyModule.ApiBaseUrl type String default 'https://api.example.com/v1'; + +create microflow MyModule.CallApi() returns string +begin + $response = rest call get @MyModule.ApiBaseUrl + '/rates' + header Accept = 'application/json' + returns string; + return $response; +end; +``` + +Then swap the endpoint per run, with no model change and nothing committed: + +```bash +# this run only — never written to the project +mxcli run --local -p app.mpr --constant MyModule.ApiBaseUrl=http://127.0.0.1:4020 + +# a test suite against the mock (--constant needs --local) +mxcli test tests/ -p app.mpr --local --constant MyModule.ApiBaseUrl=http://127.0.0.1:4020 + +# machine-local default, gitignored: every run picks it up +mxcli constant set MyModule.ApiBaseUrl http://127.0.0.1:4020 -p app.mpr + +# flip it on an app that is already running +mxcli constant set MyModule.ApiBaseUrl http://127.0.0.1:4020 -p app.mpr --apply +``` + +`constant set` refuses a name the project does not define, so a typo cannot +silently apply to nothing. `mxcli constant list -p app.mpr` shows the winning +value for every constant **and which layer set it** — read it first whenever a +run does not use the endpoint you expected. + +### The call goes through a REST client document — rewrite `BaseUrl` + +A REST client document's `BaseUrl` is a **literal**; it cannot reference a +constant. Point it at the mock by re-running the create, which is a one-line diff: + +```sql +create or modify rest client MyModule.RatesAPI ( + OpenAPI: 'specs/rates.json', + BaseUrl: 'http://127.0.0.1:4020' +); +``` + +`BaseUrl` also overrides `servers[0].url` at import time, so one contract can be +imported against the mock and later re-pointed at production. + +### You cannot edit the model at all — use a forward proxy + +See §3. This is the most work and the last resort. + +## 2. Prism: serve an OpenAPI contract as a mock + +```bash +npm install -g @stoplight/prism-cli # ~15s +prism mock specs/rates.json --port 4020 # serves the contract's `example` values +``` + +Everything below cost real time to find out and is not on Prism's front page: + +- **Prism mounts paths at the root** and ignores any base path in + `servers[0].url`. Address it as `http://127.0.0.1:4020`, not + `http://127.0.0.1:4020/v1` — otherwise every path 404s while the server looks + perfectly healthy. +- **Make `servers[0].url` absolute in the contract you import.** mxcli's OpenAPI + import only accepts an `http://` or `https://` URL as `BaseUrl`; a relative one + (`/api/v3`) is skipped with the warning *"server URL … is relative and cannot + be used as BaseUrl; set BaseUrl explicitly in CREATE REST CLIENT"*, and a + client with no `BaseUrl` fails at call time, not at import time. +- **`Prefer: code=404`** on the request forces any status the contract documents. + This is the only practical way to drive a Mendix error handler through a real + HTTP response rather than by hand-editing the model. +- **`prism mock -d`** returns schema-generated random data instead of the + `example` values. Run the suite both ways: a mapping that quietly depends on + one fixed payload passes under `example` and fails under `-d`. +- **Prism enforces the contract's `security`**, so a call with no `Authorization` + header gets a real 401. Useful — but know the ceiling before you design around + it: a REST client document's header value may be a literal, a `$Variable`, or a + literal **prefix** plus a variable (`'Bearer ' + $Token`), and nothing else. A + token that must be computed per call belongs in a `REST CALL` expression, not + in the document. +- **Cut a subset; never point Prism at a vendor's full contract.** The official + Microsoft Graph spec is 41 MB of YAML: it downloads in seconds and Prism was + still printing "Starting Prism…" when killed at a 300-second cap. Importing it + would also generate thousands of operations into the module. + +A contract small enough to read is the point. Hand-cut one path with one +`example` per response code you care about, and keep it in the project under +`specs/` next to the `.mpr` — the same relative path the `OpenAPI:` clause takes. + +## 3. Forward proxy: when the model cannot change + +For an app whose model you must not touch, redirect the JVM instead. The Mendix +runtime honours the standard Java proxy properties, and `mxcli run --local` +passes your environment through to the runtime JVM — including `JAVA_TOOL_OPTIONS`, +which mxcli **appends** to rather than replaces, so an exported value survives +even alongside `--trace`. No model change, nothing committed: + +```bash +export JAVA_TOOL_OPTIONS="-Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8080" +mxcli run --local -p app.mpr +``` + +Two things to know before committing to this route: + +- **HTTPS is the real work.** The proxy must present a certificate the JVM + trusts. WireMock 3.13.2 on Java 21 cannot generate a usable one; use mitmproxy + (which ships a CA you install into the JVM truststore) or supply your own + keystore. Plain `http://` targets need none of this — one more reason to have + the mock on loopback HTTP. +- **Loopback is not proxied.** `127.0.0.1` sits in the runtime's + `http.nonProxyHosts`, so app→mock traffic on loopback works *inside* a proxied + container without any exemption of your own. It also means a proxy on + `127.0.0.1` will not intercept loopback calls — that is not a bug to hunt. + +For **consumed OData** services specifically there is a fourth route that needs +no JVM flags: `System.ConsumedODataConfiguration` carries `ProxyConfiguration`, +`ProxyHost` and `ProxyPort` as data (see `system-module.md`), so the proxy can be +set per service at runtime. + +## Verify the mock before blaming Mendix + +Always prove the endpoint from the shell first. A Mendix error message cannot +distinguish "the mock is not running" from "the mapping is wrong". + +```bash +curl -sS -i http://127.0.0.1:4020/rates # 200 + the example payload? +curl -sS -i -H 'Prefer: code=404' http://127.0.0.1:4020/rates +``` + +Then, and only then, run the microflow and check the payload actually reached it +(`mxcli oql`, or the runtime log under `mxcli run --local`). + +## Failure modes, symptoms first + +| Symptom | Cause | Fix | +|---|---|---| +| Every path 404s, server looks fine | The client address includes the contract's base path | Address Prism at the root: `http://127.0.0.1:4020` | +| Import produced a client with no `BaseUrl` | `servers[0].url` is relative — mxcli warned and skipped it | Make it absolute, or pass `BaseUrl:` explicitly | +| Prism never finishes starting | Vendor contract is tens of MB | Cut the paths you need into a small contract | +| Mock returns 401 | The contract declares `security`; the call sent no credentials | Add the header, or drop `security` from your cut contract | +| Calls still reach the real API | Proxy properties not applied, or the target is loopback (never proxied) | Check the JVM args; prefer the constant route over a proxy | +| Endpoint swapped but the app disagrees | An override on a different layer wins | `mxcli constant list -p app.mpr` — it names the layer | +| Works with `example` values, fails in CI | The mapping depends on one fixed payload | Run `prism mock -d` locally and fix the mapping | + +## Related Skills + +- `rest-client.md` — the three ways to call a REST API; where the contract goes once you have one +- `rest-call-from-json.md` — JSON structure → entities → import mapping → `REST CALL` +- `test-app.md` — browser verification; a REST app's prerequisite is a reachable endpoint +- `test-microflows.md` — running a suite; `--constant` points it at the mock +- `run-local.md` — `mxcli run --local`, the warm loop the mock plugs into diff --git a/.claude/skills/mendix/overview-pages.md b/.claude/skills/mendix/overview-pages.md index 169c36cc5..f906412c4 100644 --- a/.claude/skills/mendix/overview-pages.md +++ b/.claude/skills/mendix/overview-pages.md @@ -534,6 +534,13 @@ navigationlist widgetName { - `action: microflow Module.MicroflowName(Param: $value)` - Call microflow with parameters - `action: show_page Module.PageName` - Navigate to page - `action: show_page Module.PageName(Param: $value)` - Navigate with parameters +- **A `show_page` argument must be the context object.** Mendix takes the page + argument from the enclosing data widget, so the only spellings that mean + anything are `$currentObject` or the name of the variable that widget is bound + to (`datasource: $Customer` → `(Customer: $Customer)` is fine). Naming any other + variable is refused as **MDL-PAGEARG01** — it used to be accepted and silently + opened the page with the context object anyway. To open a page with something + else, call a microflow that shows it. ## Handling Circular Dependencies diff --git a/.claude/skills/mendix/rest-client.md b/.claude/skills/mendix/rest-client.md index 8cdc28598..81d3e5e40 100644 --- a/.claude/skills/mendix/rest-client.md +++ b/.claude/skills/mendix/rest-client.md @@ -14,6 +14,8 @@ Mendix offers three ways to call REST APIs from microflows. Choose based on the Both REST Client approaches can be combined with **Data Transformers** (Mendix 11.9+) and **Import/Export Mappings** to map between JSON and entities. +No API to call against yet — or one you would rather not depend on while building? [mock-rest-apis.md](mock-rest-apis.md) covers standing up an endpoint you control and pointing the app at it. + --- ## Approach 0: OpenAPI Import (Fastest) @@ -44,7 +46,14 @@ This generates: - Basic auth if the spec declares it at the top level - The spec stored inside the document for Studio Pro parity -`BaseUrl` is optional. When omitted, `servers[0].url` from the spec is used. When provided, it overrides that value — useful when the spec points at production but you need to import against staging or a different version. +`BaseUrl` is optional. When omitted, `servers[0].url` from the spec is used — but only if it is +**absolute**. A relative server URL (`/api/v3`) cannot be a `BaseUrl`: the import warns +(*"server URL … is relative and cannot be used as BaseUrl"*) and leaves the client without one, +which fails at call time rather than at import time. Set `BaseUrl` explicitly in that case. + +When provided, `BaseUrl` overrides the spec's value — useful when the spec points at production +but you need to import against staging, a different version, or a local mock +(see [mock-rest-apis.md](mock-rest-apis.md)). **Preview without writing:** ```sql diff --git a/.claude/skills/mendix/test-app.md b/.claude/skills/mendix/test-app.md index ecde9ce0c..4e1ecb1c5 100644 --- a/.claude/skills/mendix/test-app.md +++ b/.claude/skills/mendix/test-app.md @@ -21,6 +21,8 @@ The devcontainer created by `mxcli init` installs: - **Chromium (headless shell)** — installed via `@playwright/cli`'s **bundled** `playwright-core`, into a shared `PLAYWRIGHT_BROWSERS_PATH`, and exposed at the stable path `/usr/local/bin/mx-headless-shell`. The generated `.playwright/cli.config.json` pins `executablePath` to that symlink. - **Docker-in-Docker** — Mendix + PostgreSQL running via `mxcli docker run` +If the app calls an external REST API, that endpoint is a prerequisite too — a verification run that depends on a live third party is not repeatable. See [mock-rest-apis.md](mock-rest-apis.md). + The app must be running before verification: ```bash diff --git a/.claude/skills/mendix/test-microflows.md b/.claude/skills/mendix/test-microflows.md index 1b1d54837..902984777 100644 --- a/.claude/skills/mendix/test-microflows.md +++ b/.claude/skills/mendix/test-microflows.md @@ -124,6 +124,18 @@ $result = call microflow MyModule.Multiply(A = 10, B = 5); / ``` +**A file may open with a header comment**, in either spelling — a `/** … */` +block or `--` lines — and it does not become part of the first test. The test's +own doc comment is the last one above its statements. A `/** … */` header may +carry `@setup`, which then applies to every test in the file. + +**One `@test` per block.** The `/` is what ends a test, so leaving it out puts +two tests in one block; that is refused by name rather than resolved, because +either way of resolving it runs one of the two and silently drops the other. +Both rules are #927: a file-level header used to swallow the first test whole — +it disappeared from the results with no error, and every later test reported +under the number of the one above it. + ### `.test.md` — Markdown Specification Tests embedded in documentation as `mdl-test` fenced code blocks: @@ -157,8 +169,14 @@ The markdown format turns your tests into living documentation. | `@expect` | Assert with a built-in | `@expect length($result) = 81` | | `@verify` | OQL post-condition on the database | `@verify select count(*) as n from Mod.E = 1` | | `@throws` | Expect error | `@throws 'validation failed'` | +| `@setup` | Microflow to run first | `@setup MyModule.ACT_SeedCustomers` | | `@cleanup` | Rollback strategy | `@cleanup rollback` (default) or `@cleanup none` | +A tag is read only when it **opens its line** (after the javadoc `*` and its +indentation). Quoting one inside a sentence — ``a test with `@expect $x = 1` +asserts …`` — is documentation, not an annotation, so a doc comment can explain +itself without giving the test assertions nobody wrote. + ### A test run leaves the project byte-identical `mxcli test` injects an `MxTest` module, builds, runs, and takes the injection @@ -191,6 +209,7 @@ An `@expect` is **a Mendix expression that must evaluate to true**, not a fixed @expect substring($result, 0, 9) = substring($result, 9, 18) @expect find($result, '0') >= 0 and $count > 3 -- and / or / not(...) @expect $status = MyModule.Status.Open -- enumeration values +@expect count($Customers) = 5 -- how many rows a list holds ``` `<>` is accepted in the annotation and rewritten to `!=` on the way to the @@ -223,6 +242,84 @@ The value is omitted rather than guessed when neither side of the comparison establishes a type (`@expect $a = $b`), because Mendix's expression engine is typed and a wrong guess would break the build instead of the test. +#### `count($list)` — the one aggregate an assertion can make + +Counting a list is not a Mendix *expression* function; it is an Aggregate list +**activity**, so it cannot appear in the decision that evaluates an assertion. +`@expect count($Scans) = 2` is nevertheless accepted: the count is lifted into +the activity you would otherwise write by hand, ahead of the decision, and the +condition compares its result. + +```mdl +/** + * @test the seed microflow writes five brands + * @cleanup none + * @expect count($Brands) = 5 + */ +retrieve $Brands from eShop.CatalogBrand; +/ +``` + +The other four aggregates (`sum`, `average`, `minimum`, `maximum`) aggregate an +**attribute** over the list, which an assertion has no way to supply, so they are +refused with that explanation. Call a microflow that returns the figure and +assert on its result: + +```mdl +$Total = call microflow eShop.QRY_OrderTotal(); +``` + +The refusal matters more than the convenience: before it, a count assertion was +dropped during parsing, and a test with no assertions left passes as long as its +body does not throw — so `@expect count($Brands) = 999` reported PASS against an +empty table (#927). + +### `@setup` — the state a test needs before it runs + +`@setup` names a **microflow** to call before the test's own statements. A +fixture in a Mendix app is a microflow, so there is nothing to declare: + +```mdl +/** + * @test the seed microflow writes five brands + * @setup eShop.ACT_SeedCatalog + * @cleanup none + * @expect count($Brands) = 5 + */ +retrieve $Brands from eShop.CatalogBrand; +/ +``` + +Repeat it to compose fixtures; they run in the order written. Declare it **once +in the file's header comment** and every test in the file gets it, with the +file's fixtures running before a test's own: + +```mdl +/** + * Seeds every test below. + * @setup eShop.ACT_SeedCatalog + */ +``` + +The header is the file's first doc comment when it carries no `@test`. It may +only carry `@setup` — `@expect`, `@verify`, `@throws` and `@cleanup` describe one +test's execution, so a header carrying one is refused by name rather than +silently ignored. + +Two consequences worth knowing: + +- **The setup runs inside the test's transaction.** Under the `@cleanup + rollback` default it is undone with the test, so every test starts from the + same state — which is what makes a fixture worth having. Under `@cleanup none` + it persists like everything else that test writes. +- **A failing setup is an ERROR, not a FAIL**, naming the microflow. The test + never ran, so it neither passed nor failed, and a suite full of assertion + mismatches caused by one broken seed is exactly what this prevents. + +`@setup` calls a microflow with no arguments; a fixture that needs arguments +gets a wrapper microflow. There is no `@teardown` — `@cleanup rollback` is the +teardown. + ### `@verify` — asserting on what the microflow wrote `@expect` can only see what a microflow **returned**. Most Mendix microflows are @@ -611,6 +708,13 @@ call microflow Sales.ValidateOrder(Total = -1); / ``` +`@throws` and `@expect` cannot be combined. A `@throws` test compiles to a +different shape — the verdict starts as a failure and only the error handler +clears it — and the `@expect` checks are not emitted into it at all, so the +assertion was counted and never evaluated. There is nothing to assert on either +way: the body was expected not to produce a result. Assert on the error with +`@throws`, or on the result with `@expect`. + --- ## Test File Organization diff --git a/.claude/skills/mendix/write-microflows.md b/.claude/skills/mendix/write-microflows.md index e629f1fd1..2224799cd 100644 --- a/.claude/skills/mendix/write-microflows.md +++ b/.claude/skills/mendix/write-microflows.md @@ -452,6 +452,40 @@ declare $Session string = ''; set $Session = 'anonymous'; -- valid, any number of times ``` +### 10. Calling a Rule or Microflow Inside an Expression + +**Error**: CE0117 - "Error(s) in expression." (MDL066) + +A Mendix **expression** has no user-callable functions. Its library is built-in +and unqualified (`length`, `toString`, `contains`, ...); microflows, rules and +Java actions are called by **activities**. So a qualified call in a value +position is not an expression at all — mxbuild rejects it whichever document it +names. + +❌ **INCORRECT:** +```mdl +declare $Active Boolean = Sample.Rule_IsActive(IsActive = $IsActive); +declare $Next Integer = Sample.MF_Increment(N = $N); -- a microflow is no better +``` + +✅ **CORRECT** — a microflow or Java action is an activity: +```mdl +$Next = call microflow Sample.MF_Increment(N = $N); +``` + +A **rule** has no call activity at all: Mendix can only evaluate one as a +decision's condition, and that is the single position where a bare qualified +call is valid MDL. + +```mdl +if Sample.Rule_IsActive(IsActive = $IsActive) then + ... +end if; +``` + +The name in that position must resolve to a real **rule** — a microflow there is +the same CE0117, and mxcli refuses the statement rather than writing it. + ## Control Flow ### IF Statements diff --git a/.claude/skills/mendix/write-rules.md b/.claude/skills/mendix/write-rules.md new file mode 100644 index 000000000..d8ab4b0c1 --- /dev/null +++ b/.claude/skills/mendix/write-rules.md @@ -0,0 +1,131 @@ +# Mendix Rule Skill + +Guidance for writing Mendix **rules** in MDL. Mendix's own reference calls a rule +["a special kind of microflow"](https://docs.mendix.com/refguide/rules/): it +returns a Boolean or an enumeration, and it can only be used from a decision. + +## When to Use This Skill + +- Writing `CREATE RULE` statements +- Deciding whether logic belongs in a rule or a microflow +- Understanding what `mxcli check` refuses in a rule body and why + +The mirrors are [write-microflows.md](./write-microflows.md) and +[write-nanoflows.md](./write-nanoflows.md) — a rule is the third flavour and +shares their body syntax exactly. + +## When to Use a Rule vs a Microflow + +| Scenario | Use | +|----------|-----| +| One condition, evaluated from several decisions | Rule | +| A named business condition the model should show by name | Rule | +| Choosing one of several enumerated outcomes from input | Rule | +| Anything that changes data | Microflow | +| Anything the user sees (page, message, download) | Microflow | +| Anything that talks to another system | Microflow | + +**Rule of thumb:** a rule *answers a question*. The moment it needs to *do* +something, it is a microflow. + +## Key Differences from Microflows + +| Aspect | Microflow | Rule | +|--------|-----------|------| +| **Return type** | Anything, including void | Boolean or enumeration — mandatory | +| **Called from** | Anywhere | A decision, and nowhere else | +| **Changes data** | Yes | No | +| **Talks to the client** | Yes | No | +| **Integration** | Yes | No | +| **Module-role security** | `grant execute on microflow …` | None — a rule has no security to grant | + +That last row is not an mxcli omission. A rule document stores no +`AllowedModuleRoles`, because a rule is never called on its own; it is reached +through the microflow that evaluates it, and that microflow's security applies. + +## Syntax + +``` +create or modify rule Sales.Rule_IsSolvent ($pCustomer: Sales.Customer) +returns Boolean +folder 'Rules' +begin + return $pCustomer/Balance >= 0; +end +/ +``` + +An enumeration rule lets one decision fan out to several branches: + +``` +create or modify rule Sales.Rule_Outcome ($pCustomer: Sales.Customer) +returns enum Sales.Outcome +begin + if $pCustomer/Balance >= 0 then + return Sales.Outcome.Approved; + else + return Sales.Outcome.Rejected; + end if; +end +/ +``` + +Calling one — the only place a rule may be called: + +``` +create or modify microflow Sales.MF_Screen ($pCustomer: Sales.Customer) +begin + if Sales.Rule_IsSolvent(pCustomer = $pCustomer) then + return; + else + return; + end if; +end +/ +``` + +The argument names are the rule's parameter names, so +`Rule_IsSolvent(pCustomer = $pCustomer)` reads the same way a microflow call does. + +## Reading and managing rules + +``` +list rules; -- `show rules` is the same statement +list rules in Sales; +describe rule Sales.Rule_IsSolvent; -- round-trippable MDL +drop rule Sales.Rule_IsSolvent; +move rule Sales.Rule_IsSolvent to folder 'Rules/Customer'; +``` + +`show microflows` lists microflows only — not nanoflows, not workflows, and not +rules. Each doctype has its own listing. + +`show callers of Sales.Rule_IsSolvent` lists the microflows whose decisions +evaluate it, and a microflow called from inside a rule's body is a normal +reference: it will not be reported as dead code. + +## What a rule may not contain + +`mxcli check` refuses these before the build, with the same function `exec` uses, +so the two cannot disagree. Each was measured against mxbuild 11.13.0: + +| Written in a rule | mxbuild says | +|---|---| +| `create` / `change` / `delete` / `commit` / `rollback` | CE0009 "This action is not supported in rules." | +| `show page`, `close page`, `show message`, validation feedback, download | CE0009 | +| `call web service` | CE0009 | +| A void, String, Integer … return type | CE0103 and CE0139 — the return type must be Boolean or an enumeration | + +A missing `returns` clause is refused too. For a microflow, void is a legitimate +choice; for a rule it means the decision calling it has nothing to branch on. + +## Validation checklist + +Before presenting a rule: + +- [ ] `returns Boolean` or `returns enum Module.Enum` is present +- [ ] The body only reads — no create/change/delete/commit/rollback +- [ ] Nothing in the body touches the client or another system +- [ ] Every path returns a value of the declared type +- [ ] The caller is a decision: `if Module.Rule_Name(Param = $Value) then` +- [ ] `mxcli check script.mdl -p app.mpr --references` passes diff --git a/.claude/skills/mendix/write-workflows.md b/.claude/skills/mendix/write-workflows.md index ed777c0b7..66d8e8b66 100644 --- a/.claude/skills/mendix/write-workflows.md +++ b/.claude/skills/mendix/write-workflows.md @@ -160,11 +160,34 @@ surface (insert path, drop path, insert condition, boundary events). ## DESCRIBE round-trip `DESCRIBE WORKFLOW Module.Name` emits **executable, re-runnable** MDL — user -tasks, decisions, splits, jump-to targets, and wait activities all come back as -statements (not comments). You can learn the exact syntax by describing a -Studio-Pro-authored workflow, and `describe → drop → exec` reproduces a workflow -that builds. (The implicit start/end activities are omitted, as they are -re-synthesised on create.) +tasks, decisions, splits, jump-to targets, wait activities and boundary events +all come back as statements (not comments). You can learn the exact syntax by +describing a Studio-Pro-authored workflow, and `describe → drop → exec` +reproduces a workflow that builds. (The implicit start/end activities are +omitted, as they are re-synthesised on create.) + +**What DESCRIBE still cannot see: event sub-processes.** mxcli has no model for +them at all, so they do not appear in the output and cannot be written from MDL. +A workflow that has one can only be edited in Studio Pro or through +`ALTER WORKFLOW` (below) — never with `CREATE OR REPLACE`. + +## Rewriting an existing workflow + +`CREATE OR REPLACE|MODIFY WORKFLOW` **rebuilds the workflow from the statement**, +so anything the script does not restate is deleted — including each boundary +event's whole handler flow. This is the failure that costs real work: it is not +reported by `mx check` afterwards, because the result is a perfectly valid +workflow that simply no longer does what it did. + +mxcli refuses the two cases where that would lose something: + +- a stored **event sub-process** — MDL cannot express one, so the rewrite is + refused outright; +- **more stored boundary events than the statement declares** — restate them and + the rewrite proceeds, which is what `describe workflow` now emits for you. + +The safe way to change one activity in a workflow carrying hand-placed structure +is `ALTER WORKFLOW`, which mutates in place and touches nothing else. ## Microflow statements for workflow tasks @@ -200,6 +223,37 @@ documented in `system-module.md`. name exactly. Mendix expressions are case-sensitive on 11.9+, so a lowercase `$workflowContext` is an undefined variable and yields `CE0117`. +## Observing a running workflow + +A workflow's characteristic failures are **runtime** failures — an instance that +starts and stops, a task that never reaches an inbox, a task page that renders +blank. None of them is visible to `mxcli check`, `mxcli lint` or `mx check`, +which all validate the model rather than the data the model no longer matches. +So do not stop at "it builds". + +Everything needed is already a skill — read the one you need rather than +hand-rolling admin-API calls: + +| To see | Read | +|---|---| +| Live instances and open tasks (OQL against the running app) | [`verify-with-oql.md`](verify-with-oql.md), [`write-oql-queries.md`](write-oql-queries.md) | +| The exception that stopped an instance | [`analyze-runtime.md`](analyze-runtime.md) — `run --local` tees the runtime log to `/.mxcli/runtime.log` | +| `System.Workflow` / `System.WorkflowUserTask` / `System.WorkflowDefinition` shapes | [`system-module.md`](system-module.md) | +| Driving a task end to end and asserting the result | [`test-app.md`](test-app.md), [`run-local.md`](run-local.md) | +| Raw admin API, incl. `POST /dev/preview_execute_oql` | [`runtime-admin-api.md`](runtime-admin-api.md) | + +Two traps worth knowing before you start: + +- **The declared return type is not what the runtime checks.** A workflow-called + microflow whose end event returns a value while the microflow declares no + return type fails at instance start with `Trying to compare + VoidConditionValue$('') to BooleanValue('true')`. `mxcli check` catches this as + **MDL004** — so do not skip it, and do not reach for `--no-check` to get past + it. Read the message in the order it is written: the receiver is the stored + outcome's condition, the argument is what the microflow actually returned. +- **A parked instance is not a failed one.** A wait or timer branch is supposed + to sit there. Check the branch before calling it a hang. + ## Validate before presenting ```bash diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f7379307..364199f53 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **A `SHOW_PAGE` widget argument that is not the context object is refused instead of ignored** — `action: show_page Mod.Detail(Car: $Other)` inside a data view bound to `$Car` opened the page with **`$Car`**. mxcli stores a widget show-page action with an empty `ParameterMappings` array and lets Mendix infer the argument from the enclosing widget, which is deliberate and required (an explicit mapping is rejected as CE0115, #296) — but an argument naming anything else was then dropped on the floor by both engines. Nothing reported it: `mx check` gave **0 errors**, because an inferred mapping is a valid mapping, and `DESCRIBE` printed `(Car: $currentObject)`, so the description read as a diagnosis of a lost mapping rather than an accurate report of a model that never held one. The builder now refuses such an argument, and `mxcli check` flags it as **MDL-PAGEARG01** with no project needed. The guard fires only where it can prove the argument is discarded: `ALTER PAGE`'s `SET`/`INSERT` build an action against a stored page they never traverse, so the context object is unknown there and those statements are unaffected. Arguments that *do* name the context object — `$currentObject`, or the variable the enclosing data widget is bound to, which is the form the skills document — are unaffected. Found while verifying mxcli-formula1 §39, whose reported half (DESCRIBE omitting the inferred mapping) was already fixed. + - **`DESCRIBE` no longer mis-reads a mapping that binds a nested leaf** (#927) — value elements were printed as the last segment of their JsonPath alone, so a project holding `(Object)|customer|name` described as `CustomerName = name`. That is a description of a model that does not exist, and re-executing mxcli's own output failed with `"name" is not a member of the JSON structure at (Object)`. Members are now rendered relative to the enclosing object element, on both engines and for both mapping kinds. Nothing was ever corrupted — the #882 guard refused the bad re-execution — but the description was wrong. diff --git a/CLAUDE.md b/CLAUDE.md index d6e902319..e731d1971 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -735,6 +735,7 @@ Regenerate after modifying `MDLLexer.g4`, `MDLParser.g4`, or any `domains/*.g4` - `.claude/skills/design-mdl-syntax.md` - **READ before designing new MDL syntax** - Design principles, decision framework, anti-patterns, checklist - `.claude/skills/write-microflows.md` - Microflow syntax, common mistakes, validation checklist - `.claude/skills/write-nanoflows.md` - Nanoflow syntax, restrictions, disallowed activities, validation checklist +- `.claude/skills/mendix/write-rules.md` - **Rules** (CREATE/LIST/DESCRIBE/DROP/MOVE RULE): a rule returns Boolean or an enumeration and is callable only from a decision; what its body may not contain and the CE numbers behind each refusal; why there is no `grant execute on rule` - `.claude/skills/write-workflows.md` - **Workflow authoring** (CREATE/DROP/ALTER WORKFLOW): activities (user task, decision, parallel split, jump, wait, boundary events), header options, gotchas. Workflows are authorable, not read-only. - `.claude/skills/create-page.md` - Page/widget syntax reference - `.claude/skills/alter-page.md` - ALTER PAGE/SNIPPET in-place modifications (SET, INSERT, DROP, REPLACE, SET Layout) @@ -807,6 +808,7 @@ Full syntax tables for all MDL statements (microflows, pages, security, navigati - ALTER WORKFLOW (SET properties, INSERT/DROP/REPLACE activities, outcomes, paths, conditions, boundary events) - CALCULATED BY microflow syntax for calculated attributes - Image collections (SHOW/DESCRIBE/CREATE/DROP) +- Rules (LIST/DESCRIBE/CREATE [OR MODIFY]/DROP/MOVE RULE): Mendix's "special kind of microflow" — returns Boolean or an enumeration, callable only from a decision. Handled as a third flow flavour beside microflows and nanoflows: its own semantic type, its own listing (`show microflows` stays microflow-only), the shared `microflowBody`, flow builder and describer. The document is the ten properties a Studio Pro rule stores, pinned against two reference rules (ako/TestApp, 11.13.0) — **no `AllowedModuleRoles`** (a rule is not independently callable, so there is no `grant execute on rule`) and **no `ReturnType`** despite gen declaring one beside `MicroflowReturnType`. Two keys only a reference document catches, both invisible to `mx check`: `ExportLevel` (Studio Pro writes "Hidden" on every rule) and `Flows` (written as the bare marker even when empty — a `MandatoryLists` entry). Rules are catalog objects and their bodies are walked for references, which together stop a microflow called only from a rule reading as dead. The body restrictions are refused at check time by the same function `exec` calls, each measured: create/change/delete/commit/rollback and client or web-service activities are **CE0009**, a non-Boolean/enum return is **CE0103 + CE0139**. Authoring is modelsdk-only; legacy refuses. See `.claude/skills/mendix/write-rules.md` - Menu documents (CREATE OR MODIFY/DESCRIBE/DROP MENU): standalone `Menus$MenuDocument`, the reusable menu a menu widget points at (Atlas_Core's `Phone_Menu`/`Tablet_Menu`) — **not** the menu inside a navigation profile, though both are built from the same items, so the item syntax is shared with `CREATE NAVIGATION`'s `MENU (...)` block. DESCRIBE is round-trippable. Written through gen+codec, which is load-bearing: Studio Pro's menu documents carry typed-array marker **3** on the item collection and each item's sub-items (the codec default), while the navigation writers hand-build items with marker **1** — unverified whether that is a latent navigation bug or a real difference, so navigation is left alone. Authoring is modelsdk-only; legacy refuses. Two traps: a menu item cannot open a page with required parameters (**CE1571**), and only `Forms$IconCollectionIcon` round-trips (glyph/image icons are flagged by DESCRIBE, not dropped silently) - Regular expressions (LIST/DESCRIBE/CREATE [OR MODIFY]/DROP REGULAR EXPRESSION): named patterns that attribute validation rules reference **by qualified name**, which is why they are documents. `modelsdk/gen` is **wrong** about the pattern's key — it binds `RegEx` where every Studio Pro document stores `Expression` (`generated/metamodel` agrees with the documents), so both engines share one raw-BSON codec in `mdl/regularexpressions`; a reader keyed on gen's name returns an empty pattern for every real document. Pinned against five Studio Pro-authored documents (Email Connector 6.4.2, Community Commons 11.5.1). Mendix validates with .NET's engine, so a pattern Go's RE2 cannot compile (lookaround — the Email Connector ships one) is stored unchanged and reported "not verifiable", never "invalid". A `validate` edge into `CATALOG.REFS` makes `show references to ` list the entities using it - Validation rules (CREATE VALIDATION RULE): binds a **regex** or a **range** to one attribute — `create validation rule for Mod.Entity.Attr regex Mod.Pattern feedback '…'`. The rule is anonymous and entity-scoped, so the statement names the attribute; re-running it replaces the rule of the same type and leaves the attribute's others alone. Unlocked by a `STORAGE-NAME OVERRIDE` in `modelsdk/gen`: it bound `RegularExpression` where Studio Pro stores `RegExIdentifier`, and the control (same script, key reverted) fails **CE0135 "No regular expression specified"** while the fixed one is 0 errors on mxbuild 11.13 with `RegExIdentifier` on disk. Range bounds are inclusive and map to Mendix's only three kinds (`from X to Y`/`from X`/`to Y` → Between/GreaterThanOrEqualTo/SmallerThanOrEqualTo); there is no strict `<`/`>`, and the old grammar's forms for it — plus an EXPRESSION rule type Mendix does not have and an inline regex literal — were removed, having never had a visitor or handler. Required/Unique stay attribute constraints (`not null error '…'` / `unique error '…'`), not a second spelling here. Rewriting an entity carrying **MaxLength or EqualsTo** is **refused** on both engines rather than silently downgraded to Required; that round trip was lossy and `mx check` stayed green, because a Required rule is valid. Both engines carry each rule's payload on READ (`ruleInfoFromGen` / `parseValidationRuleInfo`), which is what makes the refusal narrow instead of covering all of RegEx and Range — and what lets a **range bounded by another attribute** survive a rewrite even though MDL cannot author one (`describe entity` marks it with a comment rather than rendering it wrong). A rule whose payload did not survive the read is refused as firmly as an unknown type: a bare RuleInfo of the right `$Type` constrains nothing, which is the same silent downgrade wearing the right name diff --git a/cmd/mxcli/init_claudemd.go b/cmd/mxcli/init_claudemd.go index 6d486010b..e29ce53d8 100644 --- a/cmd/mxcli/init_claudemd.go +++ b/cmd/mxcli/init_claudemd.go @@ -451,6 +451,7 @@ func generateClaudeMD(projectName, mprFile string) string { w("|-------|--------|\n") w("| database-connections | External database connections (PostgreSQL, Oracle) |\n") w("| rest-client | REST API consumption |\n") + w("| mock-rest-apis | Mocking a REST dependency (Prism, forward proxy, constant swap) |\n") w("| java-actions | Custom Java actions |\n") w("| odata-data-sharing | OData services and external entities |\n") w("\n") diff --git a/cmd/mxcli/syntax/features_microflow.go b/cmd/mxcli/syntax/features_microflow.go index 4211c86e8..e8360a840 100644 --- a/cmd/mxcli/syntax/features_microflow.go +++ b/cmd/mxcli/syntax/features_microflow.go @@ -198,6 +198,33 @@ func init() { SeeAlso: []string{"microflow.create"}, }) + Register(SyntaxFeature{ + Path: "microflow.rule", + Summary: "CREATE RULE — reusable decision logic, callable only from a decision", + Keywords: []string{ + "rule", "create rule", "list rules", "describe rule", "drop rule", + "business rule", "decision logic", "reusable condition", + }, + Syntax: "CREATE [OR MODIFY] RULE Module.Name ($Param: Type)\n" + + "RETURNS Boolean | enum Module.Enum\n[FOLDER 'path']\nBEGIN\n \nEND;\n\n" + + "LIST RULES [IN Module];\nDESCRIBE RULE Module.Name;\n" + + "DROP RULE Module.Name;\nMOVE RULE Module.Name TO FOLDER 'path';\n\n" + + "A rule is called from a decision and nowhere else:\n" + + " IF Module.Rule_Name(Param = $Value) THEN ... END IF;\n\n" + + "The return type is mandatory and must be Boolean or an enumeration\n" + + "(mxbuild: CE0103/CE0139). A rule may not create, change, delete, commit\n" + + "or roll back objects, talk to the client, or call a web service\n" + + "(mxbuild: CE0009) — `mxcli check` refuses these before the build.\n\n" + + "There is no GRANT EXECUTE ON RULE: a rule is not independently callable,\n" + + "so its document carries no module-role security.", + Example: "create or modify rule Sales.Rule_IsSolvent ($pCustomer: Sales.Customer)\n" + + "returns Boolean\nfolder 'Rules'\nbegin\n return $pCustomer/Balance >= 0;\nend\n/\n\n" + + "create or modify microflow Sales.MF_Screen ($pCustomer: Sales.Customer)\nbegin\n" + + " if Sales.Rule_IsSolvent(pCustomer = $pCustomer) then\n return;\n" + + " else\n return;\n end if;\nend\n/", + SeeAlso: []string{"microflow.create", "microflow.control-flow"}, + }) + Register(SyntaxFeature{ Path: "microflow.validation", Summary: "Show validation feedback on object attributes", diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index 988560c17..857dc91be 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -110,7 +110,7 @@ func init() { "button style", "primary", "danger", "success", "icon", "linkbutton", "link button", }, - Syntax: "Action: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $val)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\nIcon: 'Module.IconCollection.IconName' -- e.g. 'Atlas_Core.Atlas_Filled.pencil'\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", + Syntax: "Action: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\nIcon: 'Module.IconCollection.IconName' -- e.g. 'Atlas_Core.Atlas_Filled.pencil'\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", Example: "ACTIONBUTTON btnSave (Caption: 'Save', Action: SAVE_CHANGES, ButtonStyle: Primary)\nACTIONBUTTON btnEdit (Caption: 'Edit',\n Action: SHOW_PAGE Module.EditPage(Item: $currentObject))\nLINKBUTTON btnDelete (Caption: 'Delete', Action: DELETE,\n Icon: 'Atlas_Core.Atlas_Filled.pencil')", SeeAlso: []string{"page.widgets"}, }) diff --git a/cmd/mxcli/syntax/features_security.go b/cmd/mxcli/syntax/features_security.go index 7481baf8b..3a78c72b2 100644 --- a/cmd/mxcli/syntax/features_security.go +++ b/cmd/mxcli/syntax/features_security.go @@ -91,14 +91,36 @@ func init() { Register(SyntaxFeature{ Path: "security.project-security", - Summary: "Set project security level and demo user toggle", + Summary: "Set project security level, demo user and guest access toggles", Keywords: []string{ "project security", "security level", "prototype", "production", "off", }, Syntax: "ALTER PROJECT SECURITY LEVEL OFF|PROTOTYPE|PRODUCTION;\nALTER PROJECT SECURITY DEMO USERS ON|OFF;", Example: "ALTER PROJECT SECURITY LEVEL PRODUCTION;\nALTER PROJECT SECURITY DEMO USERS OFF;", - SeeAlso: []string{"security.demo-user"}, + SeeAlso: []string{"security.demo-user", "security.guest-access"}, + }) + + Register(SyntaxFeature{ + Path: "security.guest-access", + Summary: "Enable anonymous (guest) access and pick the role anonymous visitors get", + Keywords: []string{ + "guest access", "anonymous", "anonymous users", "public", + "unauthenticated", "guest user role", "CE0133", + }, + Syntax: "ALTER PROJECT SECURITY GUEST ACCESS ON ROLE ;\n" + + "ALTER PROJECT SECURITY GUEST ACCESS ON; -- only when a role is already configured\n" + + "ALTER PROJECT SECURITY GUEST ACCESS OFF; -- keeps the stored role\n" + + "\n" + + "-- The role is what anonymous visitors get, so its entity access IS the app's\n" + + "-- public surface. Mendix requires one: guest access with no role fails the\n" + + "-- build (CE0133), so ON is refused unless a role is given or already stored.\n" + + "-- Mendix does not check the role exists, so mxcli does — an unknown role\n" + + "-- would build cleanly and leave visitors with nothing.", + Example: "CREATE USER ROLE Anonymous (Shop.Viewer, System.User);\n" + + "ALTER PROJECT SECURITY GUEST ACCESS ON ROLE Anonymous;\n" + + "GRANT Anonymous ON Shop.Product (read *);", + SeeAlso: []string{"security.user-role", "security.project-security"}, }) Register(SyntaxFeature{ diff --git a/cmd/mxcli/testrunner/client.go b/cmd/mxcli/testrunner/client.go index f2f7b3c9d..9060ca9e2 100644 --- a/cmd/mxcli/testrunner/client.go +++ b/cmd/mxcli/testrunner/client.go @@ -173,6 +173,11 @@ func toResult(tc TestCase, rr *runResponse) TestResult { case strings.HasPrefix(rr.Result, verdictFailPrefix): res.Status = StatusFail res.Message = strings.TrimPrefix(rr.Result, verdictFailPrefix) + case strings.HasPrefix(rr.Result, verdictSetupPrefix): + // The setup threw, so the test never ran — an ERROR, not a FAIL. + res.Status = StatusError + res.Message = "setup microflow " + strings.TrimPrefix(rr.Result, verdictSetupPrefix) + + " failed, so the test did not run" default: res.Status = StatusError res.Message = fmt.Sprintf("unrecognised verdict from the test microflow: %q", truncate(rr.Result, 200)) diff --git a/cmd/mxcli/testrunner/expect.go b/cmd/mxcli/testrunner/expect.go index 9b345d2d5..8ae3c94e3 100644 --- a/cmd/mxcli/testrunner/expect.go +++ b/cmd/mxcli/testrunner/expect.go @@ -26,6 +26,23 @@ type Expect struct { // the failure message. It is empty when no such expression can be derived // without guessing the operand's type — see actualExpr. Actual string + // Aggregates are the list aggregates Condition refers to by variable. They + // have to be computed by an activity before the condition is evaluated — + // see ExpectAggregate. + Aggregates []ExpectAggregate +} + +// ExpectAggregate is one list aggregate lifted out of an @expect condition. +// +// `count($List)` is not a Mendix expression function — counting a list is an +// Aggregate list activity, so it cannot appear in the decision that evaluates +// the assertion. The generators emit `$Var = COUNT($List);` ahead of that +// decision and the condition refers to $Var, which is the same thing the author +// would write by hand. +type ExpectAggregate struct { + Var string // generated variable holding the result, e.g. "$mxtest_count_Brands" + Op string // MDL aggregate keyword, e.g. "COUNT" + List string // the list variable being aggregated, e.g. "$Brands" } // ParseExpect parses one @expect annotation body. @@ -56,12 +73,50 @@ func ParseExpect(raw string) (Expect, error) { } return Expect{ - Raw: raw, - Condition: node.render(), - Actual: actualExpr(node), + Raw: raw, + Condition: node.render(), + Actual: actualExpr(node), + Aggregates: collectAggregates(node), }, nil } +// collectAggregates returns the aggregates in the expression, in source order +// and without repeats. Two assertions counting the same list share one +// variable, so the activity is emitted once. +func collectAggregates(n expectNode) []ExpectAggregate { + var ( + out []ExpectAggregate + seen = map[string]bool{} + walk func(expectNode) + ) + walk = func(n expectNode) { + switch e := n.(type) { + case *expectAggregate: + if !seen[e.Var] { + seen[e.Var] = true + out = append(out, ExpectAggregate{Var: e.Var, Op: e.Op, List: e.List}) + } + case *expectBinary: + walk(e.Left) + walk(e.Right) + case *expectUnary: + walk(e.Operand) + case *expectParen: + walk(e.Inner) + case *expectCall: + for _, a := range e.Args { + walk(a) + } + case *expectIfThenElse: + walk(e.Cond) + walk(e.Then) + walk(e.Else) + } + } + walk(n) + return out +} + // isAssertionShaped reports whether the expression can be a pass/fail condition. // // A comparison, a logical operator, a Boolean-returning call and a bare variable @@ -216,6 +271,20 @@ func (e *expectCall) kind() exprcheck.TypeKind { return exprcheck.KindUnknown } +// expectAggregate is a list aggregate the generators hoist into an activity. It +// renders as the variable that activity assigns, so from the condition's point +// of view it is an ordinary variable. +type expectAggregate struct { + Var string + Op string + List string +} + +func (e *expectAggregate) render() string { return e.Var } + +// Mendix's Aggregate list activity returns a Long for count. +func (e *expectAggregate) kind() exprcheck.TypeKind { return exprcheck.KindLong } + type expectParen struct{ Inner expectNode } func (e *expectParen) render() string { return "(" + e.Inner.render() + ")" } @@ -573,6 +642,9 @@ func (p *expectParser) parseIdentPrimary() (expectNode, error) { // qualified names, not calls — so an unknown name is an error, not a // user-defined function. if p.toks[p.pos+1].Kind == exprcheck.TokLParen { + if isListAggregate(t.Text) { + return p.parseAggregate() + } return p.parseCall() } @@ -587,6 +659,63 @@ func (p *expectParser) parseIdentPrimary() (expectNode, error) { return &expectVar{Text: name}, nil } +// listAggregates are Mendix's list aggregates. None of them is an expression +// function — they are Aggregate list *activities* — so a bare one in an @expect +// used to be rejected as "not a Mendix expression function", which is true and +// tells the author nothing about what to do instead. +var listAggregates = map[string]bool{ + "count": true, "sum": true, "average": true, "minimum": true, "maximum": true, +} + +func isListAggregate(name string) bool { return listAggregates[strings.ToLower(name)] } + +// parseAggregate parses `count($List)`. +// +// Only count is accepted. The other four aggregate an attribute over the list +// (`SUM($list.Amount)`), which needs the attribute's type to render a +// comparison, so they are refused with the helper-microflow workaround rather +// than guessed at. +func (p *expectParser) parseAggregate() (expectNode, error) { + nameTok := p.next() + name := strings.ToLower(nameTok.Text) + p.next() // consume '(' + + if name != "count" { + return nil, p.errorAt(nameTok, + "%s() aggregates an attribute over a list, which a test assertion cannot do "+ + "on its own — call a microflow that returns the %s and assert on its "+ + "result (count($list) is supported here)", name, name) + } + + arg := p.peek() + if arg.Kind != exprcheck.TokDollarIdent { + return nil, p.errorAt(arg, "count() counts a list variable, as in count($MyList)") + } + node, err := p.parseVariablePath() + if err != nil { + return nil, err + } + list := node.render() + if strings.ContainsAny(list, "/.") { + return nil, p.errorAt(arg, + "count() counts a list variable, not the path %s — retrieve the list into a "+ + "variable first", list) + } + if p.peek().Kind != exprcheck.TokRParen { + return nil, p.errorAt(p.peek(), "expected a closing parenthesis for count()") + } + p.next() + + return &expectAggregate{ + // Named after the list so two assertions over the same list share one + // activity, and prefixed so it cannot collide with a test's own + // variables. + Var: "$mxtest_count_" + strings.TrimPrefix(list, "$"), + Op: "COUNT", + List: list, + }, nil +} + func (p *expectParser) parseCall() (expectNode, error) { nameTok := p.next() name := nameTok.Text diff --git a/cmd/mxcli/testrunner/expect_count_test.go b/cmd/mxcli/testrunner/expect_count_test.go new file mode 100644 index 000000000..252af5537 --- /dev/null +++ b/cmd/mxcli/testrunner/expect_count_test.go @@ -0,0 +1,203 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testrunner + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// #927 bug 2: `@expect count($Brands) = 999` on a body that only retrieves the +// list reported PASS against an empty table, for any value of 999. +// +// count() is not a Mendix expression function — counting a list is an Aggregate +// list activity — so the condition could never be compiled into the decision +// that evaluates the assertion. The annotation was dropped, and a test with no +// assertions passes as long as its body does not throw. +// +// The aggregate is now lifted into the activity the author would have written by +// hand, so the assertion is really evaluated. +func TestParseExpectCountIsHoistedToAnAggregate(t *testing.T) { + exp, err := ParseExpect("count($Brands) = 5") + if err != nil { + t.Fatalf("ParseExpect: %v", err) + } + if len(exp.Aggregates) != 1 { + t.Fatalf("got %d aggregate(s), want 1: %+v", len(exp.Aggregates), exp) + } + agg := exp.Aggregates[0] + if agg.Op != "COUNT" || agg.List != "$Brands" { + t.Errorf("aggregate = %+v, want COUNT over $Brands", agg) + } + if exp.Condition != agg.Var+" = 5" { + t.Errorf("Condition = %q, want the aggregate variable compared with 5", exp.Condition) + } + if exp.Raw != "count($Brands) = 5" { + t.Errorf("Raw = %q, want the annotation as written (it is the failure message)", exp.Raw) + } + if exp.Actual != "toString("+agg.Var+")" { + t.Errorf("Actual = %q, want the observed count rendered for the failure message", exp.Actual) + } +} + +// Two assertions over the same list share one variable, so the list is counted +// once — and two different lists get two variables. +func TestParseExpectCountVariablesAreStableAndDistinct(t *testing.T) { + both, err := ParseExpect("count($Brands) > 0 and count($Types) = 4") + if err != nil { + t.Fatalf("ParseExpect: %v", err) + } + if len(both.Aggregates) != 2 { + t.Fatalf("got %d aggregate(s), want 2: %+v", len(both.Aggregates), both.Aggregates) + } + if both.Aggregates[0].Var == both.Aggregates[1].Var { + t.Errorf("two lists share a variable: %+v", both.Aggregates) + } + + same, err := ParseExpect("count($Brands) > 0 and count($Brands) < 10") + if err != nil { + t.Fatalf("ParseExpect: %v", err) + } + if len(same.Aggregates) != 1 { + t.Errorf("counting one list twice produced %d aggregates, want 1", len(same.Aggregates)) + } +} + +// The remaining aggregates need an attribute to aggregate over, which an +// assertion cannot supply. They are refused — and the message has to name the +// way out, because the old one ("not a Mendix expression function") described +// the implementation rather than the fix. +func TestParseExpectOtherAggregatesAreRefusedWithGuidance(t *testing.T) { + for _, name := range []string{"sum", "average", "minimum", "maximum"} { + _, err := ParseExpect(name + "($Orders) = 10") + if err == nil { + t.Errorf("%s(): want an error, got none", name) + continue + } + if !strings.Contains(err.Error(), "microflow") { + t.Errorf("%s(): error %q does not point at the helper-microflow workaround", + name, err) + } + } +} + +// count() over anything but a list variable cannot be turned into the activity, +// so it is an error rather than a silently dropped assertion. +func TestParseExpectCountRejectsNonListArguments(t *testing.T) { + for _, raw := range []string{ + "count($order/Lines) = 2", + "count('literal') = 1", + "count() = 0", + } { + if _, err := ParseExpect(raw); err == nil { + t.Errorf("%q: want an error, got none", raw) + } + } +} + +// A bare count is a value, not a pass/fail condition — the same rule that +// rejects `@expect $result` without a comparison. +func TestParseExpectBareCountIsNotACondition(t *testing.T) { + if _, err := ParseExpect("count($Brands)"); err == nil { + t.Error("want an error: a count is a value, not a condition") + } +} + +// The end of the chain: the generated microflow must compute the count before +// the decision that reads it, and it must parse. +func TestGenerateTestFlowsEmitsCountAggregate(t *testing.T) { + suite := &TestSuite{ + Name: "counts", + Tests: []TestCase{{ + ID: "test_1", + Name: "exactly 5 brands are seeded", + MDL: "retrieve $Brands from eShop.CatalogBrand;", + Expects: []Expect{expectOf("count($Brands) = 5")}, + Cleanup: CleanupNone, + }}, + } + + mdl := GenerateTestFlows(suite) + agg := suite.Tests[0].Expects[0].Aggregates[0] + activity := agg.Var + " = COUNT($Brands);" + if !strings.Contains(mdl, activity) { + t.Fatalf("generated flow is missing %q:\n%s", activity, mdl) + } + if strings.Index(mdl, activity) > strings.Index(mdl, "IF "+agg.Var) { + t.Errorf("the count is computed after the decision that reads it:\n%s", mdl) + } + if !strings.Contains(mdl, "retrieve $Brands from eShop.CatalogBrand;") { + t.Errorf("the body was dropped:\n%s", mdl) + } + if _, errs := visitor.Build(mdl); len(errs) > 0 { + t.Fatalf("generated flow should parse, got %v\n%s", errs[0], mdl) + } +} + +// The monolithic runner compiles every test into one microflow, so a generated +// aggregate variable has to be renamed per test exactly like the body's own — +// otherwise two tests counting a list of the same name declare it twice. +func TestGenerateTestRunnerRenamesAggregateVariables(t *testing.T) { + one := expectOf("count($List) = 1") + two := expectOf("count($List) = 2") + suite := &TestSuite{ + Name: "counts", + Tests: []TestCase{ + {ID: "test_1", Name: "first", MDL: "$List = CREATE LIST OF MfTest.Product;", Expects: []Expect{one}}, + {ID: "test_2", Name: "second", MDL: "$List = CREATE LIST OF MfTest.Product;", Expects: []Expect{two}}, + }, + } + + mdl := GenerateTestRunner(suite) + for _, want := range []string{ + "$mxtest_count_List_1 = COUNT($List_1);", + "$mxtest_count_List_2 = COUNT($List_2);", + } { + if !strings.Contains(mdl, want) { + t.Errorf("missing %q:\n%s", want, mdl) + } + } + if strings.Contains(mdl, "$mxtest_count_List =") { + t.Errorf("an un-suffixed aggregate variable collides across tests:\n%s", mdl) + } + if _, errs := visitor.Build(mdl); len(errs) > 0 { + t.Fatalf("generated runner should parse, got %v\n%s", errs[0], mdl) + } +} + +// Fail-closed control, from the file down: an aggregate this package cannot +// compile must make the test an ERROR and must not produce a microflow. That is +// what stopped bug 2's false PASS — a dropped assertion is indistinguishable +// from a passing one — and it has to keep holding for the aggregates count() +// support does not cover. +func TestUncompilableAggregateExpectIsAnErrorNotAPass(t *testing.T) { + tests, err := parseMDLTests(`/** + * @test asserts something impossible + * @expect sum($Orders) = 1 + */ +retrieve $Orders from eShop.Order; +/ +`, "bogus.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 1 { + t.Fatalf("got %d test(s), want 1", len(tests)) + } + tc := tests[0] + if len(tc.AssertionErrors) == 0 { + t.Fatal("the assertion was dropped silently — the test would report PASS") + } + if tc.AssertionCount() != 0 { + t.Errorf("AssertionCount = %d, want 0: nothing here can be evaluated", tc.AssertionCount()) + } + res, ok := assertionErrorResult(tc) + if !ok || res.Status != StatusError { + t.Errorf("result = %+v (ok=%v), want an ERROR", res, ok) + } + if mdl := GenerateTestFlows(&TestSuite{Name: "s", Tests: []TestCase{tc}}); strings.Contains(mdl, "Test_test_1") { + t.Errorf("a test that cannot assert got a microflow:\n%s", mdl) + } +} diff --git a/cmd/mxcli/testrunner/generator.go b/cmd/mxcli/testrunner/generator.go index a7ccf081b..d799f5596 100644 --- a/cmd/mxcli/testrunner/generator.go +++ b/cmd/mxcli/testrunner/generator.go @@ -55,6 +55,7 @@ func writeTestBlock(b *strings.Builder, tc TestCase, index int) { b.WriteString(fmt.Sprintf(" LOG INFO NODE 'MXTEST' 'MXTEST:RUN:%s:%s';\n", escapeMDLString(tc.ID), escapeMDLString(tc.Name))) b.WriteString(" SET $TestFailed = false;\n") + writeSetupBlock(b, tc) if tc.Throws != "" { writeThrowsTestBlock(b, tc, suffix) @@ -77,9 +78,13 @@ func writeTestBlock(b *strings.Builder, tc TestCase, index int) { // Generate assertion checks for @expect (with renamed variables) // Use flat IF blocks (no nesting) to avoid Mendix end-event issues if len(tc.Expects) > 0 { - for _, exp := range tc.Expects { - renamedExp := renameExpect(exp, varNames, suffix) - writeExpectAssertion(b, tc.ID, renamedExp) + renamed := make([]Expect, len(tc.Expects)) + for i, exp := range tc.Expects { + renamed[i] = renameExpect(exp, varNames, suffix) + } + writeExpectAggregates(b, " ", renamed) + for _, exp := range renamed { + writeExpectAssertion(b, tc.ID, exp) } } else if tc.Throws == "" { // No expectations — just check it didn't throw @@ -89,6 +94,25 @@ func writeTestBlock(b *strings.Builder, tc TestCase, index int) { } } +// writeSetupBlock writes a test's @setup calls into the monolithic runner. +// +// The counterpart of writeSetupCalls, differing only in how it reports: this +// runner has no returned verdict to carry an outcome, so a failed setup goes out +// as an ERROR line on the log protocol. It ends the runner flow the same way a +// throwing test body does — that is this runner's existing behaviour, and one of +// the reasons the endpoint runner exists. +func writeSetupBlock(b *strings.Builder, tc TestCase) { + for _, flow := range tc.Setups { + fmt.Fprintf(b, " CALL MICROFLOW %s() ON ERROR {\n", flow) + fmt.Fprintf(b, " LOG ERROR NODE 'MXTEST' 'MXTEST:ERROR:%s:Setup failed: %s';\n", + escapeMDLString(tc.ID), escapeMDLString(flow)) + b.WriteString(" SET $TestFailed = true;\n") + b.WriteString(" SET $AllPassed = false;\n") + b.WriteString(" RETURN $AllPassed;\n") + b.WriteString(" };\n") + } +} + // writeThrowsTestBlock generates code for a test that expects an error. func writeThrowsTestBlock(b *strings.Builder, tc TestCase, suffix string) { didThrowVar := "$DidThrow" + suffix @@ -188,9 +212,23 @@ func renameVariables(mdl string, names map[string]bool, suffix string) string { // rendered condition and the actual-value expression, which is why both are kept // as text rather than as a tree. func renameExpect(exp Expect, names map[string]bool, suffix string) Expect { + // An aggregate's own variable is generated, so it is not in the body's name + // set — but it is declared inside this one shared microflow, so two tests + // counting a list of the same name would declare it twice. It is renamed + // with everything else. + all := names + if len(exp.Aggregates) > 0 { + all = make(map[string]bool, len(names)+len(exp.Aggregates)) + for n := range names { + all[n] = true + } + for _, agg := range exp.Aggregates { + all[strings.TrimPrefix(agg.Var, "$")] = true + } + } rename := func(src string) string { return varPattern.ReplaceAllStringFunc(src, func(match string) string { - if names[match[1:]] { + if all[match[1:]] { return match + suffix } return match @@ -199,6 +237,16 @@ func renameExpect(exp Expect, names map[string]bool, suffix string) Expect { renamed := exp renamed.Condition = rename(exp.Condition) renamed.Actual = rename(exp.Actual) + if len(exp.Aggregates) > 0 { + renamed.Aggregates = make([]ExpectAggregate, len(exp.Aggregates)) + for i, agg := range exp.Aggregates { + renamed.Aggregates[i] = ExpectAggregate{ + Var: rename(agg.Var), + Op: agg.Op, + List: rename(agg.List), + } + } + } return renamed } diff --git a/cmd/mxcli/testrunner/generator_endpoint.go b/cmd/mxcli/testrunner/generator_endpoint.go index e5cf718a9..2adfdda78 100644 --- a/cmd/mxcli/testrunner/generator_endpoint.go +++ b/cmd/mxcli/testrunner/generator_endpoint.go @@ -14,6 +14,11 @@ import ( const ( verdictPass = "PASS" verdictFailPrefix = "FAIL:" + // verdictSetupPrefix is followed by the setup microflow that threw. It is a + // third outcome on purpose: the test never ran, so it neither passed nor + // failed, and reporting a broken fixture as a FAIL blames the code under + // test for it. + verdictSetupPrefix = "SETUP:" ) // GenerateTestFlows returns the MDL declaring one microflow per test case. @@ -54,6 +59,10 @@ func writeTestFlow(b *strings.Builder, tc TestCase) { b.WriteString("BEGIN\n") fmt.Fprintf(b, " DECLARE $Verdict String = '%s';\n", verdictPass) + // Before the body, and before a @throws test pre-sets its failing verdict: + // the fixture is not the thing expected to throw. + writeSetupCalls(b, tc) + if tc.Throws != "" { writeThrowsFlowBody(b, tc) } else { @@ -65,6 +74,22 @@ func writeTestFlow(b *strings.Builder, tc TestCase) { b.WriteString("/\n") } +// writeSetupCalls writes the @setup microflow calls that precede a test's body. +// +// Each is a plain call — a fixture is a microflow, so there is nothing to +// resolve and nothing to declare — with a handler that returns the SETUP verdict +// and stops. Continuing into a test whose preconditions were not established +// produces an assertion failure that says nothing about the code under test. +func writeSetupCalls(b *strings.Builder, tc TestCase) { + for _, flow := range tc.Setups { + fmt.Fprintf(b, " CALL MICROFLOW %s() ON ERROR {\n", flow) + fmt.Fprintf(b, " SET $Verdict = '%s';\n", + escapeMDLString(verdictSetupPrefix+flow)) + b.WriteString(" RETURN $Verdict;\n") + b.WriteString(" };\n") + } +} + // writeExpectFlowBody writes the body of a normal test: run the MDL, then check // each @expect. An error during the body short-circuits to a FAIL verdict. func writeExpectFlowBody(b *strings.Builder, tc TestCase) { @@ -73,11 +98,28 @@ func writeExpectFlowBody(b *strings.Builder, tc TestCase) { b.WriteString(line) b.WriteString("\n") } + writeExpectAggregates(b, " ", tc.Expects) for _, exp := range tc.Expects { writeExpectCheck(b, exp) } } +// writeExpectAggregates emits the Aggregate list activities the assertions need, +// after the body has produced the lists and before the first decision reads +// them. One activity per variable, however many assertions refer to it. +func writeExpectAggregates(b *strings.Builder, indent string, expects []Expect) { + seen := map[string]bool{} + for _, exp := range expects { + for _, agg := range exp.Aggregates { + if seen[agg.Var] { + continue + } + seen[agg.Var] = true + fmt.Fprintf(b, "%s%s = %s(%s);\n", indent, agg.Var, agg.Op, agg.List) + } + } +} + // writeThrowsFlowBody writes the body of an @throws test: the verdict starts as // a failure and only the error handler can clear it, so a body that completes // without throwing fails — which is the point of the annotation. diff --git a/cmd/mxcli/testrunner/parser.go b/cmd/mxcli/testrunner/parser.go index 2895dd1b2..3d756000b 100644 --- a/cmd/mxcli/testrunner/parser.go +++ b/cmd/mxcli/testrunner/parser.go @@ -25,11 +25,13 @@ type TestCase struct { // report a pass. AssertionErrors []string Verify []Verify // @verify OQL post-conditions - Setup string // @setup block reference - Cleanup string // @cleanup strategy ("rollback" or "none") - Throws string // @throws expected error message - SourceFile string // Original file path - Line int // Line number in source file + // Setups are the microflows called before the test's own statements, in + // order: the file header's first, then the test's own. See writeSetupCalls. + Setups []string + Cleanup string // @cleanup strategy ("rollback" or "none") + Throws string // @throws expected error message + SourceFile string // Original file path + Line int // Line number in source file } // AssertionCount reports how many assertions the test actually makes. @@ -126,14 +128,17 @@ func parseMDLTests(content string, sourcePath string) ([]TestCase, error) { blocks := splitTestBlocks(content) var tests []TestCase - for i, block := range blocks { - block = strings.TrimSpace(block) - if block == "" { - continue - } + fileSetups, err := headerSetups(content) + if err != nil { + return nil, fmt.Errorf("%s: %w", sourcePath, err) + } + for _, block := range blocks { // Extract javadoc comment and MDL body - doc, body, line := extractDocAndBody(block, content) + doc, body, line, err := extractDocAndBody(block) + if err != nil { + return nil, fmt.Errorf("%s: %w", sourcePath, err) + } if doc == "" { // No javadoc — skip this block (it's not a test) continue @@ -149,7 +154,10 @@ func parseMDLTests(content string, sourcePath string) ([]TestCase, error) { return nil, fmt.Errorf("%s: test %q: %w", sourcePath, annotations.Test, err) } - testID := fmt.Sprintf("test_%d", i+1) + // Number the tests, not the chunks: a chunk that holds only a file + // header is skipped, and numbering by chunk index left a gap where it + // had been — so the first test in a file could report as test_2. + testID := fmt.Sprintf("test_%d", len(tests)+1) tests = append(tests, TestCase{ ID: testID, Name: annotations.Test, @@ -157,11 +165,13 @@ func parseMDLTests(content string, sourcePath string) ([]TestCase, error) { Expects: annotations.Expects, AssertionErrors: annotations.AssertionErrors, Verify: annotations.Verify, - Setup: annotations.Setup, - Cleanup: annotations.Cleanup, - Throws: annotations.Throws, - SourceFile: sourcePath, - Line: line, + // The file's fixtures run before the test's own: they are the + // broader precondition, and a test's own setup may build on them. + Setups: append(append([]string{}, fileSetups...), annotations.Setups...), + Cleanup: annotations.Cleanup, + Throws: annotations.Throws, + SourceFile: sourcePath, + Line: line, }) } @@ -198,7 +208,10 @@ func parseMarkdownTests(content string, sourcePath string) ([]TestCase, error) { blockContent := strings.Join(blockLines, "\n") // Parse the block as a single test - doc, body, _ := extractDocAndBody(blockContent, blockContent) + doc, body, _, err := extractDocAndBody(testBlock{Text: blockContent, Line: blockStart}) + if err != nil { + return nil, fmt.Errorf("%s: %w", sourcePath, err) + } annotations := parseAnnotations(doc) if err := validateCleanup(annotations.Cleanup); err != nil { @@ -220,7 +233,7 @@ func parseMarkdownTests(content string, sourcePath string) ([]TestCase, error) { Expects: annotations.Expects, AssertionErrors: annotations.AssertionErrors, Verify: annotations.Verify, - Setup: annotations.Setup, + Setups: annotations.Setups, Cleanup: annotations.Cleanup, Throws: annotations.Throws, SourceFile: sourcePath, @@ -235,59 +248,227 @@ func parseMarkdownTests(content string, sourcePath string) ([]TestCase, error) { return tests, nil } +// headerSetups returns the @setup microflows declared in the file's header +// comment, which apply to every test in the file. +// +// The header is the file's first javadoc comment when it carries no @test. That +// is the one shape a header can have: it may sit in its own '/'-terminated +// chunk, or share a chunk with the first test (there is no '/' between them), +// and this reads the same thing either way. +// +// Declaring a fixture once for a file is what the annotation is for — otherwise +// it says no more than `call microflow X;` at the top of each body. But a header +// can only carry what a file-wide default can honour: @cleanup, @expect, @verify +// and @throws each describe one test's execution, and silently ignoring them +// here would be the same absent-annotation bug @setup is being fixed for. +func headerSetups(content string) ([]string, error) { + docs := scanDocComments(content, 1) + if len(docs) == 0 { + return nil, nil + } + a := parseAnnotations(docs[0].Text) + if a.Test != "" { + // The first comment is a test's own, so the file has no header. + return nil, nil + } + + var offending []string + if len(a.Expects) > 0 || len(a.AssertionErrors) > 0 { + offending = append(offending, "@expect") + } + if len(a.Verify) > 0 { + offending = append(offending, "@verify") + } + if a.Throws != "" { + offending = append(offending, "@throws") + } + if a.cleanupSet { + offending = append(offending, "@cleanup") + } + if len(offending) > 0 { + return nil, fmt.Errorf( + "the file header comment carries %s, which describes one test's execution and "+ + "cannot be a file-wide default — move it into that test's own doc comment "+ + "(a header may carry @setup)", strings.Join(offending, ", ")) + } + return a.Setups, nil +} + +// testBlock is one '/'-separated chunk of a test file, with the line its first +// character sits on. The line travels with the chunk because it cannot be +// recovered from the text afterwards: the previous implementation searched the +// whole file for the chunk's first 20 characters, which found the wrong chunk +// whenever two started alike and panicked on a chunk shorter than that. +type testBlock struct { + Text string + Line int +} + // splitTestBlocks splits MDL content on '/' delimiters (the microflow block terminator). -func splitTestBlocks(content string) []string { +func splitTestBlocks(content string) []testBlock { // Split on lines that are just '/' (the MDL block separator) - var blocks []string - var current strings.Builder + var blocks []testBlock + var current []string + start := 1 + lineNum := 0 scanner := bufio.NewScanner(strings.NewReader(content)) + flush := func() { + if len(current) > 0 { + blocks = append(blocks, testBlock{Text: strings.Join(current, "\n"), Line: start}) + current = nil + } + } + for scanner.Scan() { + lineNum++ line := scanner.Text() - trimmed := strings.TrimSpace(line) - if trimmed == "/" { - blocks = append(blocks, current.String()) - current.Reset() - } else { - if current.Len() > 0 { - current.WriteString("\n") - } - current.WriteString(line) + switch { + case strings.TrimSpace(line) == "/": + flush() + start = lineNum + 1 + case len(current) == 0 && strings.TrimSpace(line) == "": + // A blank line before the chunk's first content is not part of it; + // keeping it would report every test one line above where it is. + start = lineNum + 1 + default: + current = append(current, line) } } // Don't forget the last block (after the last '/' or if no '/' found) - if current.Len() > 0 { - blocks = append(blocks, current.String()) - } + flush() return blocks } -// extractDocAndBody separates the javadoc comment from the MDL body. -// Returns (docComment, body, lineNumber). -func extractDocAndBody(block string, fullContent string) (string, string, int) { - block = strings.TrimSpace(block) - - // Find /** ... */ pattern - docStart := strings.Index(block, "/**") - if docStart == -1 { - return "", block, 1 +// extractDocAndBody separates the test's javadoc comment from the MDL body. +// Returns (docComment, body, lineNumber, error). +// +// A chunk may open with several comments — a file-level header, a `--` note — +// before the test's own doc comment, because a header is not separated from the +// test below it by a '/'. The test's doc is the LAST comment in that leading +// run; everything after it is the body. +// +// Taking the FIRST `/**` instead is #927 bug 1: the header became the doc, it +// carried no @test, and the whole chunk — the real test with it — was dropped +// with no message. Scanning for the delimiters by raw substring search is bug +// 1b: a `--` line whose prose spelled them out was read as a doc comment, so +// describing the bug in a comment re-triggered it. +func extractDocAndBody(block testBlock) (string, string, int, error) { + docs := scanDocComments(block.Text, block.Line) + + // More than one @test in a chunk means a '/' separator is missing. Silently + // keeping one of them is what bug 1 did, and an author cannot tell a dropped + // test from a passing one, so this is refused rather than resolved. + var named []string + for _, d := range docs { + if name := parseAnnotations(d.Text).Test; name != "" { + named = append(named, name) + } + } + if len(named) > 1 { + return "", "", 0, fmt.Errorf( + "test %q is followed by another @test doc comment (%q) with no '/' separator "+ + "between them, so only one of the two could run: add a line containing "+ + "just '/' after the first test's statements", named[0], named[1]) } - docEnd := strings.Index(block[docStart:], "*/") - if docEnd == -1 { - return "", block, 1 + // The test's doc is the last comment of the leading run: a file-level header + // and the test's own doc live in the same chunk, in that order. + var doc *docComment + for i := range docs { + if docs[i].Leading { + doc = &docs[i] + } + } + if doc == nil { + return "", strings.TrimSpace(block.Text), block.Line, nil } - docEnd += docStart + 2 // include the */ + return doc.Text, strings.TrimSpace(block.Text[doc.End:]), doc.Line, nil +} - doc := block[docStart:docEnd] - body := strings.TrimSpace(block[docEnd:]) +// docComment is one `/** … */` comment found in a chunk. +type docComment struct { + Text string + Line int + End int // index just past the closing delimiter + // Leading is true when nothing but whitespace and other comments preceded + // it, which is where a test's doc comment sits. + Leading bool +} - // Estimate line number - line := 1 + strings.Count(fullContent[:strings.Index(fullContent, block[:20])], "\n") +// scanDocComments finds the javadoc comments in a chunk. +// +// It is a scanner rather than a substring search because the delimiters mean +// nothing outside a comment: `/**` written in the prose of a `--` line comment +// (#927 bug 1b) or inside a string literal is text, not a doc comment. +func scanDocComments(text string, startLine int) []docComment { + var ( + out []docComment + i int + line = startLine + seenBody bool + ) + for i < len(text) { + switch { + case text[i] == '\n': + line++ + i++ + case text[i] == ' ' || text[i] == '\t' || text[i] == '\r': + i++ + case strings.HasPrefix(text[i:], "--"): + // A line comment runs to the end of the line, delimiters and all. + if nl := strings.IndexByte(text[i:], '\n'); nl >= 0 { + i += nl + } else { + i = len(text) + } + case strings.HasPrefix(text[i:], "/*"): + end := strings.Index(text[i:], "*/") + if end == -1 { + // Unterminated. Whatever follows is not MDL either; leave it to + // the MDL parser to report against the real source. + return out + } + comment := text[i : i+end+2] + if strings.HasPrefix(comment, "/**") { + out = append(out, docComment{ + Text: comment, + Line: line, + End: i + len(comment), + Leading: !seenBody, + }) + } + line += strings.Count(comment, "\n") + i += len(comment) + case text[i] == '\'': + seenBody = true + i, line = skipMDLString(text, i, line) + default: + seenBody = true + i++ + } + } + return out +} - return doc, body, line +// skipMDLString advances past a single-quoted MDL string literal, which escapes +// a quote by doubling it and may span lines. +func skipMDLString(text string, i, line int) (int, int) { + for j := i + 1; j < len(text); j++ { + switch text[j] { + case '\'': + if j+1 < len(text) && text[j+1] == '\'' { + j++ + continue + } + return j + 1, line + case '\n': + line++ + } + } + return len(text), line } // annotations holds parsed javadoc annotations for a test block. @@ -296,9 +477,12 @@ type annotations struct { Expects []Expect AssertionErrors []string Verify []Verify - Setup string + Setups []string Cleanup string - Throws string + // cleanupSet records whether @cleanup was written, which the default value + // of Cleanup cannot express. Only the file-header check needs it. + cleanupSet bool + Throws string } var ( @@ -307,12 +491,19 @@ var ( // a line the pattern did not fit produced no assertion at all, and a test // with no assertions passes. Everything after @expect is now handed to // ParseExpect, which either compiles it or reports why it could not. - expectPattern = regexp.MustCompile(`@expect\s+(.+)`) - verifyPattern = regexp.MustCompile(`@verify\s+(.+)`) - testPattern = regexp.MustCompile(`@test\s+(.+)`) - setupPattern = regexp.MustCompile(`@setup\s+(\S+)`) - cleanupPattern = regexp.MustCompile(`@cleanup\s+(\S+)`) - throwsPattern = regexp.MustCompile(`@throws\s+'([^']*)'`) + // + // Every pattern is anchored to the start of the line, because an annotation + // is a javadoc tag and a tag opens its line. Matching one anywhere turned + // prose that quotes a tag into a real annotation: writing "`@expect $x = 1` + // in a sentence" gave the test an assertion nobody wrote, and "@cleanup none + // would apply here" changed the cleanup strategy. The leading `*` and its + // indentation are stripped before these run. + expectPattern = regexp.MustCompile(`^@expect\s+(.+)`) + verifyPattern = regexp.MustCompile(`^@verify\s+(.+)`) + testPattern = regexp.MustCompile(`^@test\s+(.+)`) + setupPattern = regexp.MustCompile(`^@setup\s+(\S+)`) + cleanupPattern = regexp.MustCompile(`^@cleanup\s+(\S+)`) + throwsPattern = regexp.MustCompile(`^@throws\s+'([^']*)'`) ) // parseAnnotations extracts test annotations from a javadoc comment. @@ -353,10 +544,11 @@ func parseAnnotations(doc string) annotations { } } if m := setupPattern.FindStringSubmatch(line); m != nil { - a.Setup = strings.TrimSpace(m[1]) + a.Setups = append(a.Setups, strings.TrimSpace(m[1])) } if m := cleanupPattern.FindStringSubmatch(line); m != nil { a.Cleanup = strings.TrimSpace(m[1]) + a.cleanupSet = true } if m := throwsPattern.FindStringSubmatch(line); m != nil { a.Throws = m[1] @@ -364,9 +556,32 @@ func parseAnnotations(doc string) annotations { } checkVerifyCleanup(&a) + checkThrowsExpect(&a) return a } +// checkThrowsExpect refuses an @expect on a test that expects an exception. +// +// @throws replaces the body's normal outcome: both generators emit the +// throws-shaped microflow, in which the verdict starts as a failure and only the +// error handler can clear it, and neither emits the @expect checks at all. The +// assertion was therefore never evaluated — while AssertionCount still counted +// it, so the test reported making two assertions and made one. Asserting on a +// return value the body was expected not to produce cannot be made to work, so +// it is refused rather than quietly ignored. +func checkThrowsExpect(a *annotations) { + if a.Throws == "" || len(a.Expects) == 0 { + return + } + for _, exp := range a.Expects { + a.AssertionErrors = append(a.AssertionErrors, fmt.Sprintf( + "@expect %s: this test also has @throws, so the body is expected to fail and "+ + "this assertion is never evaluated. Drop one of the two — assert on the "+ + "error with @throws, or on the result with @expect", exp.Raw)) + } + a.Expects = nil +} + // checkVerifyCleanup refuses a @verify on a test whose writes are rolled back. // // @verify asserts on rows the microflow wrote, and @cleanup rollback — the diff --git a/cmd/mxcli/testrunner/parser_annotation_scope_test.go b/cmd/mxcli/testrunner/parser_annotation_scope_test.go new file mode 100644 index 000000000..1d03d841a --- /dev/null +++ b/cmd/mxcli/testrunner/parser_annotation_scope_test.go @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testrunner + +import "testing" + +// An annotation is a javadoc tag: it opens the line. Matching one anywhere in a +// line made prose that quotes a tag into a real annotation — which is not a +// hypothetical, it is what the #927 repro file hit on its first run: a sentence +// reading `@expect count($Var) = N` on a bare retrieve` became that file's own +// assertion, and its error then reported against a test nobody had written. +func TestParseAnnotationsIgnoresTagsQuotedInProse(t *testing.T) { + a := parseAnnotations(`/** + * @test prose must not assert + * Writing ` + "`@expect $x = 1`" + ` in a sentence is documentation, and so is + * saying that @cleanup none would apply here, or that @throws 'boom' is how + * an error is expected, or referring to @verify select count(*) as n from M.E = 1. + * @expect $real = true + */`) + + if a.Test != "prose must not assert" { + t.Errorf("Test = %q", a.Test) + } + if len(a.Expects) != 1 || a.Expects[0].Raw != "$real = true" { + t.Errorf("Expects = %+v, want only the annotation that opens its line", a.Expects) + } + if len(a.AssertionErrors) != 0 { + t.Errorf("AssertionErrors = %v, want none — prose was parsed as an assertion", + a.AssertionErrors) + } + if a.Cleanup != CleanupRollback { + t.Errorf("Cleanup = %q, want the default: the prose mention must not set it", a.Cleanup) + } + if a.Throws != "" { + t.Errorf("Throws = %q, want empty", a.Throws) + } + if len(a.Verify) != 0 { + t.Errorf("Verify = %+v, want none", a.Verify) + } +} + +// False-positive control: the real spellings must all still be read, at every +// indentation a javadoc block uses. +func TestParseAnnotationsReadsTagsThatOpenTheirLine(t *testing.T) { + a := parseAnnotations(`/** + * @test still parsed + * @expect $r = 1 +@verify select count(*) as n from M.E = 1 + * @cleanup none + * @setup seed + */`) + + if a.Test != "still parsed" { + t.Errorf("Test = %q", a.Test) + } + if len(a.Expects) != 1 { + t.Errorf("Expects = %+v, want 1", a.Expects) + } + if len(a.Verify) != 1 { + t.Errorf("Verify = %+v, want 1", a.Verify) + } + if a.Cleanup != CleanupNone { + t.Errorf("Cleanup = %q, want none", a.Cleanup) + } + if len(a.Setups) != 1 || a.Setups[0] != "seed" { + t.Errorf("Setups = %v, want [seed]", a.Setups) + } +} + +// A single-line doc comment is the other shape a tag opens. +func TestParseAnnotationsSingleLineDoc(t *testing.T) { + a := parseAnnotations(`/** @test one-liner */`) + if a.Test != "one-liner" { + t.Errorf("Test = %q, want one-liner", a.Test) + } +} diff --git a/cmd/mxcli/testrunner/parser_leading_comment_test.go b/cmd/mxcli/testrunner/parser_leading_comment_test.go new file mode 100644 index 000000000..9fcb05c2a --- /dev/null +++ b/cmd/mxcli/testrunner/parser_leading_comment_test.go @@ -0,0 +1,218 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testrunner + +import ( + "strings" + "testing" +) + +// #927 bug 1: a file-level javadoc comment before the first test made that test +// vanish from the suite — not as a pass, not as a fail, not as an error. Every +// later test then reported under the number of the one above it. +// +// The cause was `extractDocAndBody` taking the FIRST `/**` in a '/'-separated +// chunk as the test's doc comment. A file header is not separated from the test +// below it by a '/', so both live in one chunk: the header became the doc, it +// carried no @test, and the whole chunk — the real test included — was dropped. +func TestParseMDLTestsLeadingFileComment(t *testing.T) { + content := `/** + * File-level description with no statement of its own. + */ + +/** + * @test exactly 5 brands are seeded + * @expect $BrandCount = 5 + * @cleanup none + */ +$BrandCount = CALL MICROFLOW eShop.QRY_CatalogBrandCount(); +/ + +/** + * @test exactly 4 types are seeded + * @expect $TypeCount = 4 + * @cleanup none + */ +$TypeCount = CALL MICROFLOW eShop.QRY_CatalogTypeCount(); +/ +` + tests, err := parseMDLTests(content, "repro.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 2 { + t.Fatalf("got %d test(s), want 2 — the header swallowed one: %+v", len(tests), tests) + } + if tests[0].Name != "exactly 5 brands are seeded" { + t.Errorf("tests[0].Name = %q, want the first test", tests[0].Name) + } + if len(tests[0].Expects) != 1 { + t.Errorf("tests[0] has %d @expect(s), want 1", len(tests[0].Expects)) + } + if tests[1].Name != "exactly 4 types are seeded" { + t.Errorf("tests[1].Name = %q, want the second test", tests[1].Name) + } +} + +// The reported line must be the test's own, not the header's. Before the fix the +// position came from a substring search for the chunk's first 20 characters, +// which found the header — so both tests in the file above reported line 5. +func TestParseMDLTestsReportsEachTestsOwnLine(t *testing.T) { + content := `-- a line comment header +/** + * @test first + */ +CALL MICROFLOW M.A(); +/ + +/** + * @test second + */ +CALL MICROFLOW M.B(); +/ +` + tests, err := parseMDLTests(content, "lines.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 2 { + t.Fatalf("got %d test(s), want 2", len(tests)) + } + for i, want := range []int{2, 8} { + if tests[i].Line != want { + t.Errorf("tests[%d] (%q) reported line %d, want %d", + i, tests[i].Name, tests[i].Line, want) + } + } +} + +// #927 bug 1b: the fusion came back when a `--` comment's PROSE spelled out the +// two javadoc delimiters — the workaround for bug 1 was to describe the bug in a +// line comment, and describing it re-triggered it. Comment delimiters were found +// by raw substring search, with no notion of already being inside a line comment. +func TestParseMDLTestsLineCommentMentioningJavadocDelimiters(t *testing.T) { + content := `-- Do not put a /** ... */ docblock at the top of this file. +/** + * @test the line comment above is prose, not a doc comment + * @expect $n = 1 + */ +$n = CALL MICROFLOW M.Count(); +/ +` + tests, err := parseMDLTests(content, "prose.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 1 { + t.Fatalf("got %d test(s), want 1 — the prose was read as a doc comment: %+v", + len(tests), tests) + } + if tests[0].Name != "the line comment above is prose, not a doc comment" { + t.Errorf("Name = %q", tests[0].Name) + } + if !strings.Contains(tests[0].MDL, "CALL MICROFLOW M.Count()") { + t.Errorf("MDL = %q, want the body below the doc comment", tests[0].MDL) + } +} + +// The body must start after the doc comment, not after the file header — a +// header-only chunk must not donate its text to the test below it. +func TestParseMDLTestsBodyExcludesLeadingComments(t *testing.T) { + content := `/** header */ +/** + * @test body boundary + */ +CALL MICROFLOW M.A(); +/ +` + tests, err := parseMDLTests(content, "body.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 1 { + t.Fatalf("got %d test(s), want 1", len(tests)) + } + if tests[0].MDL != "CALL MICROFLOW M.A();" { + t.Errorf("MDL = %q, want just the statement", tests[0].MDL) + } +} + +// Two @test doc comments in one chunk means a '/' separator is missing. Taking +// the last one would run the second test and drop the first exactly as bug 1 +// did, so this is refused instead: an author who cannot see a test cannot tell a +// dropped one from a passing one. +func TestParseMDLTestsRefusesTwoTestsInOneBlock(t *testing.T) { + content := `/** + * @test first, whose separator is missing + */ +CALL MICROFLOW M.A(); + +/** + * @test second + */ +CALL MICROFLOW M.B(); +/ +` + _, err := parseMDLTests(content, "missing-separator.test.mdl") + if err == nil { + t.Fatal("want an error naming the missing '/' separator, got none") + } + for _, want := range []string{"first, whose separator is missing", "/"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not mention %q", err, want) + } + } +} + +// False-positive control: a file with no leading comment at all must still parse +// exactly as before — the fix must not change where a plain test's doc, body or +// line come from. +func TestParseMDLTestsNoLeadingCommentUnchanged(t *testing.T) { + content := `/** + * @test plain + * @expect $r = true + */ +$r = CALL MICROFLOW M.A(); +/ +` + tests, err := parseMDLTests(content, "plain.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 1 { + t.Fatalf("got %d test(s), want 1", len(tests)) + } + tc := tests[0] + if tc.Name != "plain" || tc.Line != 1 || tc.MDL != "$r = CALL MICROFLOW M.A();" { + t.Errorf("got name=%q line=%d mdl=%q", tc.Name, tc.Line, tc.MDL) + } + if len(tc.Expects) != 1 { + t.Errorf("got %d @expect(s), want 1", len(tc.Expects)) + } +} + +// A chunk that is only a comment — a file header followed by its own '/' — is +// not a test and must not become an error either. +func TestParseMDLTestsStandaloneCommentBlock(t *testing.T) { + content := `/** + * Suite header, terminated on its own. + */ +/ + +/** + * @test the only test + */ +CALL MICROFLOW M.A(); +/ +` + tests, err := parseMDLTests(content, "standalone.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 1 { + t.Fatalf("got %d test(s), want 1", len(tests)) + } + if tests[0].ID != "test_1" { + t.Errorf("ID = %q, want test_1 — IDs number the tests, not the chunks", tests[0].ID) + } +} diff --git a/cmd/mxcli/testrunner/parser_throws_expect_test.go b/cmd/mxcli/testrunner/parser_throws_expect_test.go new file mode 100644 index 000000000..f80ff5116 --- /dev/null +++ b/cmd/mxcli/testrunner/parser_throws_expect_test.go @@ -0,0 +1,86 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testrunner + +import ( + "strings" + "testing" +) + +// @throws replaces the test body's normal outcome, so its @expect assertions are +// never emitted into the generated microflow. Counting them as assertions while +// evaluating none of them is the same silent gap that made a dropped @expect +// look like a passing one: the test reports "2 assertions" and makes one. +func TestExpectWithThrowsIsRefused(t *testing.T) { + a := parseAnnotations(`/** + * @test an error is expected + * @throws 'boom' + * @expect $result = 'never evaluated' + */`) + + if len(a.Expects) != 0 { + t.Errorf("Expects = %+v, want none kept", a.Expects) + } + if len(a.AssertionErrors) != 1 { + t.Fatalf("AssertionErrors = %v, want one explaining the combination", a.AssertionErrors) + } + msg := a.AssertionErrors[0] + for _, want := range []string{"@throws", "$result = 'never evaluated'"} { + if !strings.Contains(msg, want) { + t.Errorf("message %q does not mention %q", msg, want) + } + } + if a.Throws != "boom" { + t.Errorf("Throws = %q, want boom — the @throws itself still stands", a.Throws) + } +} + +// The ERROR has to reach the runner, not just the annotation struct: a test that +// cannot evaluate an assertion must not be generated or reported as a pass. +func TestExpectWithThrowsIsAnErrorEndToEnd(t *testing.T) { + tests, err := parseMDLTests(`/** + * @test an error is expected + * @throws 'boom' + * @expect $result = 'never evaluated' + */ +$result = CALL MICROFLOW M.Explode(); +/ +`, "throws.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 1 { + t.Fatalf("got %d test(s), want 1", len(tests)) + } + tc := tests[0] + if tc.AssertionCount() != 1 { + t.Errorf("AssertionCount = %d, want 1 — only the @throws is evaluated", + tc.AssertionCount()) + } + res, ok := assertionErrorResult(tc) + if !ok || res.Status != StatusError { + t.Errorf("result = %+v (ok=%v), want an ERROR", res, ok) + } + if mdl := GenerateTestFlows(&TestSuite{Name: "s", Tests: []TestCase{tc}}); strings.Contains(mdl, testFlowName(tc)) { + t.Errorf("a test with an assertion it cannot evaluate got a microflow:\n%s", mdl) + } +} + +// Control: @throws on its own is untouched, and so is an @expect on its own. +func TestThrowsAloneAndExpectAloneAreUnaffected(t *testing.T) { + only := parseAnnotations(`/** + * @test just throws + * @throws 'boom' + */`) + if only.Throws != "boom" || len(only.AssertionErrors) != 0 { + t.Errorf("throws alone: %+v", only) + } + + plain := parseAnnotations(`/** + * @test just expects + * @expect $result = 1 + */`) + if len(plain.Expects) != 1 || len(plain.AssertionErrors) != 0 { + t.Errorf("expect alone: %+v", plain) + } +} diff --git a/cmd/mxcli/testrunner/results.go b/cmd/mxcli/testrunner/results.go index 3e11c7b18..228f636f5 100644 --- a/cmd/mxcli/testrunner/results.go +++ b/cmd/mxcli/testrunner/results.go @@ -199,7 +199,7 @@ func ParseLogResults(logReader io.Reader, suite *TestSuite, requireAssertions bo // "MXTEST:" is the log node and the second is our protocol prefix. // We search for specific protocol actions to avoid matching the log node. protocol := "" - for _, action := range []string{"MXTEST:START:", "MXTEST:RUN:", "MXTEST:PASS:", "MXTEST:FAIL:", "MXTEST:SKIP:", "MXTEST:END:"} { + for _, action := range []string{"MXTEST:START:", "MXTEST:RUN:", "MXTEST:PASS:", "MXTEST:FAIL:", "MXTEST:ERROR:", "MXTEST:SKIP:", "MXTEST:END:"} { if idx := strings.Index(line, action); idx >= 0 { protocol = line[idx:] break @@ -268,6 +268,26 @@ func ParseLogResults(logReader io.Reader, suite *TestSuite, requireAssertions bo } } + case "ERROR": + // A setup that threw: the test never ran, so it is neither a pass nor + // a failure. Only the monolithic runner reports this way — the + // endpoint carries the same outcome back as a SETUP verdict. + msg := "" + if len(parts) >= 4 { + msg = parts[3] + } + if r, ok := resultMap[id]; ok { + r.Status = StatusError + r.Message = msg + } else { + resultMap[id] = &TestResult{ + ID: id, + Name: id, + Status: StatusError, + Message: msg, + } + } + case "SKIP": msg := "" if len(parts) >= 4 { diff --git a/cmd/mxcli/testrunner/runner.go b/cmd/mxcli/testrunner/runner.go index 12b9019ce..6c7c88f50 100644 --- a/cmd/mxcli/testrunner/runner.go +++ b/cmd/mxcli/testrunner/runner.go @@ -407,6 +407,9 @@ func ListTests(files []string, w io.Writer) error { fmt.Fprintf(w, "Found %d test(s):\n", len(suite.Tests)) for _, tc := range suite.Tests { fmt.Fprintf(w, " %s: %s\n", tc.ID, tc.Name) + for _, flow := range tc.Setups { + fmt.Fprintf(w, " @setup %s\n", flow) + } for _, exp := range tc.Expects { fmt.Fprintf(w, " @expect %s\n", exp.Raw) } diff --git a/cmd/mxcli/testrunner/setup_test.go b/cmd/mxcli/testrunner/setup_test.go new file mode 100644 index 000000000..16628821d --- /dev/null +++ b/cmd/mxcli/testrunner/setup_test.go @@ -0,0 +1,283 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testrunner + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// @setup was parsed into the test case and read by nothing: no generator, no +// runner, no reporter. `@setup Mod.Seed` did no setup and said nothing about it. +// It now names a microflow called before the test's own statements. +func TestSetupIsParsedPerTestAndRepeatable(t *testing.T) { + tests, err := parseMDLTests(`/** + * @test two fixtures, in order + * @setup eShop.ACT_SeedBrands + * @setup eShop.ACT_SeedTypes + */ +retrieve $Brands from eShop.CatalogBrand; +/ +`, "setup.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 1 { + t.Fatalf("got %d test(s), want 1", len(tests)) + } + want := []string{"eShop.ACT_SeedBrands", "eShop.ACT_SeedTypes"} + if got := tests[0].Setups; !equalStrings(got, want) { + t.Errorf("Setups = %v, want %v — repeating the annotation composes fixtures", got, want) + } +} + +// A file header's @setup applies to every test below it. This is the whole +// reason the annotation beats writing `call microflow X;` as the body's first +// line: one declaration for the file instead of copy-paste. +func TestSetupInFileHeaderAppliesToEveryTest(t *testing.T) { + tests, err := parseMDLTests(`/** + * Seeds every test in this file. + * @setup eShop.ACT_SeedCatalog + */ + +/** + * @test uses the file fixture + */ +retrieve $Brands from eShop.CatalogBrand; +/ + +/** + * @test adds one of its own + * @setup eShop.ACT_SeedOneBrand + */ +retrieve $Brands from eShop.CatalogBrand; +/ +`, "header.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 2 { + t.Fatalf("got %d test(s), want 2", len(tests)) + } + if got := tests[0].Setups; !equalStrings(got, []string{"eShop.ACT_SeedCatalog"}) { + t.Errorf("tests[0].Setups = %v, want the file's fixture", got) + } + // The file's fixture runs first: it is the broader precondition, and a test's + // own setup may depend on it. + want := []string{"eShop.ACT_SeedCatalog", "eShop.ACT_SeedOneBrand"} + if got := tests[1].Setups; !equalStrings(got, want) { + t.Errorf("tests[1].Setups = %v, want %v", got, want) + } +} + +// A header that is really a test's own doc comment must not donate anything: the +// first javadoc in the file is only a header when it carries no @test. +func TestSetupOnFirstTestIsNotFileLevel(t *testing.T) { + tests, err := parseMDLTests(`/** + * @test the first test has its own setup + * @setup eShop.ACT_SeedOne + */ +retrieve $Brands from eShop.CatalogBrand; +/ + +/** + * @test the second test has none + */ +retrieve $Brands from eShop.CatalogBrand; +/ +`, "not-header.test.mdl") + if err != nil { + t.Fatalf("parseMDLTests: %v", err) + } + if len(tests) != 2 { + t.Fatalf("got %d test(s), want 2", len(tests)) + } + if got := tests[0].Setups; !equalStrings(got, []string{"eShop.ACT_SeedOne"}) { + t.Errorf("tests[0].Setups = %v", got) + } + if len(tests[1].Setups) != 0 { + t.Errorf("tests[1].Setups = %v, want none — the first test's setup is not the file's", + tests[1].Setups) + } +} + +// A header can only carry what a file-wide default can honour. @cleanup, +// @expect, @verify and @throws describe one test's execution, and a header that +// silently ignored them would be the same silent-absence bug this annotation is +// being fixed for. +func TestFileHeaderRefusesPerTestAnnotations(t *testing.T) { + for _, tc := range []struct{ header, mentions string }{ + {" * @cleanup none", "@cleanup"}, + {" * @expect $x = 1", "@expect"}, + {" * @throws 'boom'", "@throws"}, + } { + src := "/**\n * A file header.\n" + tc.header + "\n */\n\n/**\n * @test t\n */\ncall microflow M.A();\n/\n" + _, err := parseMDLTests(src, "bad-header.test.mdl") + if err == nil { + t.Errorf("%s in a header: want an error, got none", tc.mentions) + continue + } + if !strings.Contains(err.Error(), tc.mentions) { + t.Errorf("error %q does not name %q", err, tc.mentions) + } + } +} + +// The end of the chain: the setup is called before the test's statements, and +// the generated microflow parses. +func TestGenerateTestFlowsCallsSetupFirst(t *testing.T) { + suite := &TestSuite{ + Name: "setup", + Tests: []TestCase{{ + ID: "test_1", + Name: "seeded", + MDL: "retrieve $Brands from eShop.CatalogBrand;", + Setups: []string{"eShop.ACT_SeedCatalog"}, + Expects: []Expect{expectOf("count($Brands) = 5")}, + }}, + } + + mdl := GenerateTestFlows(suite) + if !strings.Contains(mdl, "CALL MICROFLOW eShop.ACT_SeedCatalog()") { + t.Fatalf("the setup was not called:\n%s", mdl) + } + if strings.Index(mdl, "eShop.ACT_SeedCatalog") > strings.Index(mdl, "retrieve $Brands") { + t.Errorf("the setup runs after the body it is supposed to prepare:\n%s", mdl) + } + if _, errs := visitor.Build(mdl); len(errs) > 0 { + t.Fatalf("generated flow should parse, got %v\n%s", errs[0], mdl) + } +} + +// A setup that throws means the test never ran: it neither passed nor failed. +// Reporting that as a FAIL would blame the code under test for a broken fixture, +// which is the failure mode the annotation exists to prevent. +func TestSetupFailureIsReportedAsAnError(t *testing.T) { + suite := &TestSuite{ + Name: "setup", + Tests: []TestCase{{ + ID: "test_1", + Name: "seeded", + MDL: "retrieve $Brands from eShop.CatalogBrand;", + Setups: []string{"eShop.ACT_SeedCatalog"}, + }}, + } + mdl := GenerateTestFlows(suite) + if !strings.Contains(mdl, verdictSetupPrefix+"eShop.ACT_SeedCatalog") { + t.Fatalf("no SETUP verdict on the error path:\n%s", mdl) + } + + res := toResult(suite.Tests[0], &runResponse{ + OK: true, + Result: verdictSetupPrefix + "eShop.ACT_SeedCatalog", + }) + if res.Status != StatusError { + t.Errorf("status = %v, want ERROR — the test did not run", res.Status) + } + if !strings.Contains(res.Message, "eShop.ACT_SeedCatalog") { + t.Errorf("message %q does not name the setup microflow", res.Message) + } +} + +// A @throws test gets its setup too: the fixture is not the thing expected to +// throw, and it must run before the verdict is pre-set to a failure. +func TestSetupRunsOnAThrowsTest(t *testing.T) { + mdl := GenerateTestFlows(&TestSuite{ + Name: "setup", + Tests: []TestCase{{ + ID: "test_1", + Name: "rejects an empty order", + MDL: "call microflow eShop.ACT_Submit();", + Setups: []string{"eShop.ACT_SeedCatalog"}, + Throws: "validation failed", + }}, + }) + if !strings.Contains(mdl, "CALL MICROFLOW eShop.ACT_SeedCatalog()") { + t.Fatalf("a @throws test lost its setup:\n%s", mdl) + } + if strings.Index(mdl, "eShop.ACT_SeedCatalog") > strings.Index(mdl, "expected an exception") { + t.Errorf("the setup runs after the verdict is pre-set to a failure:\n%s", mdl) + } + if _, errs := visitor.Build(mdl); len(errs) > 0 { + t.Fatalf("generated flow should parse, got %v\n%s", errs[0], mdl) + } +} + +// The legacy after-startup runner reports over the log protocol rather than a +// returned verdict, so it needs its own marker — and the parser has to know it, +// or a failed setup reads as "not executed". +func TestLegacyRunnerReportsSetupFailure(t *testing.T) { + suite := &TestSuite{ + Name: "setup", + Tests: []TestCase{{ + ID: "test_1", + Name: "seeded", + MDL: "retrieve $Brands from eShop.CatalogBrand;", + Setups: []string{"eShop.ACT_SeedCatalog"}, + }}, + } + mdl := GenerateTestRunner(suite) + if !strings.Contains(mdl, "CALL MICROFLOW eShop.ACT_SeedCatalog()") { + t.Fatalf("the setup was not called:\n%s", mdl) + } + if !strings.Contains(mdl, "MXTEST:ERROR:test_1") { + t.Fatalf("no ERROR line on the setup's error path:\n%s", mdl) + } + if _, errs := visitor.Build(mdl); len(errs) > 0 { + t.Fatalf("generated runner should parse, got %v\n%s", errs[0], mdl) + } + + log := strings.NewReader( + "MXTEST: MXTEST:START:setup\n" + + "MXTEST: MXTEST:RUN:test_1:seeded\n" + + "MXTEST: MXTEST:ERROR:test_1:Setup failed: eShop.ACT_SeedCatalog\n" + + "MXTEST: MXTEST:END:setup\n") + sr := ParseLogResults(log, suite, false) + if len(sr.Tests) != 1 { + t.Fatalf("got %d result(s), want 1", len(sr.Tests)) + } + if sr.Tests[0].Status != StatusError { + t.Errorf("status = %v, want ERROR", sr.Tests[0].Status) + } + if !strings.Contains(sr.Tests[0].Message, "eShop.ACT_SeedCatalog") { + t.Errorf("message %q does not name the setup microflow", sr.Tests[0].Message) + } + if sr.PassCount() != 0 || sr.AllPassed() { + t.Errorf("a failed setup reported as passing: pass=%d allPassed=%v", + sr.PassCount(), sr.AllPassed()) + } +} + +// Control: a test with no @setup generates what it generated before. The whole +// feature must be invisible to a file that does not use it. +func TestNoSetupChangesNothing(t *testing.T) { + tc := TestCase{ + ID: "test_1", + Name: "plain", + MDL: "$r = CALL MICROFLOW M.A();", + Expects: []Expect{expectOf("$r = 1")}, + } + for _, mdl := range []string{ + GenerateTestFlows(&TestSuite{Name: "s", Tests: []TestCase{tc}}), + GenerateTestRunner(&TestSuite{Name: "s", Tests: []TestCase{tc}}), + } { + if strings.Contains(mdl, "mxtest_setup") || strings.Contains(mdl, verdictSetupPrefix) || + strings.Contains(mdl, "MXTEST:ERROR:") { + t.Errorf("a test with no setup grew setup scaffolding:\n%s", mdl) + } + } +} + +func equalStrings(a, b []string) bool { + if len(a) != len(b) { + return false + } + for i := range a { + if a[i] != b[i] { + return false + } + } + return true +} diff --git a/docs-site/src/appendixes/quick-reference.md b/docs-site/src/appendixes/quick-reference.md index 694b257fe..a080c5712 100644 --- a/docs-site/src/appendixes/quick-reference.md +++ b/docs-site/src/appendixes/quick-reference.md @@ -222,6 +222,7 @@ Nested folders use `/` separator: `'Parent/Child/Grandchild'`. Missing folders a | Revoke entity access | `REVOKE Mod.Role ON Mod.Entity;` | | | Set security level | `ALTER PROJECT SECURITY LEVEL OFF\|PROTOTYPE\|PRODUCTION;` | | | Toggle demo users | `ALTER PROJECT SECURITY DEMO USERS ON\|OFF;` | | +| Toggle guest access | `ALTER PROJECT SECURITY GUEST ACCESS ON ROLE UserRole\|OFF;` | Anonymous users; role required (CE0133) | | Create demo user | `CREATE DEMO USER 'name' PASSWORD 'pass' [ENTITY Module.Entity] (UserRole, ...);` | | | Drop demo user | `DROP DEMO USER 'name';` | | diff --git a/docs-site/src/language/security.md b/docs-site/src/language/security.md index ebbd778dc..f783b2b7c 100644 --- a/docs-site/src/language/security.md +++ b/docs-site/src/language/security.md @@ -63,6 +63,61 @@ ALTER PROJECT SECURITY DEMO USERS ON; ALTER PROJECT SECURITY DEMO USERS OFF; ``` +## Guest (Anonymous) Access + +Guest access is Studio Pro's "Anonymous users" setting: it lets people use part of +the app without signing in. It is a flag plus a user role, and the role carries +the weight -- **whatever it can read is public**. + +```sql +-- The role anonymous visitors are given. System.User is what lets an +-- unauthenticated session exist at all. +CREATE USER ROLE Anonymous (Shop.Viewer, System.User); + +ALTER PROJECT SECURITY GUEST ACCESS ON ROLE Anonymous; + +-- Grant exactly what should be public, and nothing else. +GRANT Anonymous ON Shop.Product (read *); +``` + +Turning it off keeps the stored role, so switching it back on needs no `ROLE` +clause: + +```sql +ALTER PROJECT SECURITY GUEST ACCESS OFF; +ALTER PROJECT SECURITY GUEST ACCESS ON; +``` + +### The role is mandatory + +Mendix fails the build with **CE0133** -- *"No user role for anonymous users +selected even though the feature anonymous users is enabled"* -- when guest access +is on and no role is set. `GUEST ACCESS ON` is therefore refused unless a role is +given in the statement or already stored in the project: + +``` +Error: GUEST ACCESS ON requires a role: no anonymous user role is configured, +and Mendix rejects anonymous access without one (CE0133). +``` + +### mxcli validates the role; Mendix does not + +An anonymous role that does not exist builds with **zero** extra errors in +Mendix -- the app is valid, and anonymous visitors simply get nothing. That makes +a typo invisible until someone opens the public page, so mxcli checks the name +against the project's user roles and refuses an unknown one: + +``` +Error: user role not found: Anonymus (project user roles: Administrator, User). +``` + +### Review what you made public + +Lint rule **SEC004** flags a project with guest access enabled, because any +unconstrained `read *` granted to the anonymous role is readable by the whole +internet ([DIVD-2022-00019](https://csirt.divd.nl/cases/DIVD-2022-00019/)). Add an +XPath constraint to each anonymous grant, or do not grant it. + ## See Also - [Module Roles and User Roles](./roles.md) -- defining roles diff --git a/docs-site/src/reference/domain-model/create-association.md b/docs-site/src/reference/domain-model/create-association.md index 4ddf5946f..0835677cc 100644 --- a/docs-site/src/reference/domain-model/create-association.md +++ b/docs-site/src/reference/domain-model/create-association.md @@ -7,7 +7,7 @@ TO to_module.to_entity TYPE { Reference | ReferenceSet } [ OWNER { Default | Both | Parent | Child } ] - [ DELETE_BEHAVIOR { DELETE_BUT_KEEP_REFERENCES | DELETE_AND_REFERENCES } ] + [ DELETE_BEHAVIOR { DELETE_BUT_KEEP_REFERENCES | DELETE_AND_REFERENCES | DELETE_IF_NO_REFERENCES | CASCADE | PREVENT } ] ## Description @@ -33,8 +33,11 @@ The `DELETE_BEHAVIOR` clause controls what happens when an object on the FROM si | Behavior | Description | |----------|-------------| -| `DELETE_BUT_KEEP_REFERENCES` | Delete the object and set references to null | +| `DELETE_BUT_KEEP_REFERENCES` | Delete the object and set references to null (the default) | | `DELETE_AND_REFERENCES` | Delete the object and all associated objects on the TO side | +| `CASCADE` | Alias for `DELETE_AND_REFERENCES` | +| `DELETE_IF_NO_REFERENCES` | Refuse the delete while anything still references the object | +| `PREVENT` | Alias for `DELETE_IF_NO_REFERENCES` | If `OR MODIFY` is specified, the statement is idempotent: if the association already exists, it is updated to match the new definition. diff --git a/docs-site/src/reference/security/README.md b/docs-site/src/reference/security/README.md index 73d8eabc1..0ebb58bfa 100644 --- a/docs-site/src/reference/security/README.md +++ b/docs-site/src/reference/security/README.md @@ -26,6 +26,7 @@ Mendix security operates at two levels. **Module roles** define permissions with | Show security matrix | `SHOW SECURITY MATRIX [IN module]` | | Alter project security level | `ALTER PROJECT SECURITY LEVEL OFF\|PROTOTYPE\|PRODUCTION` | | Toggle demo users | `ALTER PROJECT SECURITY DEMO USERS ON\|OFF` | +| Toggle guest access | `ALTER PROJECT SECURITY GUEST ACCESS ON [ROLE UserRole]\|OFF` | | Drop module role | `DROP MODULE ROLE module.Role` | | Drop user role | `DROP USER ROLE Name` | | Drop demo user | `DROP DEMO USER 'username'` | diff --git a/docs-site/src/tools/test-annotations.md b/docs-site/src/tools/test-annotations.md index 17406a50a..acfd84b55 100644 --- a/docs-site/src/tools/test-annotations.md +++ b/docs-site/src/tools/test-annotations.md @@ -1,111 +1,171 @@ # Test Annotations -Test annotations control test execution and define expectations. They are placed in comments within `.test.mdl` files. +Annotations describe what a test is and what must be true when it finishes. They +live in the javadoc comment above a test's statements, one per line, each opening +its line: + +```mdl +/** + * @test dealing a board writes 81 cells + * @cleanup none + * @expect $result = 'ok' + * @verify select count(*) as n from Sudoku.Cell = 81 + */ +$result = call microflow Sudoku.ACT_DealGame(); +/ +``` -## @test +| Tag | Purpose | +|-----|---------| +| `@test ` | Names the test. Required — a doc comment without it is not a test. | +| `@expect ` | A Mendix expression over the body's variables that must be true. Repeatable. | +| `@verify ` | An OQL post-condition on the database. Repeatable. | +| `@throws ''` | The body is expected to raise an error. | +| `@setup ` | A microflow to call before the body. Repeatable. | +| `@cleanup rollback\|none` | Whether the test's writes survive it. `rollback` is the default. | + +A tag is read only when it **opens its line** (after the javadoc `*` and its +indentation). Quoting one inside a sentence — `` `@expect $x = 1` `` — is +documentation, not an assertion. + +## `@test` + +Names the test, and marks the doc comment as one. Everything between that comment +and the `/` that ends the block is the test's body. + +A file may open with a header comment, in either spelling, and it is not part of +the first test. Exactly one `@test` may appear per block: the `/` is what ends a +test, and leaving it out is refused by name rather than silently running one of +the two tests it merges. + +## `@expect` + +Any Mendix expression that must evaluate to true — not a fixed `$var = value` +shape: + +```mdl +@expect $result = 'John Doe' -- equality +@expect $product/Name != 'Widget' -- inequality (<> is accepted too) +@expect length($result) = 81 -- built-in functions +@expect find($result, '0') >= 0 and $count > 3 -- and / or / not(...) +@expect $status = MyModule.Status.Open -- enumeration values +@expect count($Customers) = 5 -- how many rows a list holds +``` -Marks the start of a named test case. +`count($list)` is the one aggregate an assertion can make. Counting a list is a +Mendix Aggregate list **activity**, not an expression function, so mxcli lifts it +into that activity ahead of the decision that evaluates the condition. `sum`, +`average`, `minimum` and `maximum` aggregate an *attribute* over the list, which +an assertion has no way to supply — call a microflow that returns the figure and +assert on its result. -**Syntax:** +**An assertion the runner cannot compile is an ERROR, never a pass.** Unknown +functions, wrong arity, unbalanced parentheses, and expressions that evaluate to +a value rather than a condition are all rejected by name: -```sql --- @test +``` +ERROR a self-evident falsehood + @expect randomInt($result) = 1: randomInt() is not a Mendix expression + function at column 1 ("randomInt") ``` -**Example:** +A failing assertion reports what came back, whenever the observed value's type is +pinned down by the assertion itself: -```sql --- @test Create customer entity -CREATE PERSISTENT ENTITY MyModule.Customer ( - Name: String(200) NOT NULL -); +``` +FAIL the board is 81 squares + expected length($result) = 81, actual: 27 ``` -Each `@test` annotation starts a new test case. All MDL statements between one `@test` and the next (or end of file) belong to that test case. - -## @expect +`@expect` cannot be combined with `@throws`: a body expected to fail produces no +result to assert on, so the combination is refused rather than ignored. -Defines the expected outcome of a test case. +## `@verify` -**Syntax:** +`@expect` only sees what a microflow returned. Most Mendix microflows are side +effects, so `@verify` asserts on the rows one left behind — an OQL query, a +comparison operator, and the value it must satisfy: -```sql --- @expect +```mdl +@verify select count(*) as n from Sudoku.Cell = 81 +@verify select count(*) as n from Sudoku.Cell where Value = 0 > 0 ``` -### Expected Outcomes +The query runs after the microflow returns, over the same admin API `mxcli oql` +uses. Three rules follow: + +- The result must be **one row and one column** (`select count(*) as n …`, or one + attribute of one row). OQL requires the column to be named, so write `as n`. +- `@cleanup rollback` — the default — is **refused** with `@verify`: the writes + would be undone before the query could see them. Add `@cleanup none`. +- The **legacy after-startup runner refuses the whole suite**, because its tests + run during boot with nothing to query yet. Use `--local` or `--attach`. + +A query that cannot be evaluated is an ERROR, distinct from one that returns the +wrong value (FAIL). + +## `@throws` -| Expectation | Description | -|-------------|-------------| -| `0 errors` | The test should complete with no errors from `mx check` | -| `error` | The test is expected to produce an error | -| ` errors` | The test should produce exactly N errors | +Marks a body that is expected to raise an error. The verdict starts as a failure +and only the error handler clears it, so a body that completes normally fails the +test. -**Examples:** +```mdl +/** + * @test rejects an empty order + * @throws 'validation failed' + */ +$result = call microflow Sales.ACT_Submit(Order = $empty); +/ +``` + +## `@setup` -```sql --- @test Valid entity creation -CREATE PERSISTENT ENTITY MyModule.Customer ( - Name: String(200) NOT NULL -); --- @expect 0 errors +Names a **microflow** to call before the test's own statements — a fixture in a +Mendix app is a microflow, so there is nothing to declare: --- @test Missing module should fail -CREATE PERSISTENT ENTITY NonExistent.Entity ( - Name: String(200) -); --- @expect error +```mdl +/** + * @test the seed microflow writes five brands + * @setup eShop.ACT_SeedCatalog + * @cleanup none + * @expect count($Brands) = 5 + */ +retrieve $Brands from eShop.CatalogBrand; +/ ``` -## Complete Example - -```sql --- @test Setup: Create module and enumeration -CREATE MODULE TestSales; - -CREATE ENUMERATION TestSales.OrderStatus ( - Draft 'Draft', - Submitted 'Submitted', - Completed 'Completed' -); --- @expect 0 errors - --- @test Create entity with enum attribute -CREATE PERSISTENT ENTITY TestSales.Order ( - OrderNumber: String(50) NOT NULL, - Status: Enumeration(TestSales.OrderStatus) DEFAULT 'Draft', - TotalAmount: Decimal DEFAULT 0 -); --- @expect 0 errors - --- @test Create microflow with validation -CREATE MICROFLOW TestSales.ACT_SubmitOrder( - $order: TestSales.Order -) RETURNS Boolean AS $success -BEGIN - DECLARE $success Boolean = false; - - IF $order/TotalAmount <= 0 THEN - VALIDATION FEEDBACK $order/TotalAmount MESSAGE 'Total must be positive'; - RETURN false; - END IF; - - CHANGE $order (Status = 'Submitted'); - COMMIT $order; - SET $success = true; - RETURN $success; -END; --- @expect 0 errors - --- @test Security: grant access -CREATE MODULE ROLE TestSales.User; -GRANT EXECUTE ON MICROFLOW TestSales.ACT_SubmitOrder TO TestSales.User; -GRANT TestSales.User ON TestSales.Order (CREATE, DELETE, READ *, WRITE *); --- @expect 0 errors +Repeat it to compose fixtures; they run in the order written. Declare it once in +the file's header comment and every test in the file gets it, the file's +fixtures first: + +```mdl +/** + * Seeds every test below. + * @setup eShop.ACT_SeedCatalog + */ ``` -## Notes +A header may carry only `@setup`. `@expect`, `@verify`, `@throws` and `@cleanup` +describe one test's execution, so a header carrying one is refused by name. + +The setup runs **inside the test's transaction**, so under the `@cleanup +rollback` default it is undone with the test and every test starts from the same +state. A failing setup is an **ERROR** naming the microflow, not a FAIL: the test +never ran, and a broken fixture should not read as a broken feature. + +`@setup` calls a microflow with no arguments — a fixture that needs arguments +gets a wrapper microflow. There is no `@teardown`; `@cleanup rollback` is the +teardown. + +## `@cleanup` + +`rollback` (the default) wraps the test in a transaction that is rolled back when +it returns, so its writes do not survive it. `none` leaves them in place — needed +whenever a later test, or a `@verify`, has to see them. An unknown strategy is a +parse error. + +## Related Pages -- Test cases run sequentially; earlier test cases set up state for later ones -- The test runner validates using `mx check` after executing each test case (or the full script) -- Tests that modify the project are run against an isolated copy, not the original MPR file +- [Test Formats](test-formats.md) — the `.test.mdl` and `.test.md` file layouts +- [Running Tests](running-tests.md) — `mxcli test`, `--local`, `--watch`, `--attach` diff --git a/docs-site/src/tools/test-formats.md b/docs-site/src/tools/test-formats.md index 30ebda4b2..65fda71ff 100644 --- a/docs-site/src/tools/test-formats.md +++ b/docs-site/src/tools/test-formats.md @@ -1,95 +1,94 @@ # Test Formats -mxcli supports two test file formats: `.test.mdl` for pure MDL tests and `.test.md` for literate tests with documentation. - -## .test.mdl Format - -Pure MDL scripts with test annotations. Each test case is marked with `@test` and optionally `@expect`. - -```sql --- @test Create a new module -CREATE MODULE TestModule; --- @expect 0 errors - --- @test Create entity with attributes -CREATE PERSISTENT ENTITY TestModule.Customer ( - Name: String(200) NOT NULL, - Email: String(200) UNIQUE, - IsActive: Boolean DEFAULT true +`mxcli test` reads two file formats: `.test.mdl` for plain MDL tests, and +`.test.md` for tests embedded in documentation. Both describe the same thing — a +named test, some MDL to run against the app, and what must be true afterwards — +and both use the same [annotations](test-annotations.md). + +A test's body calls into the app. It is not a script that builds a project: the +statements run against a booted runtime, and the assertions are about what they +returned or wrote. + +## `.test.mdl` + +A test is a javadoc comment followed by its statements. A line containing just +`/` ends it, the way it ends any MDL block: + +```mdl +/** + * @test concatenating a name + * @expect $result = 'John Doe' + */ +$result = call microflow MyModule.ConcatNames( + FirstName = 'John', LastName = 'Doe' ); --- @expect 0 errors - --- @test Create enumeration -CREATE ENUMERATION TestModule.Status ( - Active 'Active', - Inactive 'Inactive' -); --- @expect 0 errors - --- @test Create microflow -CREATE MICROFLOW TestModule.ACT_Activate( - $customer: TestModule.Customer -) RETURNS Boolean AS $result -BEGIN - DECLARE $result Boolean = false; - CHANGE $customer (IsActive = true); - COMMIT $customer; - SET $result = true; - RETURN $result; -END; --- @expect 0 errors +/ + +/** + * @test the seed microflow writes five brands + * @cleanup none + * @expect count($Brands) = 5 + */ +retrieve $Brands from eShop.CatalogBrand; +/ ``` -## .test.md Format +Two rules about layout are worth knowing, because getting them wrong used to be +silent: -Literate test files that combine prose documentation with embedded MDL code blocks. The test runner extracts and executes the MDL code blocks while ignoring the prose. - -````markdown -# Customer Module Tests +- **A file may open with a header comment** — a `/** … */` block or `--` lines — + and it is not part of the first test. The test's own doc comment is the last + one above its statements. A `/** … */` header may carry + [`@setup`](test-annotations.md#setup), which then applies to every test in the + file. +- **One `@test` per block.** Because the `/` is what ends a test, omitting it + merges two tests into one. That is refused by name rather than resolved, since + either resolution runs one of the two and drops the other. -This test suite validates the customer management module. +## `.test.md` -## Entity Creation +The same tests, inside `mdl-test` fenced code blocks, with prose around them. One +block is one test; everything outside the fences is documentation and is ignored. -Create the customer entity with standard fields: - -```sql -CREATE PERSISTENT ENTITY TestModule.Customer ( - Name: String(200) NOT NULL, - Email: String(200) -); -``` +````markdown +# Customer module -## Association Setup +## Names are concatenated in display order -Link customers to their orders: +Given a first and a last name, `ConcatNames` returns them separated by a space: -```sql -CREATE ASSOCIATION TestModule.Order_Customer - FROM TestModule.Order - TO TestModule.Customer - TYPE Reference; +```mdl-test +/** + * @test concatenating a name + * @expect $result = 'John Doe' + */ +$result = call microflow MyModule.ConcatNames( + FirstName = 'John', LastName = 'Doe' +); ``` ```` -The `.test.md` format is useful for: -- Documenting test intent alongside the test code -- Creating test suites that serve as tutorials -- Sharing test cases with non-technical stakeholders +Use it when the reasoning around a test is worth as much as the test — a +specification that stays honest because it runs. + +## File organisation -## File Organization +`mxcli test` takes a file or a directory. A directory run picks up every +`*.test.mdl` and `*.test.md` in it (not recursively) and reports them as one +suite. ``` tests/ -├── domain-model.test.mdl # Entity and association tests -├── microflows.test.mdl # Microflow logic tests -├── security.test.mdl # Access rule tests -├── pages.test.mdl # Page creation tests -└── integration.test.md # Full integration test with docs +├── domain-model.test.mdl +├── microflows.test.mdl +├── security.test.mdl +└── ordering.test.md ``` -## Naming Conventions +Names are yours to choose; the `.test.mdl` / `.test.md` suffix is what makes a +file a test file. + +## Related Pages -- Use descriptive file names that indicate what is being tested -- Group related tests in the same file -- Use the `@test` annotation to name individual test cases within a file +- [Test Annotations](test-annotations.md) — `@test`, `@expect`, `@verify`, `@throws`, `@cleanup` +- [Running Tests](running-tests.md) — `mxcli test`, `--local`, `--watch`, `--attach` diff --git a/docs-site/src/tools/testing.md b/docs-site/src/tools/testing.md index 65772d97a..06018e692 100644 --- a/docs-site/src/tools/testing.md +++ b/docs-site/src/tools/testing.md @@ -1,6 +1,9 @@ # Testing -mxcli includes a testing framework for validating Mendix projects using MDL test files. Tests verify that MDL scripts execute correctly and that the resulting project passes validation. +mxcli includes a testing framework for a Mendix app's own logic: each test calls +into the running app — a microflow, a retrieve — and asserts on what came back or +what was written. It is not a check that an MDL script parses; that is +[`mxcli check`](../tutorial/validation.md). ## Overview @@ -8,17 +11,17 @@ The testing framework supports two test file formats: | Format | Extension | Description | |--------|-----------|-------------| -| MDL test files | `.test.mdl` | Pure MDL scripts with test annotations | -| Markdown test files | `.test.md` | Literate tests with prose and embedded MDL code blocks | +| MDL test files | `.test.mdl` | Tests as MDL blocks with annotations | +| Markdown test files | `.test.md` | Literate tests with prose and embedded `mdl-test` blocks | ## Prerequisites -Tests execute against a real Mendix runtime. **Docker is one way to get one, not the only one** — `--local` uses mxcli's own runtime with no daemon, and is the faster path (see [Running Tests](running-tests.md) for the mode comparison and the `--watch` / `--attach` loops). On the Docker path the test runner: +Tests execute against a real Mendix runtime. **Docker is one way to get one, not the only one** — `--local` uses mxcli's own runtime with no daemon, and is the faster path (see [Running Tests](running-tests.md) for the mode comparison and the `--watch` / `--attach` loops). Either way the runner: -1. Creates a fresh Mendix project using `mx create-project` -2. Executes the MDL test script against the project -3. Validates the result with `mx check` -4. Reports pass/fail results +1. Installs a temporary `MxTest` module holding one microflow per test +2. Boots (or reuses) the app and invokes each test over a token-guarded endpoint +3. Evaluates that test's assertions and reports pass, fail or error +4. Removes what it installed, leaving the project byte-identical ## Quick Start @@ -33,33 +36,39 @@ mxcli test tests/sales.test.mdl -p app.mpr ## Test Workflow 1. Write test files using `.test.mdl` or `.test.md` format -2. Add `@test` and `@expect` annotations for assertions +2. Name each test with `@test` and assert with `@expect` / `@verify` / `@throws` 3. Run tests with `mxcli test` 4. Review results ## Example Test -```sql +```mdl -- tests/customer.test.mdl --- @test Create customer entity -CREATE PERSISTENT ENTITY MyFirstModule.Customer ( - Name: String(200) NOT NULL, - Email: String(200) -); --- @expect 0 errors - --- @test Add association -CREATE ASSOCIATION MyFirstModule.Order_Customer - FROM MyFirstModule.Order - TO MyFirstModule.Customer - TYPE Reference; --- @expect 0 errors +/** + * @test a new customer is active by default + * @expect $customer/IsActive = true + */ +$customer = call microflow MyFirstModule.ACT_CreateCustomer(Name = 'Acme'); +/ + +/** + * @test the seed microflow writes five customers + * @cleanup none + * @expect count($customers) = 5 + * @verify select count(*) as n from MyFirstModule.Customer = 5 + */ +call microflow MyFirstModule.ACT_Seed(); +retrieve $customers from MyFirstModule.Customer; +/ ``` +An assertion the runner cannot evaluate is reported as an ERROR, never as a +pass — see [Test Annotations](test-annotations.md). + ## Playwright UI Testing -For browser-based testing that verifies widgets render correctly in the DOM, see [Playwright Testing](playwright.md). While `mxcli test` validates that MDL scripts execute correctly and the project passes `mx check`, Playwright testing goes further by verifying the running application in a real browser -- checking that widgets are visible, forms accept input, and navigation works. +For browser-based testing that verifies widgets render correctly in the DOM, see [Playwright Testing](playwright.md). `mxcli test` asserts on what the app's logic returns and writes; Playwright testing asserts on what the app renders -- that widgets are visible, forms accept input, and navigation works. ```bash # MDL validation (this page) @@ -72,7 +81,7 @@ mxcli playwright verify tests/ -p app.mpr ## Related Pages - [Test Formats](test-formats.md) -- `.test.mdl` and `.test.md` file formats -- [Test Annotations](test-annotations.md) -- `@test` and `@expect` annotations +- [Test Annotations](test-annotations.md) -- `@test`, `@expect`, `@verify`, `@throws`, `@cleanup` - [Running Tests](running-tests.md) -- `mxcli test` command and Docker requirements - [Playwright Testing](playwright.md) -- Browser-based UI testing with playwright-cli - [Diff](diff.md) -- Comparing scripts against project state diff --git a/docs-site/src/tutorial/create-entity.md b/docs-site/src/tutorial/create-entity.md index a06964f5e..65eed3e1c 100644 --- a/docs-site/src/tutorial/create-entity.md +++ b/docs-site/src/tutorial/create-entity.md @@ -79,7 +79,7 @@ CREATE ASSOCIATION MyModule.Order_Product DELETE_BEHAVIOR PREVENT; ``` -Options: `PREVENT` (block deletion if referenced), `DELETE` (cascade delete), or leave it out for the default behavior. +Options: `PREVENT` (block deletion if referenced), `CASCADE` (delete the associated objects too), or leave it out for the default behavior — delete the object and null out the references. ## Using OR MODIFY for idempotent scripts diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index 37b663659..d50c675be 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -413,7 +413,7 @@ Studio Pro picks the dropdown label from the referenced microflow's return type (`System.ConsumedODataConfiguration` vs `list of System.HttpHeader`). -## Microflows & Nanoflows +## Microflows, Nanoflows & Rules | Statement | Syntax | Notes | |-----------|--------|-------| @@ -430,6 +430,13 @@ return type (`System.ConsumedODataConfiguration` vs | Create nanoflow | `create [or modify] nanoflow Module.Name (params) returns type [folder 'path'] begin ... end;` | Same body syntax as microflows | | Move nanoflow | `move nanoflow Module.Name to folder 'path';` | | | Nanoflow restrictions | N/A | No Java actions, ErrorEvent, REST calls, database queries, external actions, download file, workflow actions, import/export mappings, JSON transformation, show home page | +| Show rules | `show rules [in module];` | `list rules` is the same statement | +| Describe rule | `describe rule Module.Name;` | Round-trippable | +| Create rule | `create [or modify] rule Module.Name (params) returns Boolean\|enum Module.Enum [folder 'path'] begin ... end;` | Same body syntax as microflows | +| Drop rule | `drop rule Module.Name;` | | +| Move rule | `move rule Module.Name to folder 'path';` | | +| Call a rule | `if Module.Rule_Name(Param = $Value) then ... end if;` | A decision is the ONLY place a rule can be called | +| Rule restrictions | N/A | Return type must be Boolean or an enumeration (mxbuild CE0103/CE0139); no create/change/delete/commit/rollback, no client interaction, no web-service calls (CE0009). There is no `grant execute on rule` — a rule stores no module-role security | ## Microflows - Supported Statements @@ -453,7 +460,8 @@ it is for pages. | Retrieve (DB) | `retrieve $Var from Module.Entity [where condition];` | Database XPath retrieve | | Retrieve (Assoc) | `retrieve $list from $Parent/Module.AssocName;` | Retrieve by association | | Add to list | `add expression to $list;` | Also accepts existing `add $item to $list;` form | -| Call microflow | `$Result = call microflow Module.Name (Param = $value);` | | +| Call microflow | `$Result = call microflow Module.Name (Param = $value);` | A Mendix **expression** cannot call anything — `declare $r Boolean = Module.Name(...)` is CE0117 (MDL066) | +| Call a rule | `if Module.SomeRule (Param = $value) then ... end if;` | A decision is the **only** place a rule can be evaluated; there is no call activity for one. The name must resolve to a rule — a microflow there is CE0117 | | Call microflow on a queue | `call microflow Module.Name (Param = $value) in queue Module.Queue;` | Background execution; the queue must exist (CE1613) | | Call Java action on a queue | `call java action Module.Name (Param = $value) in queue Module.Queue;` | The Java action must `returns void`, else CE7038 | | Call nanoflow | `$Result = call nanoflow Module.Name (Param = $value);` | | @@ -549,6 +557,8 @@ Nested folders use `/` separator: `'Parent/Child/Grandchild'`. Missing folders a | Revoke entity access (partial) | `revoke Mod.Role on Mod.Entity (read (attr));` | Partial — downgrades specific rights | | Set security level | `alter project security level off\|prototype\|production;` | | | Toggle demo users | `alter project security demo users on\|off;` | | +| Enable guest access | `alter project security guest access on role UserRole;` | Anonymous users. The role is what visitors get — its entity access is the public surface. Mendix fails the build without one (CE0133), so `on` is refused unless a role is given or already stored. mxcli validates the role exists; Mendix does not | +| Disable guest access | `alter project security guest access off;` | Keeps the stored role, so re-enabling needs no `role` clause | | Create demo user | `create demo user 'name' password 'pass' [entity Module.Entity] (UserRole, ...);` | | | Drop demo user | `drop demo user 'name';` | | diff --git a/docs/03-development/PED_MCP_CAPABILITIES.md b/docs/03-development/PED_MCP_CAPABILITIES.md index 80d8c5c0c..c7c3d468f 100644 --- a/docs/03-development/PED_MCP_CAPABILITIES.md +++ b/docs/03-development/PED_MCP_CAPABILITIES.md @@ -78,7 +78,7 @@ blank = absent, `—` = n/a). `cmd/mcpprobe -method tools/list` is the source. | `ped_create_module` | ✓ | Create a module (+ its domain model). Flushes to disk immediately. | | `ped_create_document` | ✓ | Create standalone documents (enum, microflow, page, …). "Never create domain models." Backend: `CreateEnumeration`. | | `ped_update_document` | ✓ | Operation-based set/add/remove at JSON paths. Backend: entities, attributes, associations (incl. removes). | -| `ped_check_errors` | ✓ | Validate documents (run after the final write). Backend: after every write. | +| `ped_check_errors` | ✓ | Validate documents (run after the final write). Backend: after every write — but see **Error-list lag** below. | | `pg_read_page` | ✓ | **Pages only** — separate read path. (Not yet used.) | | `pg_write_page` | ✓ | **Pages only** — separate write path; PED is *forbidden* for pages. (Not yet used.) | | `oql_generate` | ✓ | NL → OQL for a module. (Agent helper; not used.) | @@ -104,6 +104,50 @@ Captured live 2026-06-23 (`cmd/mcpprobe -method tools/list`). The 11.11 matrix a Tools already present in 11.11 and unchanged (do **not** re-add as "new"): `ped_list_folder`, `oql_generate`, `search_mendix_knowledge_base`, `read_skill`, `glob`, `read_file`, `write_file`. + +**Error-list lag — `ped_check_errors` is not synchronous with the write (issue #945).** +It reads the error list Studio Pro maintains on a **background thread**, so it +reports the model as of some moment *before* the call. Measured live (PED 1.0.0) +with a 20ms poll interval, **26 samples** spanning both directions and three +write-load levels (0, 10 and 30 preceding writes): + +| | min | median | max | +|---|---|---|---| +| write → error becomes **visible** | 77ms | 82ms | 115ms | +| fix → error stops being **shown** | 75ms | 83ms | 128ms | + +It is one symmetric debounce, and it did **not** grow with load. A settled check +itself costs ~11ms, so re-asking is cheap; only the initial wait is not. + +> **Measure it with a fine poll interval.** The first pass at this reported +> 170-350ms and concluded the lag "grows under load". Both were artifacts of +> polling at 50-100ms — the granularity, not the lag. Re-measuring at 20ms +> collapsed the spread to 77-128ms and removed the load dependence entirely. + +Both directions are damaging, so waiting is the only fix: + +- asked too early it answers `No errors found.` for a document that was just + broken — a **silent** miss, and `ped_update_document` is no backstop because it + reports op-level failures only (a duplicate name comes back as `SUCCESS`); +- it equally still reports an error a just-applied op has already **cleared**, + which fails a perfectly good statement on a leftover verdict. A multi-op ALTER + passes through intermediate states that are legitimately invalid, so this is + not hypothetical. + +**`Backend.pedCheckDocument` owns the pacing for every call site**: sleep +`settleDelay` (250ms, ~2x the observed max) before the first ask, then re-ask +across `settleWindow` (250ms) at `settleInterval` while the answer stays clean, +short-circuiting on an error. Re-asking is only safe *after* the delay — past it +a dirty verdict is current rather than left over. It lives in the shared helper +rather than in each caller so a new write path cannot forget it; the cost is one +settle per backend operation, and no operation validates more than once (checked: +`renameAttribute` and `applyInPlaceEntityChanges` are early-return branches of +`UpdateEntity`, not a loop). + +Verified live against the immediate check as a control, 6 trials each: the +immediate check missed a real error **6/6** and falsely failed on a cleared +transient **6/6**; the settled check was **0/6** on both. + **New authoring capability — attribute default values (implemented, `domainmodel.go`).** PED's `DomainModels$StoredValue.defaultValue` is settable via a `ped_update_document` path-op (`/entities/N/attributes/M/value/defaultValue`); the create constructor still can't carry it, so it's set as a follow-up after the attribute exists (`applyAttributeDefaults`). Verified live on 11.12: enums store **bare** (`Draft`, not `MES.WorkOrderStatus.Draft`); PED accepts bare or qualified input but normalises to bare. **Gated on the project Mendix version (11.12+), not the server version** (frozen at 1.0.0 — see the caveat under Server identity). The entity/attribute `$constructor` schema is otherwise unchanged from 11.11; its text now hints `DomainModels$Index` and `DomainModels$ValidationRule` are addable the same constructor-then-path-op way — candidates to wire next, like defaults. **New authoring capability — navigation (implemented, `navigation.go`; issue #699).** The project-level `Navigation$NavigationDocument` has no dedicated PED tool, but it is reachable through the **generic** document tools (`ped_read_document` / `ped_update_document` with `documentType` set and `documentName` **omitted** — project-level). `CREATE OR REPLACE NAVIGATION` on a **web** profile is authored as path-ops: scalar leaves are `set` in place (`/profiles/N/homePage/page`, `/profiles/N/loginPageSettings/page`), a currently-null element (`notFoundHomepage`) is `set` whole, and the menu (`menuItemCollection.items`, an array) is **cleared then rebuilt** — removes and adds go in **separate** calls (PED forbids batching add+remove on one array path). Menu items are `Menus$MenuItem` constructors: `caption` is a **plain string** (PED wraps it into `Texts$Text`), the action is `Pages$PageClientAction` (page ref under `pageSettings.page`), `Pages$MicroflowClientAction` (`microflowSettings.microflow`), or `Pages$NoClientAction` (container); sub-items recurse. Verified live on 11.12 by reproducing a full menu (3 page items + an Admin container with 2 children). Gates/limits: `ped_check_errors` cannot address the project-level nav doc, so the `ped_update_document` result is the validation gate (a bad page ref fails the op); **native** profiles and **role-based home pages** are rejected (not wired); and Studio Pro often answers a nav write with `-32000 Request timed out` while the edit still applies (slow nav re-render) — the backend surfaces an actionable hint rather than auto-retrying the non-idempotent op. diff --git a/docs/05-mdl-specification/10-bson-mapping.md b/docs/05-mdl-specification/10-bson-mapping.md index b7e8337e9..c9416589d 100644 --- a/docs/05-mdl-specification/10-bson-mapping.md +++ b/docs/05-mdl-specification/10-bson-mapping.md @@ -633,9 +633,9 @@ owner both; -- Creates 1-to-1: Customer has one Order | MDL Behavior | BSON DeleteBehavior.Type | |--------------|--------------------------| -| `DELETE_BUT_KEEP_REFERENCES` | `"DeleteMeButKeepReferences"` | -| `DELETE_CASCADE` | `"DeleteMeAndReferences"` | -| (default) | `"DeleteMeIfNoReferences"` | +| `DELETE_BUT_KEEP_REFERENCES` (and the default, when the clause is omitted) | `"DeleteMeButKeepReferences"` | +| `DELETE_AND_REFERENCES`, `CASCADE` | `"DeleteMeAndReferences"` | +| `DELETE_IF_NO_REFERENCES`, `PREVENT` | `"DeleteMeIfNoReferences"` | --- diff --git a/docs/05-mdl-specification/11-model-sdk-mapping.md b/docs/05-mdl-specification/11-model-sdk-mapping.md index 54e1f16d4..68de2801b 100644 --- a/docs/05-mdl-specification/11-model-sdk-mapping.md +++ b/docs/05-mdl-specification/11-model-sdk-mapping.md @@ -365,7 +365,8 @@ assoc := &domainmodel.Association{ | MDL | Go Constant | |-----|-------------| | `DELETE_BUT_KEEP_REFERENCES` | `DeleteBehaviorTypeDeleteMeButKeepReferences` | -| `DELETE_CASCADE` | `DeleteBehaviorTypeDeleteMeAndReferences` | +| `DELETE_AND_REFERENCES`, `CASCADE` | `DeleteBehaviorTypeDeleteMeAndReferences` | +| `DELETE_IF_NO_REFERENCES`, `PREVENT` | `DeleteBehaviorTypeDeleteMeIfNoReferences` | ### Association Struct diff --git a/docs/11-proposals/PROPOSAL_rule_documents.md b/docs/11-proposals/PROPOSAL_rule_documents.md new file mode 100644 index 000000000..0b9fc6f29 --- /dev/null +++ b/docs/11-proposals/PROPOSAL_rule_documents.md @@ -0,0 +1,381 @@ +--- +title: Rule documents — read, author, describe, catalog +status: done +date: 2026-08-21 +related: + - PROPOSAL_microflow_inheritance_split_statement.md + - PROPOSAL_codegen_ownership.md +--- + +# Proposal: Rule documents — read, author, describe, catalog + +**Status:** Done — slices 1–4 shipped 2026-08-21 +**Date:** 2026-08-21 + +A `Microflows$Rule` is the one document type mxcli can *reference* but cannot +read, write, describe, move, or count. Two upstream issues have now landed on +the consequences of that gap rather than on the gap itself, and both were fixed +one symptom at a time. + +## Problem Statement + +[Mendix's reference](https://docs.mendix.com/refguide/rules/) calls a rule "a +special kind of microflow" that returns a Boolean or an enumeration, can only be +used from a decision, and cannot modify data, talk to the client, or do +integration. Structurally it is a microflow: the same object collection, the +same flows, the same return type. + +mxcli treats it as a foreign object. Measured on a Mendix 11.13 project +carrying one rule (`Sample.Rule_IsActive`) and two microflows: + +| Surface | Today | +|---|---| +| `LIST FOLDERS IN Sample` | ✅ `Rule Rule_IsActive` — works, for free, via `ListDocumentUnits` + `types.DocumentKind` | +| `if Sample.Rule_IsActive(...) then` | ✅ authorable (a decision's condition), after #939 | +| `show callers of Sample.Rule_IsActive` | ✅ after #939 | +| `SHOW RULES` | ❌ no such statement | +| `DESCRIBE RULE Sample.Rule_IsActive` | ❌ parse error | +| `CREATE` / `ALTER` / `DROP RULE` | ❌ no grammar; a rule can only be authored in Studio Pro | +| `MOVE RULE` / `FOLDER` clause | ❌ absent from `ast.MoveDocumentTypeByKeyword` (31 doctypes, no rule) | +| `CATALOG.OBJECTS` | ❌ a rule is not an object; `SELECT … WHERE ModuleName='Sample'` returns only the microflows | +| `search 'Rule_IsActive'` | ❌ no matches | +| lint / `GRAPH_DEAD_ASSETS` | ❌ a microflow called **only** from inside a rule's body is reported dead | + +The last row is the one that silently misleads. A rule's body is never walked, +so every document it calls is invisible to the reference graph — the same shape +as the scheduled-event gap CLAUDE.md already records, one layer deeper. + +### Why this keeps producing issues + +Both rule issues so far were *absences* dressed as bugs: + +- **#723 §A4** — `IsRule` was unimplemented on the modelsdk backend, so every + `if Module.SomeRule(…)` became an expression split (CE0117). Fixed the read + half; left the write half. +- **#939** — the write half. `splitConditionToGen` had no `RuleSplitCondition` + case, so a decision was stored with no condition at all (CE0080), and the + reporter's three secondary symptoms each had their own cause. + +Neither is the last one, because the underlying state is unchanged: the model +has a document type mxcli cannot round-trip. Every feature that enumerates +documents has to remember to exclude rules, and each one that forgets is a new +issue. + +## The shape of the fix: a rule is a third flow flavour + +The repo already carries this pattern twice. `createNanoflowStatement` is a +**verbatim mirror** of `createMicroflowStatement` (`MDLMicroflow.g4:16` and +`:27`), sharing `microflowBody`, the flow builder, the describer and the +validator; the differences are a distinct `$Type`, a `flowBuilder.isNanoflow` +flag, and a disallowed-activity list. A rule is the same relationship with a +different restriction list. + +That is what makes "full support" a tractable change rather than a second +microflow implementation: almost none of the work is new, and the parts that +are new are a validator and a `$Type`. + +## BSON Structure + +`Microflows$Rule` — properties in `initRule`'s declared order +(`modelsdk/gen/microflows/types.go`), cross-checked against +`generated/metamodel` (`MicroflowsRule`, an 11.6.0 snapshot): + +| Key | Type | Note | +|---|---|---| +| `Name` | string | | +| `Documentation` | string | | +| `Excluded` | bool | | +| `ExportLevel` | enum | | +| `ObjectCollection` | `Microflows$MicroflowObjectCollection` | identical to a microflow's | +| `Flows` | list | identical to a microflow's | +| `MicroflowReturnType` | `DataTypes$*Type` | `DataTypes$BooleanType` or `DataTypes$EnumerationType` (with `Enumeration: "Mod.Enum"`) | +| `MarkAsUsed` | bool | | +| `ReturnVariableName` | string | Studio Pro wrote `"Variable"` on both reference rules (and `""` on a microflow in the same app) | +| `ApplyEntityAccess` | bool | | + +A rule carries **none** of the microflow-only keys: no `AllowedModuleRoles`, no +`AllowConcurrentExecution` / `ConcurrenyErrorMessage` / `ConcurrencyErrorMicroflow`, +no `Url` / `UrlSearchParameters`, no `MicroflowActionInfo` / `WorkflowActionInfo`, +no `StableId`. + +**Pinned against Studio Pro.** [`ako/TestApp`](https://github.com/ako/TestApp) +(Mendix 11.13.0) carries two rules authored in Studio Pro — `Rules.Rule1` +(Boolean return, String parameter, `ReturnValue: "length($pName)>0"`) and +`Rules.Rule2` (enumeration return `Rules.RuleResult`, entity parameter +`Pages.Bus`, `ReturnValue: "Rules.RuleResult.Approved"`). Both store exactly the +ten properties above and nothing else. + +Two things the reference settles that inference had wrong or unproven: + +- **`ReturnType` is not written.** gen declares it (a pre-7 legacy sibling of + `MicroflowReturnType`, and absent from `generated/metamodel` 11.6.0); Studio + Pro 11.13 writes only `MicroflowReturnType`. mxcli must not invent it. This + is *not* the `Interval`/`IntervalType` carry-through case — there is nothing + to carry. +- **The microflow-only keys really are absent.** A Studio Pro microflow in the + same app (`Microflows.SplitMerge`) stores 19 properties including + `AllowConcurrentExecution`, `AllowedModuleRoles`, `ConcurrencyErrorMicroflow`, + `ConcurrenyErrorMessage`, `MicroflowActionInfo`, `StableId`, `Url`, + `UrlSearchParameters` and `WorkflowActionInfo`. A rule stores none of them — + measured on both rules, not assumed from the type definition. + +### The parameter type — the question the reference rules answered + +`generated/metamodel` lists three sibling types: + +- `Microflows$MicroflowParameterObject` — the canvas object (`RelativeMiddlePoint`, + `Size`, `VariableType`, `IsRequired`, `DefaultValue`) +- `Microflows$MicroflowParameter` — a `MicroflowParameterBase` (`Name`, `ParameterType`) +- `Microflows$RuleParameter` — the same `MicroflowParameterBase` shape + +Measured against real documents, the metamodel's split is **not** what storage +does: every microflow in a blank 11.13 app — Studio Pro-authored marketplace +modules included (Administration, FeedbackModule, NanoflowCommons) — stores its +canvas parameter as `$Type: Microflows$MicroflowParameter` carrying the +*ParameterObject* shape. mxcli writes the same thing, so mxcli is right and the +metamodel naming is an SDK-side view. + +**Resolved by the reference rules: a rule's canvas parameter is +`Microflows$MicroflowParameter`, carrying the ParameterObject shape** +(`DefaultValue`, `Documentation`, `HasVariableNameBeenChanged`, `IsRequired`, +`Name`, `RelativeMiddlePoint`, `Size`, `VariableType`) — byte-for-byte the same +shape a microflow uses. `Microflows$RuleParameter` appears in neither reference +rule, so it is an SDK-side name with no storage counterpart here, and the rule +writer reuses `microflowParameterToGen` unchanged. + +### `RuleCall` — already fixed, recorded here for completeness + +`Microflows$RuleCall` stores the rule reference under **`Microflow`**, not +`Rule` (rules share the microflow namespace). `modelsdk/gen` bound `Rule` on +both the encode and decode side; #939 applied a `STORAGE-NAME OVERRIDE` and +struck the row off `modelsdk/gen/keyaudit_test.go`. Any new code touching a rule +call must not reintroduce the SDK name. + +## Proposed MDL Syntax + +The whole surface mirrors microflows, because the document does. + +```mdl +create or modify rule Sample.Rule_IsActive ($Customer: Sample.Customer) returns Boolean +folder 'Rules' +begin + return $Customer/IsActive and $Customer/Balance > 0; +end +/ +``` + +`returns` accepts Boolean or an enumeration and nothing else — the restriction +is Mendix's, and omitting it is CE-level invalid rather than a default. + +```mdl +show rules; +show rules in Sample; + +describe rule Sample.Rule_IsActive; -- round-trippable, like describe microflow + +drop rule Sample.Rule_IsActive; + +move rule Sample.Rule_IsActive to folder 'Rules/Customer'; +``` + +The decision form that calls one already exists and does not change: + +```mdl +if Sample.Rule_IsActive(Customer = $Customer) then + ... +end if; +``` + +No new verbs, no new property syntax, `Module.Element` throughout — the design +checklist in `.claude/skills/design-mdl-syntax.md` is satisfied by construction +because every form is the microflow form with one word changed. + +## Implementation Plan + +Four slices. Each is independently shippable and the first two are read-only. + +### Slice 1 — read and describe (no new BSON writes) + +| File | Change | +|---|---| +| `sdk/microflows/microflows.go` | `microflows.Rule` — its own struct, mirroring `Nanoflow` | +| `mdl/backend/microflow.go` | `ListRules` / `GetRule` on the interface | +| `mdl/backend/modelsdk/microflow.go` | implement via `mprread.ListUnitsWithContainer[*genMf.Rule]` — the decoder already registers `Microflows$Rule` | +| `mdl/backend/mpr/` | legacy implementation via `listUnitsByType("Microflows$Rule")`, which `IsRule` already calls | +| `mdl/backend/mock/` | `Func`-field stubs | +| grammar `MDLCatalog.g4` | `showOrList RULES (IN …)?` beside `NANOFLOWS`; `DESCRIBE RULE qualifiedName` | +| `mdl/ast/`, `mdl/visitor/` | nodes + listener | +| `mdl/executor/cmd_microflows_show.go` | `SHOW RULES`, and `DESCRIBE RULE` reusing the microflow describer | + +### Slice 2 — catalog, references and lint + +| File | Change | +|---|---| +| `mdl/catalog/builder.go` | `ListRules()` on `CatalogReader`; `cachedRules()` | +| `mdl/catalog/builder_objects.go` | a `RULE` row in `CATALOG.OBJECTS` | +| `mdl/catalog/builder_references.go` | call `emitActionRefs("RULE", …)` over each rule's object collection, so documents a rule calls stop reading as dead | +| `mdl/catalog/builder_strings.go` | index rule text for `search` | + +**Both halves at once, or neither.** Adding the object type without walking rule +bodies would report every rule as dead; walking bodies without the object type +leaves `show callers` half-populated. #939 deliberately stopped at the edge for +this reason. + +### Slice 3 — authoring + +| File | Change | +|---|---| +| grammar `MDLMicroflow.g4` | `createRuleStatement` / `dropRuleStatement`, mirroring `createNanoflowStatement` verbatim | +| `mdl/ast/ast_microflow.go` | `CreateRuleStmt` (or a `FlowKind` on the existing node) | +| `mdl/executor/cmd_rules_create.go` | thin handler; `flowBuilder` gains `isRule` beside `isNanoflow` | +| `mdl/executor/validate_rule.go` | the restriction list (below) — the `ValidateNanoflowBody` precedent | +| `mdl/backend/*/` | `CreateRule` / `UpdateRule` / `DeleteRule`; modelsdk writes `Microflows$Rule` with the ten keys, reusing `microflowToGen`'s object/flow serialization | +| `mdl/ast/ast.go` | `"RULE"` in `MoveDocumentTypeByKeyword`; `FOLDER` clause on create | + +The validator refuses what Mendix refuses, quoting the doc: + +- return type is not Boolean or an enumeration +- create / change / delete / commit / rollback object +- show page, close page, show message, validation feedback, download file +- call web service, generate document, import/export XML + +Each needs a measured CE number before it ships as an error rather than a +warning — the #931/#939 rule: measure against mxbuild, do not argue from docs. + +### Slice 4 — surfacing + +`mxcli syntax` topic, `docs-site/src`, `MDL_QUICK_REFERENCE.md`, the +`write-microflows` skill, LSP completion/hover, and `describe`-roundtrip +coverage in `mdl-examples/doctype-tests/`. + +## Version Compatibility + +Rules have existed since Mendix 5 and the document shape is unchanged across the +supported range (9/10/11). No `sdk/versions/*.yaml` gate is expected. The one +version-sensitive point is `ReturnType` vs `MicroflowReturnType` (see Open +Questions), which is a pre-7 legacy sibling of the kind `Interval` / +`IntervalType` already is for scheduled events — carry it through untouched on +modify, derive it on create, and pin it against a real document per version. + +## Test Plan + +- `mdl-examples/doctype-tests/` — a rule with a Boolean return, one with an + enumeration return, a decision calling each, and a `describe` → re-exec + round trip **into a module where the rule does not exist** (replaying over + the original reports "Unchanged" whether or not the clause survived). +- `mdl-examples/bug-tests/939-rule-split-condition.mdl` becomes fully + self-contained: it currently needs a Studio Pro-authored rule because MDL + cannot create one. +- Backend round-trip tests per engine, asserting the ten stored keys and the + **absence** of the microflow-only ones. +- A catalog test that a microflow called only from a rule's body is not in + `GRAPH_DEAD_ASSETS` — with the pre-change control. +- Negative tests (`*.fail.mdl`) for each restriction, each carrying the CE + number it prevents. + +## Design decisions + +Settled 2026-08-21. **A rule is handled the way a nanoflow is** — its own +statements, its own listing, its own semantic type — not as a variant of a +microflow. + +1. **A rule gets its own semantic type**, `microflows.Rule`, mirroring + `microflows.Nanoflow` (which is already a distinct struct, not an alias of + `Microflow`). Its own `CreateRuleStmt`, its own + `ListRules`/`GetRule`/`CreateRule`/`UpdateRule`/`DeleteRule` on the backend + interface. The document shape supports this: a rule is a microflow minus nine + properties, and the nine are exactly the ones a rule has no concept of. +2. **`SHOW`/`LIST MICROFLOWS` lists microflows only** — not nanoflows, not + workflows, and not rules. `SHOW RULES` / `LIST RULES` is the separate + listing, via the existing `showOrList` rule so both spellings work, as they + do for nanoflows. +3. **There is no `GRANT EXECUTE ON RULE`.** Both reference rules store no + `AllowedModuleRoles` — a rule is not independently callable, so it has no + module-role security to grant. This is a real divergence from the nanoflow + mirror and the one place the parallel stops. + +Two consequences worth stating, because they are what "like a nanoflow" buys: + +- A rule's surface is **much smaller than a nanoflow's**. A nanoflow is + reachable from pages, navigation and widget actions; a rule is reachable only + from a decision. None of `mdl/backend/widgetobj`, `sdk/pages`, + `modelsdk/gen/navigation` or the page grammar needs to learn about rules. +- `microflowBody`, the flow builder, the describer and the validator are shared, + exactly as the nanoflow shares them. + +### Still open + +- **`ReturnVariableName`.** Both reference rules carry `"Variable"`; the + microflow in the same app carries `""`. Whether that is a Studio Pro default + for the rule editor or authored text is not established by two samples. It + must at minimum be **preserved** on modify; whether MDL grows a surface for it + is a separate call. + +## Measured: a foldered rule is stored like any other document + +TestApp now places both rules in a `MyRulesRule` folder, and the containment is +exactly a microflow's: the rule's unit row is `ContainmentName: "Documents"` +with `ContainerID` pointing at the folder unit, which is itself `Folders` under +the module. Nothing about a rule's placement is special. + +Two consequences, both shrinking the plan: + +- `LIST FOLDERS IN Rules` already renders the foldered rules correctly, with no + change — #932's `ListDocumentUnits` walk is containment-generic. +- The MOVE/`FOLDER` work in Slice 3 is **one entry in + `ast.MoveDocumentTypeByKeyword`**, not a placement implementation. + +## Measured: the rule call #939 writes matches Studio Pro's + +`Rules.MicroflowUsingRule` now carries a decision calling `Rules.Rule1`, so the +shape is no longer validated only against the legacy engine. Studio Pro stores: + +``` +SplitCondition Microflows$RuleSplitCondition + RuleCall Microflows$RuleCall + Microflow "Rules.Rule1" ← the storage key, not "Rule" + ParameterMappings [2, {Microflows$RuleCallParameterMapping, + Parameter: "Rules.Rule1.pName", Argument: "'abc'"}] +``` + +Every element of that is what #939 now writes: the same `$Type`s, the same +`Microflow` storage key (independent confirmation of the `initRuleCall` +override), the same typed-array marker **2**, and the same fully-qualified +`Parameter` — which is how the flow builder already constructs it +(`call.Name + "." + name`). + +Two round-trip results on that document: + +- `DESCRIBE MICROFLOW` renders `if Rules.Rule1(pName = 'abc') then`, and + `show callers of Rules.Rule1` lists the microflow — the read and reference + paths exercised against a Studio Pro document for the first time. +- Re-executing that describe output over the app and re-checking gives **0 + errors, against a 0-error baseline** (mxbuild 11.13.0, baseline run first). + Diffing the two documents, the `SplitCondition` block does not appear in the + diff at all — the rebuild reproduces it exactly. + +What *does* differ in that diff is the surrounding flow graph, and all of it is +pre-existing and general to microflows: the connection-index widths and +`CaseValues` shape below, plus two more of the same kind — Studio Pro keeps a +flow's bezier control vectors (`"15;0"`, `"0;15"`) where a `@curve` annotation +did not survive describe, and the object collection's ordering differs. None of +these are rule-specific and none change the check result. + +## Adjacent findings, out of scope + +Measured while pinning the rule shape, both on Studio Pro documents in TestApp +and both **general to microflows**, not rule-specific: + +- **`CaseValues` on a sequence flow.** Studio Pro writes the bare marker `[2]` + where mxcli writes an explicit `[2, {Microflows$NoCase}]`, and on a boolean + split it labels only the *false* branch, leaving the true branch's list empty + where mxcli labels both. Both load; mxcli's form is what the legacy writer has + always produced. +- **`OriginConnectionIndex` / `DestinationConnectionIndex` width.** Studio Pro + stores int64; mxcli writes int32. Same class as the widths recorded as out of + scope in #931 (`CanvasHeight`, `TabIndex`, `PopupWidth`), now observed on + `Microflows$SequenceFlow` as well. + +Neither has a known symptom. They are recorded here so the next person to diff a +rule against Studio Pro does not mistake them for something this feature +introduced. diff --git a/docs/11-proposals/PROPOSAL_test_setup_annotation.md b/docs/11-proposals/PROPOSAL_test_setup_annotation.md new file mode 100644 index 000000000..ebdedc0e5 --- /dev/null +++ b/docs/11-proposals/PROPOSAL_test_setup_annotation.md @@ -0,0 +1,244 @@ +--- +title: "@setup for mxcli test" +status: done +date: 2026-08-21 +--- + +# Proposal: `@setup` — give a test the state it needs, and say so + +**Status:** Done — implemented, minus the pre-flight validation (open question 4, declined) +**Date:** 2026-08-21 + +## Problem Statement + +`@setup` is parsed today and read by nothing. `parseAnnotations` matches it into +`TestCase.Setup`, and no runner, generator or reporter ever looks at that field. +`@setup CreateTestData` in a `.test.mdl` file is a no-op that reports nothing. + +It arrived with the original framework in +[proposal-mdl-test-framework.md](proposal-mdl-test-framework.md) (draft, +2026-02-23) as one row of an annotation table — "Reference to setup block" — and +was never designed past that row. There is no syntax anywhere for declaring a +setup block, no execution semantics, and no other mention of setup in the +proposal body. `git log -S` confirms the field and its pattern landed with the +first testrunner commit and have not been touched since. + +This is the same silent-absence class as the three defects fixed in #927 and +#926 before it — an annotation present in the file and absent from the run — but +it fails in the *other* direction. A dropped `@expect` makes a test pass when it +should fail. A dropped `@setup` means the fixture never exists, so the test +usually **fails confusingly**: it asserts against an empty database and reports a +mismatch that has nothing to do with the code under test. Where the test asserts +nothing, it passes vacuously. + +Two things follow. The annotation must stop being a no-op, whichever way that is +resolved. And the shape it takes should be the one that earns its place over the +line of MDL an author can already write. + +There is no BSON in this feature. It touches the test runner's generated +microflows and its own file format; no Mendix document type is read or written. + +## What a setup can and cannot do here + +The design space is set by how a test executes, so this comes first. + +`mxcli test --local` (and `--attach`) invokes **one microflow per HTTP request** +against the generated endpoint. Per request, the handler +(`cmd/mxcli/testrunner/endpoint.go`): + +- creates a **fresh system context**, +- when `rollback=1`, calls `ctx.startTransaction()`, runs the microflow, and + rolls back in a `finally`, +- refuses any microflow not named `MxTest.Test_*`. + +Three consequences, and they decide the design: + +1. **The transaction is per request.** Anything that must be undone with the test + has to run *inside the test's own microflow call*. A separate request runs in + its own context and its own transaction; its writes are not covered by the + test's rollback. +2. **A once-per-file fixture therefore cannot be rolled back at all.** The + endpoint has no seam for a transaction spanning several requests. Whatever a + suite-level setup writes stays written when the run ends. Under `--local` that + is the `_test` database and merely untidy; under `--attach` it is + **the developer's own dev database**, which mxcli would be seeding behind + their back. That asymmetry is the strongest single argument in this document. +3. **A setup emitted into the test's microflow needs no protocol change.** No + Java handler edit, no new route, no client change, and it behaves identically + on `--local`, `--attach`, Docker and the legacy after-startup runner. Anything + requiring a new request needs all four to agree. + +## Proposed design + +**`@setup ` names a microflow to call before the test's own +statements, in the test's own transaction.** It is repeatable, and it may be +declared once for a whole file. + +```mdl +/** + * Seeds every test in this file. A header comment's annotations apply to the + * tests below it. + * @setup eShop.ACT_SeedCatalog + */ + +/** + * @test the catalogue seeds five brands + * @expect count($Brands) = 5 + */ +retrieve $Brands from eShop.CatalogBrand; +/ + +/** + * @test a brand can be renamed + * @setup eShop.ACT_SeedOneBrand + * @expect $brand/Name = 'Renamed' + */ +retrieve $brand from eShop.CatalogBrand; +$brand = call microflow eShop.ACT_Rename(Brand = $brand, Name = 'Renamed'); +/ +``` + +The generated microflow is what an author would write by hand, with the setup +calls first: + +```mdl +CREATE OR REPLACE MICROFLOW MxTest.Test_test_1 () +RETURNS String AS $Verdict +BEGIN + DECLARE $Verdict String = 'PASS'; + CALL MICROFLOW eShop.ACT_SeedCatalog() ON ERROR { + SET $Verdict = 'SETUP:eShop.ACT_SeedCatalog'; + RETURN $Verdict; + }; + retrieve $Brands from eShop.CatalogBrand; + ... +``` + +### The rules + +- **It is a microflow, not a new kind of block.** A fixture in a Mendix app is a + microflow; the runner's whole job is calling microflows. Naming one needs no + declaration syntax, no cross-block reference resolution, and no new concept in + the file format. +- **Setups run in order, before the body**, file-level ones first, then the + test's own. Repeating the annotation is how you compose two fixtures. +- **A setup runs inside the test's transaction.** Under the `@cleanup rollback` + default it is undone with the test, so every test starts from the same state — + which is the property that makes a fixture worth having. Under `@cleanup none` + it persists, like everything else that test writes. +- **A failing setup is an ERROR, not a FAIL.** The distinction is the point: the + test never ran, so it neither passed nor failed, and a suite full of assertion + mismatches caused by one broken seed is the failure mode this annotation exists + to prevent. The verdict protocol gains a third prefix (`SETUP:`) alongside + `PASS` and `FAIL:`. +- **An unresolvable microflow is refused, by name, before anything runs.** + Fail-closed, as with every other annotation in this package: a `@setup` naming + a microflow that does not exist must not produce a run that quietly did no + setup. This falls out of injection — the generated flows go in through `mxcli + exec`, whose check names the missing microflow and writes nothing — so it needs + no pre-flight of its own. +- **`--list` shows it**, so `mxcli test --list` says what a test depends on + without booting anything. + +### Why this over the alternatives + +| Alternative | Why not | +|---|---| +| **A named setup *block* in the `.test.mdl` file**, as the original table row implies | Needs a declaration syntax, a reference resolver, and an error for every way the two can disagree — to express something a microflow already expresses. It also puts fixture logic in a file the app cannot run, so nothing else in the project can reuse it. | +| **Once per file, one request before the tests** | Cannot be rolled back (consequence 2 above), so it seeds the developer's own database under `--attach`. Also needs a teardown story, an ordering story with `--filter`, and agreement across four runners. Worth revisiting only with a suite-scoped transaction, which the endpoint cannot express today. | +| **Leave it to the author: `call microflow X;` as the body's first line** | Already possible, and for a single test it is honestly fine. What it cannot do is attribute the failure (a throwing seed reports "exception during execution" of the *test*) or apply to a whole file without copy-paste. Those two are the feature. | +| **Delete `@setup`** | Cheapest, and a real option — see Open Questions. It leaves the copy-paste case unimproved but costs nothing to maintain. | + +### What this does not do + +- No teardown counterpart. `@cleanup rollback` already covers the common case, + and a `@teardown` that runs after a rolled-back test would be asserting against + state that no longer exists — the trap `@verify` hit. +- No parameters. `@setup Mod.Flow` calls a microflow with no arguments; a fixture + that needs arguments gets a wrapper microflow. +- No sharing across files. File scope is the largest scope that stays inside one + test's transaction. + +## Implementation Plan + +Order: parse → generate → report → validate. Each step is independently testable +and the first two are the whole feature. + +### Files to modify + +| File | Change | +|------|--------| +| `cmd/mxcli/testrunner/parser.go` | `Setup` becomes `[]string` (repeatable); `setupPattern` already anchored; merge the file header's setups ahead of each test's | +| `cmd/mxcli/testrunner/parser.go` | Header annotations: a leading doc comment with no `@test` currently yields no test and is discarded — keep its annotations as file-level defaults. #927's `scanDocComments` already isolates the header, which is what makes this cheap | +| `cmd/mxcli/testrunner/generator_endpoint.go` | Emit the setup calls at the top of `writeExpectFlowBody` / `writeThrowsFlowBody`, each with an `ON ERROR` handler setting the `SETUP:` verdict and returning | +| `cmd/mxcli/testrunner/generator.go` | Same for the monolithic runner — no variable suffix is needed after all, since a fixture call binds nothing. It reports over the log protocol rather than a returned verdict, so a setup failure emits a new `MXTEST:ERROR:` line — added to the marker list `ParseLogResults` scans, which today knows only START/RUN/PASS/FAIL/SKIP/END | +| `cmd/mxcli/testrunner/generator_endpoint.go` | `verdictSetupPrefix` beside `verdictPass` / `verdictFailPrefix` | +| `cmd/mxcli/testrunner/client.go` | `toResult` maps a `SETUP:` verdict to `StatusError` with the microflow named — a fourth arm on the switch that already tells "the test failed" from "the microflow threw". Its `default` reports an unrecognised verdict, so an unhandled `SETUP:` would error rather than pass, which is the right way round to get this wrong | +| `cmd/mxcli/testrunner/results.go` | Nothing: `StatusError` is already counted with the failures | +| ~~`cmd/mxcli/testrunner/runner.go`~~ | ~~Validate the generated flows with `mxcli check … --references` before `exec`~~ — **not implemented** (open question 4). An unknown `@setup` microflow is caught by `mxcli exec`'s own check when the flows are injected, which names it; the whole-script pre-flight would have refused suites for unrelated references | +| `cmd/mxcli/testrunner/runner.go` | `--list` prints each test's setups | +| `.claude/skills/mendix/test-microflows.md`, `docs-site/src/tools/test-annotations.md` | Document the annotation, the ordering, and the ERROR-not-FAIL rule | +| `mdl-examples/doctype-tests/` | A `.test.mdl` exercising file-level and per-test setup | + +The validation step was the one with a blast radius — `--references` resolves +*every* reference in the generated flows, not just the setups — and it was +dropped from the implementation for that reason. It is not needed for the +failure it was meant to catch: injecting the generated flows already runs +`mxcli exec`, whose own check names an unresolvable microflow and refuses to +write, so an unknown `@setup` still cannot produce a run that quietly did no +setup. + +## Version Compatibility + +None. This is mxcli's own test-file format and its own generated microflows; no +Mendix version gate is involved. The generated MDL uses `CALL MICROFLOW … ON +ERROR`, which the runner already emits for every test body. + +The legacy after-startup runner supports the feature (it is a generated +statement, not a protocol change), unlike `@cleanup rollback` and `@verify`, +which it cannot honour. + +## Test Plan + +Unit, in `cmd/mxcli/testrunner`: + +- `@setup` parses repeatably, and file-level setups precede a test's own. +- A file header's annotations reach the tests below it — with the control that a + header carrying `@test` is still the two-tests-in-one-block error from #927. +- The generated flow calls the setups **before** the body, and the emitted MDL + parses (`visitor.Build`, as the existing generator tests do). +- Monolith: two tests with the same setup do not collide after renaming. +- A `SETUP:` verdict becomes `StatusError`, is counted with the failures, and + names the microflow. +- Control: a test with no `@setup` generates byte-identical MDL to today. + +End to end, the way #927's count support was verified — unit tests prove the +parser, not Mendix: exec the generated flows into a real project and run +`mx check`, with a pristine-copy baseline. A generated `CALL MICROFLOW … ON +ERROR` that does not compile is invisible to `mxcli check`. + +And the control that matters: revert the generator change and confirm the setup +microflow is absent from the generated MDL — the symptom this proposal exists to +fix is an annotation that generates nothing. + +## Open Questions + +1. **Is the feature worth its weight at all?** Deleting `@setup` — pattern, + field, and the one control test that pins it as parseable — is a defensible + answer and costs nothing to maintain. The case for keeping it rests on two + things: failure attribution (ERROR naming the seed, instead of a FAIL blaming + the test) and a file-level fixture without copy-paste. If neither is worth a + day, delete it and say so in the docs. +2. **Should a bare `@setup` with no argument mean "this block is the fixture"?** + It would let a fixture live in the test file without a microflow — at the cost + of the second meaning for one tag, and of fixture logic the app cannot reuse. + Recommended: no, at least not first. +3. **Does the header-annotation concept want to be general?** Once a file header + can carry `@setup`, the question of whether it can carry `@cleanup` — a + file-wide default — answers itself in the affirmative, and that is a bigger + change than this proposal. Recommended: `@setup` only, and refuse the others + in a header by name rather than ignoring them. +4. **Should the `--references` validation land at all?** It is the only way to + refuse an unknown setup microflow before boot, and it can turn an existing + suite red for unrelated reasons. Alternative: resolve only the setup names, + via a targeted lookup rather than a whole-script check. diff --git a/mdl-examples/bug-tests/901-association-delete-behavior-prevent.mdl b/mdl-examples/bug-tests/901-association-delete-behavior-prevent.mdl new file mode 100644 index 000000000..85327f746 --- /dev/null +++ b/mdl-examples/bug-tests/901-association-delete-behavior-prevent.mdl @@ -0,0 +1,77 @@ +-- Repro: DELETE_BEHAVIOR PREVENT reported success and wrote +-- DeleteMeButKeepReferences, destroying a stored DELETE_AND_REFERENCES. +-- Upstream #901. +-- +-- describe -> delete_behavior DELETE_AND_REFERENCES +-- create or modify ... DELETE_BEHAVIOR PREVENT +-- -> Modified association: Del901.Child_Parent +-- describe -> delete_behavior DELETE_BUT_KEEP_REFERENCES +-- +-- Three bugs, layered: +-- +-- 1. buildDeleteBehavior read only the CASCADE token, so PREVENT, +-- DELETE_IF_NO_REFERENCES and DELETE_AND_REFERENCES all fell through to +-- the zero value. A legal behaviour, so nothing downstream noticed. +-- 2. ALTER ASSOCIATION SET built the stored value from +-- ast.DeleteBehavior.String(), which spells the prevent case +-- "DeleteIfNoReferences" — not one of Mendix's three DeletingBehavior +-- values. Fixing (1) alone put that string on disk. +-- 3. DESCRIBE emitted `delete_behavior DELETE_CASCADE`, which is not a +-- token, so describe -> edit -> exec died on any cascading association. +-- +-- `mx check` is NOT the oracle here. Measured on mxbuild 11.6.6: a project +-- carrying the out-of-domain "DeleteIfNoReferences" builds with 0 errors, the +-- same as a correct one. Studio Pro is the one that refuses it. So verify by +-- reading the values back, which is what the DESCRIBEs below are for. +-- +-- Expected after the fix: every DESCRIBE echoes the behaviour that was asked +-- for, `mx check` reports 0 errors, and no .mxunit contains a DeletingBehavior +-- outside {DeleteMeAndReferences, DeleteMeButKeepReferences, +-- DeleteMeIfNoReferences}. + +create module Del901; + +create persistent entity Del901.Parent ( Name: String(100) ); +create persistent entity Del901.Child ( Title: String(100) ); + +-- Baseline: cascade. Starting from the default would make a silent fallback to +-- DELETE_BUT_KEEP_REFERENCES indistinguishable from success. +create association Del901.Child_Parent + from Del901.Child to Del901.Parent + type Reference owner Default storage column + delete_behavior CASCADE; + +describe association Del901.Child_Parent; +-- expect: delete_behavior DELETE_AND_REFERENCES + +-- The reported statement. Before the fix this printed "Modified association" +-- and wrote DeleteMeButKeepReferences. +create or modify association Del901.Child_Parent + from Del901.Child to Del901.Parent + type Reference owner Default storage column + delete_behavior PREVENT; + +describe association Del901.Child_Parent; +-- expect: delete_behavior DELETE_IF_NO_REFERENCES + +-- The canonical spelling of cascade, whose alias CASCADE always worked. Not in +-- the report, and downgraded exactly the same way. +create or modify association Del901.Child_Parent + from Del901.Child to Del901.Parent + type Reference owner Default storage column + delete_behavior DELETE_AND_REFERENCES; + +describe association Del901.Child_Parent; +-- expect: delete_behavior DELETE_AND_REFERENCES + +-- The ALTER path, where bug (2) lives. Before the fix this stored the +-- out-of-domain string "DeleteIfNoReferences". +alter association Del901.Child_Parent set delete_behavior PREVENT; + +describe association Del901.Child_Parent; +-- expect: delete_behavior DELETE_IF_NO_REFERENCES + +alter association Del901.Child_Parent set delete_behavior DELETE_BUT_KEEP_REFERENCES; + +describe association Del901.Child_Parent; +-- expect: delete_behavior DELETE_BUT_KEEP_REFERENCES diff --git a/mdl-examples/bug-tests/927-test-mdl-leading-comment-and-count.test.mdl b/mdl-examples/bug-tests/927-test-mdl-leading-comment-and-count.test.mdl new file mode 100644 index 000000000..3884d97ec --- /dev/null +++ b/mdl-examples/bug-tests/927-test-mdl-leading-comment-and-count.test.mdl @@ -0,0 +1,96 @@ +/** + * ISSUE #927 -- two defects in .test.mdl handling, both of which made a suite + * report green while asserting less than it looked like it did. + * + * This header is itself the repro for bug 1. A file-level doc comment is not + * separated from the test below it by a '/', so both landed in one chunk, the + * header was taken as that test's doc comment, it carried no test annotation -- + * and the whole chunk, real test included, was dropped with no message of any + * kind. Every later test then reported under the number of the one above it. + * + * (Prose in a doc comment must not spell an annotation out: the parser reads + * one anywhere in a line, so an example written literally here would be picked + * up as this file's own. The names below are quoted without their leading @.) + * + * No project is needed to see the parse half: + * + * mxcli test mdl-examples/bug-tests/927-test-mdl-leading-comment-and-count.test.mdl --list + * + * Expected output -- four tests, numbered from 1: + * + * Found 4 test(s): + * test_1: a leading doc comment must not swallow this test + * expect count($Scans) = 2 + * test_2: the line comment above is prose, not a doc comment + * expect count($Scans) >= 0 + * test_3: a false count must fail, not pass + * expect count($Scans) = 999 + * test_4: an aggregate that cannot be compiled is an ERROR, never a PASS + * ERROR: sum() aggregates an attribute over a list, which a test + * assertion cannot do on its own -- call a microflow that returns + * the sum and assert on its result + * + * Before the fix the same command reported two tests -- test_1 and test_2 here + * were gone, and the two that remained were renumbered over them. + */ + +/** + * @test a leading doc comment must not swallow this test + * @expect count($Scans) = 2 + * @cleanup none + */ +retrieve $Scans from HomeScan.HomeScan; +/ + +-- Bug 1b: the workaround for bug 1 was to write the file header as `--` line +-- comments instead. That re-triggered the fusion as soon as one of those lines +-- described the bug, because the delimiters were found by raw substring search +-- with no notion of already being inside a comment. This line spells both of +-- them out -- /** and */ -- and the test below it must still be found. + +/** + * @test the line comment above is prose, not a doc comment + * @expect count($Scans) >= 0 + * @cleanup none + */ +retrieve $Scans from HomeScan.HomeScan; +/ + +/** + * ISSUE #927 bug 2 -- a count assertion on a bare retrieve reported PASS + * unconditionally, against an empty table, for every value the reporter tried. + * + * count() is not a Mendix expression function: counting a list is an Aggregate + * list activity, so the condition could never be compiled into the decision + * that evaluates the assertion. The annotation was dropped during parsing, and + * a test with no assertions left passes as long as its body does not throw. + * + * The count is now lifted into the activity an author would write by hand -- + * `$mxtest_count_Scans = COUNT($Scans);` ahead of the decision -- so the + * assertion is really evaluated. Against a table with anything but 999 rows, + * this test FAILS, and the failure names the observed count: + * + * mxcli test mdl-examples/bug-tests/927-test-mdl-leading-comment-and-count.test.mdl -p app.mpr --local + * + * FAIL a false count must fail, not pass + * expected count($Scans) = 999, actual: 2 + * + * @test a false count must fail, not pass + * @expect count($Scans) = 999 + * @cleanup none + */ +retrieve $Scans from HomeScan.HomeScan; +/ + +/** + * The other four aggregates need an attribute to aggregate over, which an + * assertion cannot supply -- so they are still refused. That is the half of + * bug 2 that must not regress: an assertion the runner cannot evaluate has + * exactly one safe outcome, and passing is not it. + * + * @test an aggregate that cannot be compiled is an ERROR, never a PASS + * @expect sum($Scans) = 1 + * @cleanup none + */ +retrieve $Scans from HomeScan.HomeScan; +/ diff --git a/mdl-examples/bug-tests/939-rule-split-condition.mdl b/mdl-examples/bug-tests/939-rule-split-condition.mdl new file mode 100644 index 000000000..31597e6e2 --- /dev/null +++ b/mdl-examples/bug-tests/939-rule-split-condition.mdl @@ -0,0 +1,93 @@ +-- ============================================================================ +-- Issue mendixlabs/mxcli#939 — a decision that calls a rule lost its Condition +-- ============================================================================ +-- +-- `if Module.SomeRule(param = $x) then` is stored as a Microflows$ExclusiveSplit +-- whose SplitCondition is a Microflows$RuleSplitCondition wrapping a RuleCall. +-- The modelsdk engine (the default) had no case for it: splitConditionToGen +-- returned nil and the caller skipped SetSplitCondition, so the decision was +-- written with no Condition at all. +-- +-- Measured on mxbuild 11.13.0, one variable per run: +-- +-- default engine → [CE0080] "The 'Condition' property is required." +-- at Decision 'Sample.Rule_IsActive(IsActive = $IsActive)' +-- MXCLI_ENGINE=legacy → 0 errors (the control: same script, same project) +-- after the fix → 0 errors, and the written BSON matches legacy's +-- +-- It was silent because the READ side had been implemented (#723): DESCRIBE +-- rendered `if true then` while the Decision's caption still showed the call, so +-- the round-trip produced MDL that looked right. Re-applying an identical +-- statement over a Studio Pro-authored (or legacy-written) microflow therefore +-- turned a clean project into a broken one — the reporter's sequence. +-- +-- Second half of the same defect: modelsdk/gen binds Microflows$RuleCall.Rule to +-- the BSON key "Rule", but Mendix stores it as "Microflow" (rules share the +-- microflow namespace). Fixing only the missing case would have produced the same +-- CE0080 for a different reason. See the STORAGE-NAME OVERRIDE in initRuleCall. +-- +-- ---------------------------------------------------------------------------- +-- Running this example +-- +-- MDL cannot author rules, and no module in a blank Mendix 11 app ships one, so +-- the reproduction needs a rule created in Studio Pro (a Boolean parameter +-- `IsActive`, returning `$IsActive`) named Sample.Rule_IsActive. With that in +-- place: +-- +-- mxcli exec 939-rule-split-condition.mdl -p app.mpr +-- mx check app.mpr # 0 errors +-- mxcli -p app.mpr -c "describe microflow Sample.MF_UseRule" +-- # `if Sample.Rule_IsActive(...)`, +-- # not `if true` +-- mxcli exec 939-rule-split-condition.mdl -p app.mpr # "Unchanged microflow" — the +-- # re-apply is now elided +-- +-- The MDL below is deliberately left executable rather than commented out: it +-- parses without a project, so `make check-mdl` covers the grammar path, and it +-- is the exact statement that reproduced the failure. +-- ============================================================================ + +create or modify microflow Sample.MF_UseRule ($IsActive: Boolean) +begin + if Sample.Rule_IsActive(IsActive = $IsActive) then + return; + else + return; + end if; +end +/ + +-- ---------------------------------------------------------------------------- +-- Three further defects reported on the same issue, all fixed: +-- +-- * A rule (or microflow, or Java action) call in an EXPRESSION position is +-- CE0117 "Error(s) in expression." on BOTH engines — a Mendix expression has +-- no user-callable functions. Now MDL066, an error, with no project needed; +-- `exec` refuses the script. The decision position needs the project to +-- judge (only a rule may be called there), so the flow builder refuses a +-- qualified call whose name is not a rule. +-- * `mxcli check -p` used to false-positive on the WORKING form above — +-- "Unexpected token after expression … glued keywords such as 'emptyor'" +-- with an empty location. mdl/exprcheck did not model a qualified call, so +-- the trailing `(` was reported as leftover. It fired on the valid form as +-- much as the invalid ones, so it carried no signal. +-- * `show callers of Sample.Rule_IsActive` found nothing even with the +-- condition stored correctly: a rule is not an activity, and the reference +-- extractor walked only action activities. A decision's rule call now emits +-- a `call` edge into CATALOG.REFS. +-- +-- Uncommenting the statement below is the MDL066 reproduction; `mxcli check` +-- reports it with no project. +-- +-- create or modify microflow Sample.MF_UseRuleVar ($IsActive: Boolean) returns Boolean +-- begin +-- declare $Active Boolean = Sample.Rule_IsActive(IsActive = $IsActive); +-- return $Active; +-- end +-- +-- Still open, and not part of this fix: rules are not catalog OBJECTS, so they +-- do not appear in `show structure` and a microflow called only from inside a +-- rule's body is still reported as dead by QUAL004 / GRAPH_DEAD_ASSETS. Adding +-- the object type without also indexing rule bodies would report every rule +-- itself as dead, so it needs both halves at once. +-- ---------------------------------------------------------------------------- diff --git a/mdl-examples/bug-tests/939-showpage-argument-ignored.mdl b/mdl-examples/bug-tests/939-showpage-argument-ignored.mdl new file mode 100644 index 000000000..1ac606997 --- /dev/null +++ b/mdl-examples/bug-tests/939-showpage-argument-ignored.mdl @@ -0,0 +1,70 @@ +-- mxcli-formula1 §39 (adjacent finding) +-- +-- A SHOW_PAGE widget argument naming anything other than the enclosing widget's +-- context object was discarded in silence: the page opened with the context +-- object, `mx check` reported 0 errors, and DESCRIBE printed the inferred +-- mapping. Nothing anywhere said the written argument had been ignored. +-- +-- The two ACCEPTED forms below are the ones the skills document; the refused +-- form is commented out because this script is executed by the test suite. + +create module PgArg; + +create or modify entity PgArg.Car ( + Name: string(100) +); + +create or modify page PgArg.Detail ( + Title: 'Detail', + Layout: Atlas_Core.Atlas_Default, + Params: { $Car: PgArg.Car } +) { + dataview dvDetail (DataSource: $Car) { + textbox tbName (Attribute: Name) + } +} + +-- ACCEPTED: $currentObject in a database-backed list. +create or modify page PgArg.Overview ( + Title: 'Overview', + Layout: Atlas_Core.Atlas_Default +) { + listview lvCars (DataSource: DATABASE PgArg.Car) { + actionbutton btnOpen ( + Caption: 'Open', + Action: SHOW_PAGE PgArg.Detail(Car: $currentObject) + ) + } +} + +-- ACCEPTED: the context variable by its own name — the enclosing data view is +-- bound to $Car, so $Car and $currentObject denote the same object. +create or modify page PgArg.Caller ( + Title: 'Caller', + Layout: Atlas_Core.Atlas_Default, + Params: { $Car: PgArg.Car } +) { + dataview dvCar (DataSource: $Car) { + actionbutton btnGo ( + Caption: 'Go', + Action: SHOW_PAGE PgArg.Detail(Car: $Car) + ) + } +} + +-- REFUSED (MDL-PAGEARG01): $Other is not the context object, so Mendix would +-- open Detail with $Car. Uncomment to see the guard fire; `mxcli check` flags it +-- without a project. +-- +-- create or modify page PgArg.TwoParam ( +-- Title: 'TwoParam', +-- Layout: Atlas_Core.Atlas_Default, +-- Params: { $Car: PgArg.Car, $Other: PgArg.Car } +-- ) { +-- dataview dvTwo (DataSource: $Car) { +-- actionbutton btnBad ( +-- Caption: 'Go', +-- Action: SHOW_PAGE PgArg.Detail(Car: $Other) +-- ) +-- } +-- } diff --git a/mdl-examples/bug-tests/drop-attribute-orphaned-validation-rule.mdl b/mdl-examples/bug-tests/drop-attribute-orphaned-validation-rule.mdl new file mode 100644 index 000000000..f053e450c --- /dev/null +++ b/mdl-examples/bug-tests/drop-attribute-orphaned-validation-rule.mdl @@ -0,0 +1,42 @@ +-- Repro: DROP ATTRIBUTE left an orphaned validation rule behind (CE1613). +-- +-- [error] [CE1613] "The selected attribute 'DropAttr.Account.AccountNumber' +-- no longer exists." at Validation rule of entity 'DropAttr.Account' +-- The app contains: 1 errors. +-- +-- Two bugs, each hiding the other: +-- +-- 1. The executor's cleanup compared a BY_NAME qualified-name reference +-- against an element ID, so it matched nothing and kept every rule. +-- 2. Even once removed, the write was dropped: a child list emptied by an +-- update is "clean", so the codec passed the STORED rules through. +-- +-- Bug 2 only bites when the update removes the LAST member of a list, which is +-- why every attribute below has exactly one rule. A second rule would dirty the +-- list and the whole thing would appear to work. +-- +-- Expected after the fix: `mx check` reports 0 errors. + +create module DropAttr; +create module role DropAttr.User; + +create persistent entity DropAttr.Account ( + AccountNumber: String(20) NOT NULL ERROR 'Account number is required', + Balance: Decimal +) +INDEX (AccountNumber); + +grant DropAttr.User on DropAttr.Account (READ (AccountNumber, Balance), WRITE (AccountNumber, Balance)); + +-- The entity's only validation rule, its only index column, and one of its two +-- member-access entries all point at this attribute. +alter entity DropAttr.Account drop attribute AccountNumber; + +-- The Attributes list has the same failure mode, and there it is worse than a +-- dangling reference: dropping the only attribute reported success and wrote +-- nothing at all, so the attribute was still on disk afterwards. +create persistent entity DropAttr.Solo ( + OnlyAttr: String(20) +); + +alter entity DropAttr.Solo drop attribute OnlyAttr; diff --git a/mdl-examples/doctype-tests/30-guest-access.mdl b/mdl-examples/doctype-tests/30-guest-access.mdl new file mode 100644 index 000000000..ac7fbb1d5 --- /dev/null +++ b/mdl-examples/doctype-tests/30-guest-access.mdl @@ -0,0 +1,52 @@ +-- Guest (anonymous) access (issue mendixlabs/mxcli#924) +-- +-- Studio Pro's "Anonymous users" setting: part of the app is usable without +-- signing in. It is a flag plus a user role, and the role is the important +-- half — whatever it can read is the app's public surface. +-- +-- Before this syntax existed the flag was readable (SHOW PROJECT SECURITY) but +-- not writable, so a public-facing app could not be built unattended: every +-- headless run ended with a manual step in Studio Pro. +-- +-- Two Mendix behaviours shape the syntax, both measured on mxbuild 11.13: +-- +-- * The role is mandatory. Guest access on with no role fails the build with +-- CE0133, so GUEST ACCESS ON is refused unless a role is given here or +-- already stored in the project. +-- * Mendix does NOT check the role exists. A misspelled role builds with the +-- same error count as a valid one and leaves anonymous visitors with no +-- access at all, so mxcli validates the name itself. +-- +-- Run: +-- mxcli exec mdl-examples/doctype-tests/30-guest-access.mdl -p app.mpr +-- mxcli -p app.mpr -c "show project security" + +create or modify entity Catalog.Product ( + Name: String(200), + Price: Decimal, + Published: Boolean +); + +create module role Catalog.Visitor description 'Read-only access to published products'; +create module role Catalog.Editor description 'Maintains the product catalogue'; + +-- The role anonymous visitors are given. System.User is what lets an +-- unauthenticated session exist at all. +create user role Anonymous (Catalog.Visitor, System.User); +create user role CatalogAdmin (Catalog.Editor, System.User); + +alter project security level production; +alter project security guest access on role Anonymous; + +-- Grant exactly what should be public, and nothing else. The XPath constraint +-- is the difference between "the catalogue is public" and "every row is +-- public" — see lint rule SEC004 / DIVD-2022-00019. +grant Catalog.Visitor on Catalog.Product (read *) where '[Published = true()]'; +grant Catalog.Editor on Catalog.Product (create, delete, read *, write *); + +-- Toggling access off keeps the stored role, so switching it back on needs no +-- ROLE clause. Both statements are no-ops on a project already in that state. +alter project security guest access off; +alter project security guest access on; + +show project security; diff --git a/mdl-examples/doctype-tests/rules.mdl b/mdl-examples/doctype-tests/rules.mdl new file mode 100644 index 000000000..812ee9dbf --- /dev/null +++ b/mdl-examples/doctype-tests/rules.mdl @@ -0,0 +1,75 @@ +-- ============================================================================ +-- Rules (Microflows$Rule) +-- ============================================================================ +-- +-- Mendix's reference calls a rule "a special kind of microflow" that returns a +-- Boolean or an enumeration and may only be used from a decision. mxcli handles +-- it the way it handles a nanoflow: its own statements, its own listing, the +-- shared microflow body. +-- +-- list rules [in Module] +-- describe rule Module.Name -- round-trippable +-- create [or modify] rule ... +-- drop rule Module.Name +-- move rule Module.Name to folder '...' +-- +-- There is deliberately no `grant execute on rule`: a rule is not independently +-- callable, so its document stores no AllowedModuleRoles (measured on two +-- Studio Pro-authored rules, Mendix 11.13.0). +-- +-- What a rule may NOT contain is refused by `mxcli check`, not left to the +-- build. Measured on mxbuild 11.13.0: +-- +-- create/change/delete/commit/rollback → CE0009 "This action is not +-- supported in rules." +-- void or non-Boolean/enum return → CE0103 + CE0139 +-- ============================================================================ + +create module RuleDocs; +/ + +create or modify enumeration RuleDocs.Outcome ( + Approved 'Approved', + Rejected 'Rejected' +) +/ + +create or modify entity RuleDocs.Customer ( + Name: String(100), + Balance: Decimal +) +/ + +-- A Boolean rule, the common case: a decision calls it and branches on true/false. +create or modify rule RuleDocs.Rule_IsSolvent ($pCustomer: RuleDocs.Customer) +returns Boolean +folder 'Rules' +begin + return $pCustomer/Balance >= 0; +end +/ + +-- An enumeration rule: the decision branches on the enumeration's values. +create or modify rule RuleDocs.Rule_Outcome ($pCustomer: RuleDocs.Customer) +returns enum RuleDocs.Outcome +begin + if $pCustomer/Balance >= 0 then + return RuleDocs.Outcome.Approved; + else + return RuleDocs.Outcome.Rejected; + end if; +end +/ + +-- A rule is called from a decision, and only from a decision. The reference +-- edge this produces is what makes `show callers of RuleDocs.Rule_IsSolvent` +-- work — and what stops a microflow called only from a rule reading as dead. +create or modify microflow RuleDocs.MF_Screen ($pCustomer: RuleDocs.Customer) +begin + if RuleDocs.Rule_IsSolvent(pCustomer = $pCustomer) then + return; + else + return; + end if; +end +/ diff --git a/mdl-examples/doctype-tests/setup-annotation.test.mdl b/mdl-examples/doctype-tests/setup-annotation.test.mdl new file mode 100644 index 000000000..02cec8363 --- /dev/null +++ b/mdl-examples/doctype-tests/setup-annotation.test.mdl @@ -0,0 +1,73 @@ +/** + * The setup annotation, end to end: a fixture microflow called before each test. + * + * This header declares a fixture for every test in the file. It is the shape + * that makes the annotation worth more than `call microflow X;` at the top of + * each body — one declaration instead of copy-paste — and it is only readable + * because a header comment stopped being swallowed by the first test (#927). + * + * A header may carry only this annotation. The four that describe one test's + * execution -- the assertion tags, the expected-error tag and the cleanup tag -- + * are refused here by name. + * + * (Sample output below is written without the leading @, deliberately: a tag is + * read wherever it opens a line, so spelling one out in prose would give this + * file annotations nobody wrote.) + * + * No project is needed to see what the annotation resolved to: + * + * mxcli test mdl-examples/doctype-tests/setup-annotation.test.mdl --list + * + * Found 4 test(s): + * test_1: the file fixture applies with nothing else + * setup MyModule.ACT_SeedCatalog + * expect count($Brands) >= 1 + * test_2: a test composes its own fixture after the file's + * setup MyModule.ACT_SeedCatalog + * setup MyModule.ACT_SeedOneBrand + * ... + * + * @setup MyModule.ACT_SeedCatalog + */ + +/** + * @test the file fixture applies with nothing else + * @cleanup none + * @expect count($Brands) >= 1 + */ +retrieve $Brands from MyModule.CatalogBrand; +/ + +/** + * The file's fixtures run first: they are the broader precondition, and a + * test's own setup may build on them. Repeating the annotation is how two + * fixtures compose. + * + * @test a test composes its own fixture after the file's + * @setup MyModule.ACT_SeedOneBrand + * @cleanup none + * @expect count($Brands) >= 2 + */ +retrieve $Brands from MyModule.CatalogBrand; +/ + +/** + * The fixture runs inside the test's transaction, so the @cleanup rollback + * default undoes it along with everything the test wrote — which is what lets + * every test start from the same state. + * + * @test the fixture is rolled back with the test + * @expect count($Brands) >= 1 + */ +retrieve $Brands from MyModule.CatalogBrand; +/ + +/** + * A @throws test keeps its fixture: the seed is not the thing expected to + * throw, and it runs before the verdict is pre-set to a failure. + * + * @test a test expecting an error still gets its fixture + * @throws 'validation failed' + */ +call microflow MyModule.ACT_SubmitEmptyOrder(); +/ diff --git a/mdl/ast/ast.go b/mdl/ast/ast.go index 005c1a2d4..ad2910b3a 100644 --- a/mdl/ast/ast.go +++ b/mdl/ast/ast.go @@ -60,6 +60,7 @@ const ( DocumentTypeMicroflow DocumentType = "MICROFLOW" DocumentTypeSnippet DocumentType = "SNIPPET" DocumentTypeNanoflow DocumentType = "NANOFLOW" + DocumentTypeRule DocumentType = "RULE" DocumentTypeEntity DocumentType = "ENTITY" DocumentTypeEnumeration DocumentType = "ENUMERATION" DocumentTypeConstant DocumentType = "CONSTANT" @@ -102,6 +103,7 @@ var MoveDocumentTypeByKeyword = map[string]DocumentType{ "PAGE": DocumentTypePage, "MICROFLOW": DocumentTypeMicroflow, "NANOFLOW": DocumentTypeNanoflow, + "RULE": DocumentTypeRule, "SNIPPET": DocumentTypeSnippet, "BUILDINGBLOCK": DocumentTypeBuildingBlock, "LAYOUT": DocumentTypeLayout, diff --git a/mdl/ast/ast_microflow.go b/mdl/ast/ast_microflow.go index 784686f7c..bd5eccf49 100644 --- a/mdl/ast/ast_microflow.go +++ b/mdl/ast/ast_microflow.go @@ -80,6 +80,32 @@ type CreateNanoflowStmt struct { func (s *CreateNanoflowStmt) isStatement() {} +// CreateRuleStmt represents: CREATE RULE Module.Name (params) RETURNS type BEGIN body END +// +// Mirrors CreateNanoflowStmt: a rule shares a microflow's body, so the fields +// are the same minus the ones a rule document has no property for (a rule stores +// no AllowedModuleRoles, so there is nothing to grant). +type CreateRuleStmt struct { + Name QualifiedName + Parameters []MicroflowParam + ReturnType *MicroflowReturnType + Body []MicroflowStatement + Documentation string + Comment string + Folder string // Folder path within module + CreateOrModify bool + Excluded bool // @excluded — document excluded from project +} + +func (s *CreateRuleStmt) isStatement() {} + +// DropRuleStmt represents: DROP RULE Module.Name +type DropRuleStmt struct { + Name QualifiedName +} + +func (s *DropRuleStmt) isStatement() {} + // DropNanoflowStmt represents: DROP NANOFLOW Module.Name type DropNanoflowStmt struct { Name QualifiedName diff --git a/mdl/ast/ast_query.go b/mdl/ast/ast_query.go index 6760c0cdb..33085474d 100644 --- a/mdl/ast/ast_query.go +++ b/mdl/ast/ast_query.go @@ -32,6 +32,7 @@ const ( ShowAssociation ShowMicroflows ShowNanoflows + ShowRules ShowPages ShowSnippets ShowLayouts @@ -128,6 +129,8 @@ func (t ShowObjectType) String() string { return "MICROFLOWS" case ShowNanoflows: return "NANOFLOWS" + case ShowRules: + return "RULES" case ShowPages: return "PAGES" case ShowSnippets: @@ -325,6 +328,7 @@ const ( DescribeContractMessage // DESCRIBE CONTRACT MESSAGE Service.MessageName DescribeJsonStructure // DESCRIBE JSON STRUCTURE Module.Name DescribeNanoflow // DESCRIBE NANOFLOW Module.Name + DescribeRule // DESCRIBE RULE Module.Name DescribeImportMapping // DESCRIBE IMPORT MAPPING Module.Name DescribeExportMapping // DESCRIBE EXPORT MAPPING Module.Name DescribeModel // DESCRIBE MODEL Module.Name (agent-editor Model document) @@ -409,6 +413,8 @@ func (t DescribeObjectType) String() string { return "JSON STRUCTURE" case DescribeNanoflow: return "NANOFLOW" + case DescribeRule: + return "RULE" case DescribeImportMapping: return "IMPORT MAPPING" case DescribeExportMapping: diff --git a/mdl/ast/ast_security.go b/mdl/ast/ast_security.go index 9c03d99e7..35da7c09d 100644 --- a/mdl/ast/ast_security.go +++ b/mdl/ast/ast_security.go @@ -192,6 +192,12 @@ type AlterProjectSecurityStmt struct { SecurityLevel string // DemoUsersEnabled is set for ALTER PROJECT SECURITY DEMO USERS ON/OFF DemoUsersEnabled *bool + // GuestAccessEnabled is set for ALTER PROJECT SECURITY GUEST ACCESS ON/OFF + GuestAccessEnabled *bool + // GuestUserRole is the user role anonymous visitors get, from the optional + // ROLE clause on GUEST ACCESS ON. Empty means "keep whatever is stored" — + // never "clear it"; the executor refuses ON when nothing is stored either. + GuestUserRole string } func (s *AlterProjectSecurityStmt) isStatement() {} diff --git a/mdl/backend/mcp/backend.go b/mdl/backend/mcp/backend.go index 7b3c1e88b..a7046bd9f 100644 --- a/mdl/backend/mcp/backend.go +++ b/mdl/backend/mcp/backend.go @@ -5,6 +5,7 @@ package mcp import ( "fmt" "os" + "time" "github.com/mendixlabs/mxcli/mdl/backend" "github.com/mendixlabs/mxcli/mdl/types" @@ -68,6 +69,12 @@ type Backend struct { // computed once per connection. capsCache *Capabilities + // settleDelay / settleWindow pace every ped_check_errors call against Studio + // Pro's lagging error list; see pedCheckDocument. Both zero means "ask once, + // now", which is what the unit tests want. New sets them. + settleDelay time.Duration + settleWindow time.Duration + // dirty holds module names whose live (in-memory) domain model has diverged // from the on-disk .mpr because of writes this session. Reads of a dirty // module are reconstructed from MCP instead of the stale local reader — @@ -139,6 +146,8 @@ func New(mcpURL, dial string) *Backend { return &Backend{ mcpURL: mcpURL, dial: dial, + settleDelay: settleDelay, + settleWindow: settleWindow, schemaFetched: map[string]bool{}, dirty: map[string]bool{}, synthetic: map[model.ID]string{}, diff --git a/mdl/backend/mcp/check_settle_test.go b/mdl/backend/mcp/check_settle_test.go new file mode 100644 index 000000000..fe799942f --- /dev/null +++ b/mdl/backend/mcp/check_settle_test.go @@ -0,0 +1,102 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mcp + +import ( + "testing" + "time" +) + +// pedCheckDocument must not believe the first verdict it gets. ped_check_errors +// reads Studio Pro's background error list, which lags the write it should +// reflect (measured live, 26 samples: 75-128ms in both directions). Asked too +// early it answers "No errors found." for a document that was just broken, and +// ped_update_document is no backstop — it reports op-level failures only. +func TestPedCheckDocument_ReasksWhileVerdictIsClean(t *testing.T) { + checks := 0 + f := newFakePED(t, func(name string, _ map[string]any) (string, bool) { + if name != "ped_check_errors" { + return "SUCCESS", false + } + checks++ + if checks == 1 { + return "No errors found.", false // the stale, pre-write verdict + } + return "'M.WF': - Duplicate name 'ReviewOrder'.", false + }) + b := &Backend{client: f.connectClient(t), settleWindow: 500 * time.Millisecond} + + err := b.pedCheckDocument("Workflows$Workflow", "M.WF") + if err == nil { + t.Fatal("a stale clean verdict was accepted; the lagged error was never seen") + } + if checks < 2 { + t.Errorf("ped_check_errors called %d time(s); a clean verdict must be re-asked", checks) + } +} + +// The lag cuts both ways: the error list also still reports an error a +// just-applied op has already cleared. Asking before the delay has elapsed fails +// a perfectly good statement on a leftover verdict. +func TestPedCheckDocument_WaitsBeforeTheFirstAsk(t *testing.T) { + start := time.Now() + var firstAsk time.Duration + checks := 0 + f := newFakePED(t, func(name string, _ map[string]any) (string, bool) { + if name != "ped_check_errors" { + return "SUCCESS", false + } + checks++ + if checks == 1 { + firstAsk = time.Since(start) + } + return "'M.WF': - Duplicate name 'ReviewOrder'.", false + }) + const delay = 200 * time.Millisecond + // No window: past the delay a dirty verdict is current and short-circuits. + b := &Backend{client: f.connectClient(t), settleDelay: delay} + + if err := b.pedCheckDocument("Workflows$Workflow", "M.WF"); err == nil { + t.Fatal("the error must be surfaced") + } + if firstAsk < delay { + t.Errorf("first ped_check_errors at %v, before the %v settle delay: a leftover verdict would be believed", firstAsk, delay) + } + if checks != 1 { + t.Errorf("ped_check_errors called %d times; past the delay an error must short-circuit", checks) + } +} + +// Control: a zero-value Backend asks once and returns, so the unit tests that do +// not care about pacing stay fast. +func TestPedCheckDocument_NoSettleConfiguredIsOneImmediateAsk(t *testing.T) { + checks := 0 + f := newFakePED(t, func(name string, _ map[string]any) (string, bool) { + if name == "ped_check_errors" { + checks++ + return "No errors found.", false + } + return "SUCCESS", false + }) + b := &Backend{client: f.connectClient(t)} + + start := time.Now() + if err := b.pedCheckDocument("Workflows$Workflow", "M.WF"); err != nil { + t.Fatalf("pedCheckDocument: %v", err) + } + if elapsed := time.Since(start); elapsed > 50*time.Millisecond { + t.Errorf("took %v with no settle configured; it must ask once and return", elapsed) + } + if checks != 1 { + t.Errorf("ped_check_errors called %d times, want 1", checks) + } +} + +// New() is what production uses, so that is where the pacing has to be wired — +// a Backend built any other way silently races. +func TestNewBackend_CarriesTheSettlePacing(t *testing.T) { + b := New("http://localhost/mcp", "") + if b.settleDelay != settleDelay || b.settleWindow != settleWindow { + t.Errorf("New() settle = %v/%v, want %v/%v", b.settleDelay, b.settleWindow, settleDelay, settleWindow) + } +} diff --git a/mdl/backend/mcp/domainmodel.go b/mdl/backend/mcp/domainmodel.go index a79451968..5d2757ebe 100644 --- a/mdl/backend/mcp/domainmodel.go +++ b/mdl/backend/mcp/domainmodel.go @@ -1189,10 +1189,64 @@ func (b *Backend) pedCheckErrors(moduleName string) error { return b.pedCheckDocument(domainModelDocType, moduleName) } -// pedCheckDocument validates an arbitrary document and surfaces any errors. +// The settle constants pace pedCheckDocument against Studio Pro's error-list lag. +// +// ped_check_errors reads the error list Studio Pro maintains on a BACKGROUND +// thread, so it reports the model as of some moment before the call. Measured +// live (PED 1.0.0) with a 20ms poll interval, 26 samples spanning both +// directions and three write-load levels (0, 10 and 30 preceding writes): +// +// write -> error becomes visible 77-115ms (median 82ms) +// fix -> error stops being shown 75-128ms (median 83ms) +// +// It is one symmetric debounce, and it did NOT grow with load. (An earlier +// figure of ~170-350ms in the issue #945 notes was an artifact of polling at +// 50-100ms intervals — the granularity, not the lag.) A settled check itself +// costs ~11ms, which is why re-asking is cheap and only the initial wait is not. +// +// Both directions of the lag are damaging, so waiting is the only fix. Asked too +// early the check answers "No errors found." for a document that was just broken +// — a SILENT miss, and ped_update_document is no backstop because it reports +// op-level failures only (a duplicate name comes back as SUCCESS). It equally +// still reports an error a just-applied op has already cleared, which would fail +// a perfectly good statement on a leftover verdict; a multi-op ALTER passes +// through intermediate states that are legitimately invalid, so that is not +// hypothetical. +// +// Hence settleDelay (~2x the observed max) before the first ask, then +// settleWindow of re-asking while the answer stays clean, in case the lag runs +// longer somewhere unmeasured. Re-asking is only safe after settleDelay: past +// it, a dirty verdict is current rather than left over. +const ( + settleDelay = 250 * time.Millisecond + settleWindow = 250 * time.Millisecond + settleInterval = 50 * time.Millisecond +) + +// pedCheckDocument validates an arbitrary document and surfaces any errors, +// waiting out the error-list lag first (see the settle constants). Every write +// path validates through here, so the pacing cannot be forgotten by a new call +// site — the cost is one settle per backend operation, and no operation checks +// more than once. +func (b *Backend) pedCheckDocument(docType, docName string) error { + time.Sleep(b.settleDelay) + deadline := time.Now().Add(b.settleWindow) + for { + if err := b.checkDocumentNow(docType, docName); err != nil { + return err + } + if !time.Now().Before(deadline) { + return nil + } + time.Sleep(settleInterval) + } +} + +// checkDocumentNow asks for the current verdict without waiting for the error +// list to settle — which is why only pedCheckDocument should call it. // ped_check_errors reports a clean document as "No errors found." (with // isError=false); any other text is the validation error(s). -func (b *Backend) pedCheckDocument(docType, docName string) error { +func (b *Backend) checkDocumentNow(docType, docName string) error { res, err := b.client.CallTool("ped_check_errors", map[string]any{ "documents": []map[string]any{ {"documentType": docType, "documentName": docName}, diff --git a/mdl/backend/mcp/microflow.go b/mdl/backend/mcp/microflow.go index 64eb87067..2f17a51a3 100644 --- a/mdl/backend/mcp/microflow.go +++ b/mdl/backend/mcp/microflow.go @@ -184,6 +184,27 @@ func mfKey(m *microflows.Microflow) string { // Nanoflow reads/writes live in nanoflow.go (they reuse buildFlowDocContent). func (b *Backend) IsRule(qualifiedName string) (bool, error) { return b.reader.IsRule(qualifiedName) } +// Rule reads delegate to the local reader, like nanoflow reads: MDL cannot yet +// author a rule over MCP, so there is nothing session-local to merge. +func (b *Backend) ListRules() ([]*microflows.Rule, error) { return b.reader.ListRules() } + +func (b *Backend) GetRule(id model.ID) (*microflows.Rule, error) { return b.reader.GetRule(id) } + +// Rule authoring is not available over MCP — Studio Pro's Model API has no rule +// verb, so there is nothing to delegate to. +func (b *Backend) CreateRule(*microflows.Rule) error { + return errUnsupported("CreateRule") +} +func (b *Backend) UpdateRule(*microflows.Rule) error { + return errUnsupported("UpdateRule") +} +func (b *Backend) DeleteRule(model.ID) error { + return errUnsupported("DeleteRule") +} +func (b *Backend) MoveRule(*microflows.Rule) error { + return errUnsupported("MoveRule") +} + // pedMiddlePoint converts an executor object position to a PED relativeMiddlePoint. // The executor's layout engine already computes these coordinates (the same // value the MPR writer serializes as RelativeMiddlePoint), so reusing them makes diff --git a/mdl/backend/mcp/unsupported_gen.go b/mdl/backend/mcp/unsupported_gen.go index 761a9c484..85f7d3589 100644 --- a/mdl/backend/mcp/unsupported_gen.go +++ b/mdl/backend/mcp/unsupported_gen.go @@ -1077,6 +1077,11 @@ func (unsupportedBackend) SetProjectDemoUsersEnabled(_ model.ID, _ bool) (err0 e return } +func (unsupportedBackend) SetProjectGuestAccess(_ model.ID, _ bool, _ string) (err0 error) { + err0 = errUnsupported("SetProjectGuestAccess") + return +} + func (unsupportedBackend) SetProjectSecurityLevel(_ model.ID, _ string) (err0 error) { err0 = errUnsupported("SetProjectSecurityLevel") return diff --git a/mdl/backend/mcp/workflow.go b/mdl/backend/mcp/workflow.go index b81ec20d2..6842c9bb6 100644 --- a/mdl/backend/mcp/workflow.go +++ b/mdl/backend/mcp/workflow.go @@ -8,6 +8,7 @@ import ( "strings" "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/mdl/backend/wfnames" "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/workflows" ) @@ -567,6 +568,7 @@ type mcpWorkflowMutator struct { moduleName string workflowName string ops []pedOpEntry + applied bool // an activity-level op was sent to PED (see Save) } var _ backend.WorkflowMutator = (*mcpWorkflowMutator)(nil) @@ -607,13 +609,24 @@ func (m *mcpWorkflowMutator) SetPropertyWithEntity(prop, value, entity string) e } // Save flushes the accumulated property sets to Studio Pro and validates. +// +// Activity-level ops have already been sent by the time Save runs (they apply +// eagerly through apply), but they must still be validated: ped_update_document +// only reports op-level failures, so a model-consistency error like CE0495 +// "Duplicate name" comes back as SUCCESS and is visible nowhere except +// ped_check_errors. Save is the single place that check belongs — running it +// per op would pay the error-list settle each time (see pedCheckDocument) and +// would fail on the intermediate states a multi-op ALTER legitimately passes +// through. func (m *mcpWorkflowMutator) Save() error { - if len(m.ops) == 0 { + if len(m.ops) == 0 && !m.applied { return nil } - qn := m.moduleName + "." + m.workflowName - if err := m.backend.pedUpdateDoc(workflowDocType, qn, m.ops...); err != nil { - return err + qn := m.qn() + if len(m.ops) > 0 { + if err := m.backend.pedUpdateDoc(workflowDocType, qn, m.ops...); err != nil { + return err + } } m.backend.markDirty(m.moduleName) return m.backend.pedCheckDocument(workflowDocType, qn) @@ -627,49 +640,75 @@ func (m *mcpWorkflowMutator) qn() string { return m.moduleName + "." + m.workflo type activityRefMatch struct { arrayPath string index int + name string // the activity's own name (activityRef may be its caption) +} + +// wfLocation is a resolved activity location together with the set of every +// activity name in the workflow. Both come out of a single walk of the flow +// tree, so deduplicating an inserted activity's name costs no extra PED reads. +type wfLocation struct { + arrayPath string // PED path of the activities array holding the match + index int // the match's index within that array + actPath string // full PED path to the matched activity element + name string // the matched activity's own name (ref may be its caption) + taken map[string]bool } // resolve finds an activity reference (caption or name, optional 1-based @position) -// anywhere in the flow tree and returns its containing-array path, its index, and -// the full path to the activity element itself. -func (m *mcpWorkflowMutator) resolve(ref string, atPos int) (arrayPath string, index int, actPath string, err error) { +// anywhere in the flow tree and returns its location plus the workflow's +// activity-name set. +func (m *mcpWorkflowMutator) resolve(ref string, atPos int) (wfLocation, error) { var matches []activityRefMatch - if err = m.searchActivities("/flow/activities", ref, &matches); err != nil { - return "", 0, "", err + taken := map[string]bool{} + if err := m.searchActivities("/flow/activities", ref, &matches, taken); err != nil { + return wfLocation{}, err } var pick activityRefMatch switch { case len(matches) == 0: - return "", 0, "", fmt.Errorf("activity %q not found in workflow %q", ref, m.qn()) + return wfLocation{}, fmt.Errorf("activity %q not found in workflow %q", ref, m.qn()) case atPos > 0: if atPos > len(matches) { - return "", 0, "", fmt.Errorf("activity %q @%d not found (only %d matches)", ref, atPos, len(matches)) + return wfLocation{}, fmt.Errorf("activity %q @%d not found (only %d matches)", ref, atPos, len(matches)) } pick = matches[atPos-1] case len(matches) > 1: - return "", 0, "", fmt.Errorf("ambiguous activity %q (%d matches); use @N to disambiguate", ref, len(matches)) + return wfLocation{}, fmt.Errorf("ambiguous activity %q (%d matches); use @N to disambiguate", ref, len(matches)) default: pick = matches[0] } - return pick.arrayPath, pick.index, fmt.Sprintf("%s/%d", pick.arrayPath, pick.index), nil + return wfLocation{ + arrayPath: pick.arrayPath, + index: pick.index, + actPath: fmt.Sprintf("%s/%d", pick.arrayPath, pick.index), + name: pick.name, + taken: taken, + }, nil } // searchActivities walks an activities array and every descendant sub-flow // (each activity's outcome flows, then its boundary-event flows, in order), -// appending every activity whose name or caption equals ref. The depth-first, -// in-order traversal matches DESCRIBE, so @N numbering lines up. -func (m *mcpWorkflowMutator) searchActivities(arrayPath, ref string, matches *[]activityRefMatch) error { +// appending every activity whose name or caption equals ref and recording every +// activity name it passes in taken. The depth-first, in-order traversal matches +// DESCRIBE, so @N numbering lines up. +// +// Name collection rides along on the search rather than being its own pass +// because each level costs a PED round-trip; the two consumers always want both. +func (m *mcpWorkflowMutator) searchActivities(arrayPath, ref string, matches *[]activityRefMatch, taken map[string]bool) error { acts, err := m.readArrayRaw(arrayPath) if err != nil { return err } for i, a := range acts { + if name := mapString(a, "name"); name != "" { + taken[name] = true + } if mapString(a, "name") == ref || mapString(a, "caption") == ref { - *matches = append(*matches, activityRefMatch{arrayPath: arrayPath, index: i}) + *matches = append(*matches, activityRefMatch{arrayPath: arrayPath, index: i, name: mapString(a, "name")}) } actPath := fmt.Sprintf("%s/%d", arrayPath, i) for _, sub := range m.subFlowArrays(actPath, mapString(a, "$Type")) { - if err := m.searchActivities(sub, ref, matches); err != nil { + if err := m.searchActivities(sub, ref, matches, taken); err != nil { return err } } @@ -751,10 +790,11 @@ func mapString(m map[string]any, key string) string { return s } -// apply sends activity-level ops to PED immediately (the existing -// INSERT/DROP/REPLACE activity ops do the same; only the workflow-level SETs -// defer through m.set()/Save()). +// apply sends activity-level ops to PED immediately (only the workflow-level +// SETs defer through m.set()/Save()). It records that a write happened so Save +// still validates the document even when no property set was queued. func (m *mcpWorkflowMutator) apply(ops ...pedOpEntry) error { + m.applied = true return m.backend.pedUpdateDoc(workflowDocType, m.qn(), ops...) } @@ -774,10 +814,11 @@ func (m *mcpWorkflowMutator) SetActivityProperty(activityRef string, atPos int, default: return fmt.Errorf("ALTER WORKFLOW set activity %s is not supported by the MCP backend (supported: page, description, due_date, targeting_microflow, targeting_xpath)", prop) } - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath switch strings.ToLower(prop) { case "targeting_microflow": return m.setUserTargetingLeaf(actPath, "Workflows$MicroflowUserTargeting", "microflow", value) @@ -806,10 +847,12 @@ func (m *mcpWorkflowMutator) setUserTargetingLeaf(actPath, wantType, field, valu } func (m *mcpWorkflowMutator) InsertAfterActivity(activityRef string, atPos int, activities []workflows.WorkflowActivity) error { - arrayPath, idx, _, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + arrayPath, idx := loc.arrayPath, loc.index + wfnames.Dedup(activities, loc.taken) ops := make([]pedOpEntry, 0, len(activities)) for i, a := range activities { mapped, err := mapWorkflowActivity(a) @@ -819,22 +862,29 @@ func (m *mcpWorkflowMutator) InsertAfterActivity(activityRef string, atPos int, at := idx + 1 + i ops = append(ops, pedOpEntry{Path: arrayPath, Operation: pedOperation{Type: "add", Value: mapped, Index: &at}}) } - return m.backend.pedUpdateDoc(workflowDocType, m.qn(), ops...) + return m.apply(ops...) } func (m *mcpWorkflowMutator) DropActivity(activityRef string, atPos int) error { - arrayPath, idx, _, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } - return m.backend.pedUpdateDoc(workflowDocType, m.qn(), removeAtOp(arrayPath, idx)) + return m.apply(removeAtOp(loc.arrayPath, loc.index)) } func (m *mcpWorkflowMutator) ReplaceActivity(activityRef string, atPos int, activities []workflows.WorkflowActivity) error { - arrayPath, idx, _, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + arrayPath, idx := loc.arrayPath, loc.index + // Free the outgoing activity's name before deduplicating: the remove and the + // adds go to PED in one call, so the replaced activity is gone by the time + // any name is resolved, and a replacement reusing its name is the ordinary + // in-place edit rather than a collision (issue #944). + delete(loc.taken, loc.name) + wfnames.Dedup(activities, loc.taken) mapped := make([]map[string]any, 0, len(activities)) for _, a := range activities { mm, err := mapWorkflowActivity(a) @@ -851,17 +901,19 @@ func (m *mcpWorkflowMutator) ReplaceActivity(activityRef string, atPos int, acti at := idx + i ops = append(ops, pedOpEntry{Path: arrayPath, Operation: pedOperation{Type: "add", Value: mm, Index: &at}}) } - return m.backend.pedUpdateDoc(workflowDocType, m.qn(), ops...) + return m.apply(ops...) } // --- outcome / path / branch ops (all live in an activity's `outcomes` array) --- // InsertOutcome adds a named outcome (with an optional sub-flow) to a user task. func (m *mcpWorkflowMutator) InsertOutcome(activityRef string, atPos int, outcomeName string, activities []workflows.WorkflowActivity) error { - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath + wfnames.Dedup(activities, loc.taken) el := map[string]any{"$Type": "Workflows$UserTaskOutcome", "value": outcomeName} if err := attachSubFlow(el, activities); err != nil { return err @@ -871,10 +923,11 @@ func (m *mcpWorkflowMutator) InsertOutcome(activityRef string, atPos int, outcom // DropOutcome removes a user-task outcome by its value ("Default" matches a void outcome). func (m *mcpWorkflowMutator) DropOutcome(activityRef string, atPos int, outcomeName string) error { - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath return m.dropFromActivityArray(actPath, "outcomes", activityRef, "outcome", func(o pedOutcomeElem) bool { return o.valueString() == outcomeName || (strings.EqualFold(outcomeName, "Default") && o.SType == "Workflows$VoidConditionOutcome") @@ -883,10 +936,12 @@ func (m *mcpWorkflowMutator) DropOutcome(activityRef string, atPos int, outcomeN // InsertPath adds a concurrent path (with an optional sub-flow) to a parallel split. func (m *mcpWorkflowMutator) InsertPath(activityRef string, atPos int, pathCaption string, activities []workflows.WorkflowActivity) error { - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath + wfnames.Dedup(activities, loc.taken) el := map[string]any{"$Type": "Workflows$ParallelSplitOutcome"} if err := attachSubFlow(el, activities); err != nil { return err @@ -897,10 +952,11 @@ func (m *mcpWorkflowMutator) InsertPath(activityRef string, atPos int, pathCapti // DropPath removes a parallel-split path. Paths have no stored name; the caption // "Path N" addresses the N-th path, and an empty caption drops the last one. func (m *mcpWorkflowMutator) DropPath(activityRef string, atPos int, pathCaption string) error { - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath paths, err := m.readActivityArray(actPath, "outcomes") if err != nil { return err @@ -923,10 +979,12 @@ func (m *mcpWorkflowMutator) DropPath(activityRef string, atPos int, pathCaption // InsertBranch adds a condition branch (true/false/default/enum-value) to a decision. func (m *mcpWorkflowMutator) InsertBranch(activityRef string, atPos int, condition string, activities []workflows.WorkflowActivity) error { - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath + wfnames.Dedup(activities, loc.taken) el := branchOutcomeElement(condition) if err := attachSubFlow(el, activities); err != nil { return err @@ -936,10 +994,11 @@ func (m *mcpWorkflowMutator) InsertBranch(activityRef string, atPos int, conditi // DropBranch removes a decision branch by name (true/false/default, or an enum value). func (m *mcpWorkflowMutator) DropBranch(activityRef string, atPos int, branchName string) error { - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath return m.dropFromActivityArray(actPath, "outcomes", activityRef, "branch", func(o pedOutcomeElem) bool { switch strings.ToLower(branchName) { case "true": @@ -959,10 +1018,12 @@ func (m *mcpWorkflowMutator) DropBranch(activityRef string, atPos int, branchNam // InsertBoundaryEvent attaches a (non-)interrupting timer boundary event, with an // optional handler sub-flow, to a user task or call-microflow activity. func (m *mcpWorkflowMutator) InsertBoundaryEvent(activityRef string, atPos int, eventType, delay string, activities []workflows.WorkflowActivity) error { - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath + wfnames.Dedup(activities, loc.taken) el := boundaryEventElement(eventType, delay) if err := attachSubFlow(el, activities); err != nil { return err @@ -972,10 +1033,11 @@ func (m *mcpWorkflowMutator) InsertBoundaryEvent(activityRef string, atPos int, // DropBoundaryEvent removes the activity's (first) boundary event. func (m *mcpWorkflowMutator) DropBoundaryEvent(activityRef string, atPos int) error { - _, _, actPath, err := m.resolve(activityRef, atPos) + loc, err := m.resolve(activityRef, atPos) if err != nil { return err } + actPath := loc.actPath events, err := m.readActivityArray(actPath, "boundaryEvents") if err != nil { return err diff --git a/mdl/backend/mcp/workflow_dedup_test.go b/mdl/backend/mcp/workflow_dedup_test.go new file mode 100644 index 000000000..f1071f6d4 --- /dev/null +++ b/mdl/backend/mcp/workflow_dedup_test.go @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mcp + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/sdk/workflows" +) + +// Issue #945. A CALL MICROFLOW activity is named after its target microflow, so +// inserting a call to a microflow the workflow already calls produces a +// colliding name before anything looks at it. Measured against live Studio Pro +// (11.x, PED 1.0.0): ped_update_document accepts the duplicate with SUCCESS and +// does NOT auto-rename, and ped_check_errors then reports +// "Duplicate name 'X'. (at locations: /flow/activities/1, /flow/activities/2)". +// A `set` on the activity's /name is refused ("Element type does not support +// renaming"), so the name has to be right at add time — there is no repair. +func TestWFInsertAfterActivity_DeduplicatesName(t *testing.T) { + f, m := wfMutatorFake(t) + dup := &workflows.CallMicroflowTask{Microflow: "M.ACT_ReviewOrder"} + dup.Name = "ReviewOrder" // the fake flow already has a "ReviewOrder" user task + dup.Caption = "Review the order" + + if err := m.InsertAfterActivity("ReviewOrder", 0, []workflows.WorkflowActivity{dup}); err != nil { + t.Fatalf("InsertAfterActivity: %v", err) + } + + ops := wfUpdateOps(t, f) + if !strings.Contains(ops, `"name":"ReviewOrder_2"`) { + t.Errorf("inserted activity kept the colliding name (CE0495 in Studio Pro): %s", ops) + } +} + +// Control: a name that collides with nothing must be sent through untouched. +func TestWFInsertAfterActivity_LeavesFreeNameAlone(t *testing.T) { + f, m := wfMutatorFake(t) + fresh := &workflows.CallMicroflowTask{Microflow: "M.ACT_Ship"} + fresh.Name = "ShipOrder" + + if err := m.InsertAfterActivity("ReviewOrder", 0, []workflows.WorkflowActivity{fresh}); err != nil { + t.Fatalf("InsertAfterActivity: %v", err) + } + + ops := wfUpdateOps(t, f) + if !strings.Contains(ops, `"name":"ShipOrder"`) || strings.Contains(ops, `"ShipOrder_2"`) { + t.Errorf("a free name must not be renamed: %s", ops) + } +} + +// Nested names count too: uniqueness is workflow-wide, and the fake flow's +// "Decide" lives at the top level while the new activity goes into an outcome +// sub-flow. +func TestWFInsertOutcome_DeduplicatesAgainstWholeWorkflow(t *testing.T) { + f, m := wfMutatorFake(t) + nested := &workflows.CallMicroflowTask{Microflow: "M.ACT_Decide"} + nested.Name = "Decide" + + if err := m.InsertOutcome("ReviewOrder", 0, "Escalate", []workflows.WorkflowActivity{nested}); err != nil { + t.Fatalf("InsertOutcome: %v", err) + } + + ops := wfUpdateOps(t, f) + if !strings.Contains(ops, `"name":"Decide_2"`) { + t.Errorf("sub-flow activity kept a name taken at the top level: %s", ops) + } +} + +// Two calls to the same microflow inserted in one statement collide with each +// other, not just with what is stored. +func TestWFInsertAfterActivity_DeduplicatesWithinTheBatch(t *testing.T) { + f, m := wfMutatorFake(t) + a := &workflows.CallMicroflowTask{Microflow: "M.ACT_Notify"} + a.Name = "Notify" + b := &workflows.CallMicroflowTask{Microflow: "M.ACT_Notify"} + b.Name = "Notify" + + if err := m.InsertAfterActivity("ReviewOrder", 0, []workflows.WorkflowActivity{a, b}); err != nil { + t.Fatalf("InsertAfterActivity: %v", err) + } + + ops := wfUpdateOps(t, f) + if !strings.Contains(ops, `"name":"Notify"`) || !strings.Contains(ops, `"name":"Notify_2"`) { + t.Errorf("batch inserts collided with each other: %s", ops) + } +} + +// ped_update_document reports only op-level failures — a duplicate name comes +// back as SUCCESS. ped_check_errors is the only thing that sees it, so an ALTER +// that touched nothing but activities must still reach it. +func TestWFSave_ValidatesAfterActivityOnlyAlter(t *testing.T) { + f, m := wfMutatorFake(t) + act := &workflows.CallMicroflowTask{Microflow: "M.ACT_Ship"} + act.Name = "ShipOrder" + if err := m.InsertAfterActivity("ReviewOrder", 0, []workflows.WorkflowActivity{act}); err != nil { + t.Fatalf("InsertAfterActivity: %v", err) + } + if err := m.Save(); err != nil { + t.Fatalf("Save: %v", err) + } + if _, ok := f.callByName("ped_check_errors"); !ok { + t.Error("an activity-only ALTER never validated: ped_check_errors was not called") + } +} + +// Control: a mutator that did nothing must not call PED at all. +func TestWFSave_NoOpDoesNotValidate(t *testing.T) { + f, m := wfMutatorFake(t) + if err := m.Save(); err != nil { + t.Fatalf("Save: %v", err) + } + if _, ok := f.callByName("ped_check_errors"); ok { + t.Error("Save with no ops must not call ped_check_errors") + } +} + +// A DROP-only ALTER is a write too, so it validates as well. +func TestWFSave_ValidatesAfterDropOnlyAlter(t *testing.T) { + f, m := wfMutatorFake(t) + if err := m.DropActivity("Decide", 0); err != nil { + t.Fatalf("DropActivity: %v", err) + } + if err := m.Save(); err != nil { + t.Fatalf("Save: %v", err) + } + if _, ok := f.callByName("ped_check_errors"); !ok { + t.Error("a drop-only ALTER never validated: ped_check_errors was not called") + } +} + +// Issue #944. A REPLACE whose replacement reuses the original's name is an +// in-place edit, not a collision — the original leaves in the same PED call. +// PR #204 propagated the file backends' rename into this backend as deliberate +// "parity"; it was neither deliberate nor correct. +func TestWFReplaceActivity_SameNameKeepsName(t *testing.T) { + f, m := wfMutatorFake(t) + rep := &workflows.CallMicroflowTask{Microflow: "M.ACT_Review"} + rep.Name = "ReviewOrder" // the same activity, edited in place + rep.Caption = "Review the order (v2)" + + if err := m.ReplaceActivity("ReviewOrder", 0, []workflows.WorkflowActivity{rep}); err != nil { + t.Fatalf("ReplaceActivity: %v", err) + } + ops := wfUpdateOps(t, f) + if !strings.Contains(ops, `"name":"ReviewOrder"`) || strings.Contains(ops, `"ReviewOrder_2"`) { + t.Errorf("same-name replace renamed the activity: %s", ops) + } +} + +// Control: only the outgoing name is freed. Colliding with a surviving activity +// still dedupes. +func TestWFReplaceActivity_StillDedupesAgainstSurvivors(t *testing.T) { + f, m := wfMutatorFake(t) + rep := &workflows.CallMicroflowTask{Microflow: "M.ACT_Decide"} + rep.Name = "Decide" // "Decide" is a different activity that survives + + if err := m.ReplaceActivity("ReviewOrder", 0, []workflows.WorkflowActivity{rep}); err != nil { + t.Fatalf("ReplaceActivity: %v", err) + } + if ops := wfUpdateOps(t, f); !strings.Contains(ops, `"name":"Decide_2"`) { + t.Errorf("replacement colliding with a survivor must dedupe: %s", ops) + } +} + +// The reference may be the caption, so the freed name has to come from the +// resolved activity. The fake's "ReviewOrder" task has caption "Review the order". +func TestWFReplaceActivity_ResolvedByCaption(t *testing.T) { + f, m := wfMutatorFake(t) + rep := &workflows.CallMicroflowTask{Microflow: "M.ACT_Review"} + rep.Name = "ReviewOrder" + + if err := m.ReplaceActivity("Review the order", 0, []workflows.WorkflowActivity{rep}); err != nil { + t.Fatalf("ReplaceActivity: %v", err) + } + if ops := wfUpdateOps(t, f); !strings.Contains(ops, `"name":"ReviewOrder"`) || strings.Contains(ops, `"ReviewOrder_2"`) { + t.Errorf("resolved-by-caption replace renamed the activity: %s", ops) + } +} diff --git a/mdl/backend/mcp/workflow_test.go b/mdl/backend/mcp/workflow_test.go index 9d3f840e9..b41c887c8 100644 --- a/mdl/backend/mcp/workflow_test.go +++ b/mdl/backend/mcp/workflow_test.go @@ -607,15 +607,21 @@ func TestWFNestedActivityResolution(t *testing.T) { m := &mcpWorkflowMutator{backend: &Backend{client: f.connectClient(t)}, moduleName: "M", workflowName: "WF"} // Resolve lands in the nested flow. - arrayPath, index, actPath, err := m.resolve("NestedCall", 0) + loc, err := m.resolve("NestedCall", 0) if err != nil { t.Fatalf("resolve nested: %v", err) } - if arrayPath != "/flow/activities/1/outcomes/0/flow/activities" || index != 0 { - t.Fatalf("nested resolve = %q[%d], want /flow/activities/1/outcomes/0/flow/activities[0]", arrayPath, index) + if loc.arrayPath != "/flow/activities/1/outcomes/0/flow/activities" || loc.index != 0 { + t.Fatalf("nested resolve = %q[%d], want /flow/activities/1/outcomes/0/flow/activities[0]", loc.arrayPath, loc.index) } - if actPath != "/flow/activities/1/outcomes/0/flow/activities/0" { - t.Fatalf("actPath = %q", actPath) + if loc.actPath != "/flow/activities/1/outcomes/0/flow/activities/0" { + t.Fatalf("actPath = %q", loc.actPath) + } + // The same walk collects every activity name, nested ones included. + for _, want := range []string{"Start", "Review", "NestedCall", "End"} { + if !loc.taken[want] { + t.Errorf("resolve did not record activity name %q: %v", want, loc.taken) + } } // DropActivity on the nested ref removes from the nested array. diff --git a/mdl/backend/microflow.go b/mdl/backend/microflow.go index 9db108fc4..dee451ad7 100644 --- a/mdl/backend/microflow.go +++ b/mdl/backend/microflow.go @@ -33,6 +33,16 @@ type MicroflowBackend interface { DeleteNanoflow(id model.ID) error MoveNanoflow(nf *microflows.Nanoflow) error + // ListRules / GetRule read rule documents (Microflows$Rule). A rule is its + // own doctype, not a microflow variant: ListMicroflows does not return them, + // exactly as it does not return nanoflows or workflows. + ListRules() ([]*microflows.Rule, error) + GetRule(id model.ID) (*microflows.Rule, error) + CreateRule(rule *microflows.Rule) error + UpdateRule(rule *microflows.Rule) error + DeleteRule(id model.ID) error + MoveRule(rule *microflows.Rule) error + // IsRule reports whether the given qualified name refers to a rule // (Microflows$Rule) rather than a microflow. The flow builder uses this // to decide whether an IF condition that looks like a function call diff --git a/mdl/backend/mock/backend.go b/mdl/backend/mock/backend.go index 1b700ae35..ea430a555 100644 --- a/mdl/backend/mock/backend.go +++ b/mdl/backend/mock/backend.go @@ -93,6 +93,12 @@ type MockBackend struct { MoveMicroflowFunc func(mf *microflows.Microflow) error ParseMicroflowFromRawFunc func(raw map[string]any, unitID, containerID model.ID) *microflows.Microflow ParseMicroflowBSONFunc func(contents []byte, unitID, containerID model.ID) (*microflows.Microflow, error) + ListRulesFunc func() ([]*microflows.Rule, error) + GetRuleFunc func(id model.ID) (*microflows.Rule, error) + CreateRuleFunc func(rule *microflows.Rule) error + UpdateRuleFunc func(rule *microflows.Rule) error + DeleteRuleFunc func(id model.ID) error + MoveRuleFunc func(rule *microflows.Rule) error ListNanoflowsFunc func() ([]*microflows.Nanoflow, error) GetNanoflowFunc func(id model.ID) (*microflows.Nanoflow, error) CreateNanoflowFunc func(nf *microflows.Nanoflow) error @@ -141,6 +147,7 @@ type MockBackend struct { GetProjectSecurityFunc func() (*security.ProjectSecurity, error) SetProjectSecurityLevelFunc func(unitID model.ID, level string) error SetProjectDemoUsersEnabledFunc func(unitID model.ID, enabled bool) error + SetProjectGuestAccessFunc func(unitID model.ID, enabled bool, guestUserRole string) error AddUserRoleFunc func(unitID model.ID, name string, moduleRoles []string, manageAllRoles bool) error AlterUserRoleModuleRolesFunc func(unitID model.ID, userRoleName string, add bool, moduleRoles []string) error RemoveUserRoleFunc func(unitID model.ID, name string) error diff --git a/mdl/backend/mock/mock_microflow.go b/mdl/backend/mock/mock_microflow.go index ea915436b..41a150e8c 100644 --- a/mdl/backend/mock/mock_microflow.go +++ b/mdl/backend/mock/mock_microflow.go @@ -65,6 +65,48 @@ func (m *MockBackend) ParseMicroflowBSON(contents []byte, unitID, containerID mo return nil, fmt.Errorf("MockBackend.ParseMicroflowBSON not configured") } +func (m *MockBackend) ListRules() ([]*microflows.Rule, error) { + if m.ListRulesFunc != nil { + return m.ListRulesFunc() + } + return nil, fmt.Errorf("MockBackend.ListRules not configured") +} + +func (m *MockBackend) GetRule(id model.ID) (*microflows.Rule, error) { + if m.GetRuleFunc != nil { + return m.GetRuleFunc(id) + } + return nil, fmt.Errorf("MockBackend.GetRule not configured") +} + +func (m *MockBackend) CreateRule(rule *microflows.Rule) error { + if m.CreateRuleFunc != nil { + return m.CreateRuleFunc(rule) + } + return fmt.Errorf("MockBackend.CreateRule not configured") +} + +func (m *MockBackend) UpdateRule(rule *microflows.Rule) error { + if m.UpdateRuleFunc != nil { + return m.UpdateRuleFunc(rule) + } + return fmt.Errorf("MockBackend.UpdateRule not configured") +} + +func (m *MockBackend) DeleteRule(id model.ID) error { + if m.DeleteRuleFunc != nil { + return m.DeleteRuleFunc(id) + } + return fmt.Errorf("MockBackend.DeleteRule not configured") +} + +func (m *MockBackend) MoveRule(rule *microflows.Rule) error { + if m.MoveRuleFunc != nil { + return m.MoveRuleFunc(rule) + } + return fmt.Errorf("MockBackend.MoveRule not configured") +} + func (m *MockBackend) ListNanoflows() ([]*microflows.Nanoflow, error) { if m.ListNanoflowsFunc != nil { return m.ListNanoflowsFunc() diff --git a/mdl/backend/mock/mock_security.go b/mdl/backend/mock/mock_security.go index cbe181122..47691ee60 100644 --- a/mdl/backend/mock/mock_security.go +++ b/mdl/backend/mock/mock_security.go @@ -3,6 +3,8 @@ package mock import ( + "fmt" + "github.com/mendixlabs/mxcli/mdl/backend" "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/model" @@ -30,6 +32,13 @@ func (m *MockBackend) SetProjectDemoUsersEnabled(unitID model.ID, enabled bool) return nil } +func (m *MockBackend) SetProjectGuestAccess(unitID model.ID, enabled bool, guestUserRole string) error { + if m.SetProjectGuestAccessFunc != nil { + return m.SetProjectGuestAccessFunc(unitID, enabled, guestUserRole) + } + return fmt.Errorf("MockBackend.SetProjectGuestAccess not configured") +} + func (m *MockBackend) AddUserRole(unitID model.ID, name string, moduleRoles []string, manageAllRoles bool) error { if m.AddUserRoleFunc != nil { return m.AddUserRoleFunc(unitID, name, moduleRoles, manageAllRoles) diff --git a/mdl/backend/modelsdk/domainmodel_alter.go b/mdl/backend/modelsdk/domainmodel_alter.go index df943eeba..b5f08b994 100644 --- a/mdl/backend/modelsdk/domainmodel_alter.go +++ b/mdl/backend/modelsdk/domainmodel_alter.go @@ -144,17 +144,51 @@ func (b *Backend) UpdateEntity(domainModelID model.ID, entity *domainmodel.Entit ge.SetRaw(raw) } - // When the update removes ALL indexes but the stored entity had some, the fresh - // (empty) Indexes list on ge is "clean" (untouched), so the codec passes the - // raw indexes through unchanged — a dropped indexed attribute would then leave - // an orphaned index pointing at a GUID that no longer exists, which crashes - // `mx check` with an unhandled AggregateException (ledger #39). Touch the list - // (append + remove) to mark it dirty so the codec re-emits it as empty, - // clearing the raw. A non-empty new index list is already dirty (entityToGen - // appended to it), so this is only needed for the all-removed case. - if len(entity.Indexes) == 0 && len(orig.IndexesItems()) > 0 { - ge.AddIndexes(genDm.NewIndex()) - ge.RemoveIndexes(0) + // When an update empties a child list, the fresh (empty) list on ge is "clean" + // — entityToGen appended nothing to it — so the codec passes the STORED raw + // bytes through unchanged and the removal silently does not happen. Touching + // the list (append + remove) marks it dirty, so the codec re-emits it as empty + // and clears the raw. A list that still has members is already dirty from + // entityToGen's appends, which is why this is only needed for the last one out. + // + // That "last one out" is the whole trap: any test that removes one of two + // members passes against the broken code. Measured on 11.13.0, one member each: + // + // Indexes orphaned index → `mx check` dies with an unhandled + // AggregateException (ledger #39) + // ValidationRules rule outlives the attribute it constrains → CE1613 + // Attributes DROP ATTRIBUTE reports success and writes nothing at all + // + // AccessRules and EventHandlers share the mechanism; they are covered here + // rather than left to be rediscovered one error code at a time. + for _, l := range []struct { + newLen, storedLen int + touch func() + }{ + {len(entity.Attributes), len(orig.AttributesItems()), func() { + ge.AddAttributes(genDm.NewAttribute()) + ge.RemoveAttributes(0) + }}, + {len(entity.ValidationRules), len(orig.ValidationRulesItems()), func() { + ge.AddValidationRules(genDm.NewValidationRule()) + ge.RemoveValidationRules(0) + }}, + {len(entity.Indexes), len(orig.IndexesItems()), func() { + ge.AddIndexes(genDm.NewIndex()) + ge.RemoveIndexes(0) + }}, + {len(entity.AccessRules), len(orig.AccessRulesItems()), func() { + ge.AddAccessRules(genDm.NewAccessRule()) + ge.RemoveAccessRules(0) + }}, + {len(entity.EventHandlers), len(orig.EventHandlersItems()), func() { + ge.AddEventHandlers(genDm.NewEventHandler()) + ge.RemoveEventHandlers(0) + }}, + } { + if l.newLen == 0 && l.storedLen > 0 { + l.touch() + } } // Rebuild the list in place: drop all, re-add in original order swapping the diff --git a/mdl/backend/modelsdk/emptied_list_write_test.go b/mdl/backend/modelsdk/emptied_list_write_test.go new file mode 100644 index 000000000..6cb5e6a63 --- /dev/null +++ b/mdl/backend/modelsdk/emptied_list_write_test.go @@ -0,0 +1,174 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// emptiedListEntity creates an entity carrying exactly ONE attribute and ONE +// validation rule on it, then hands back a fresh read of it. +// +// One of each is the whole point. A child list that still has members after an +// update is dirty (entityToGen appended to it), so the codec re-emits it and +// everything looks fine; the raw-passthrough bug only shows when the update +// removes the LAST member. A fixture with two of anything passes against the +// broken writer — measured, not assumed. +func emptiedListEntity(t *testing.T, name string) (*Backend, string, model.ID, *domainmodel.Entity) { + // modID, not the domain-model ID: GetDomainModel is keyed by MODULE, while + // UpdateEntity takes the domain model's own ID. Carrying the module ID and + // re-reading the domain model keeps that straight across a reconnect. + t.Helper() + proj := copyFixture(t) + + b := New() + if err := b.Connect(proj); err != nil { + t.Fatalf("connect: %v", err) + } + mod, err := b.GetModuleByName("MyFirstModule") + if err != nil || mod == nil { + t.Fatalf("GetModuleByName(MyFirstModule) = %v, %v", mod, err) + } + dm, err := b.GetDomainModel(mod.ID) + if err != nil || dm == nil { + t.Fatalf("GetDomainModel = %v, %v", dm, err) + } + + ent := &domainmodel.Entity{Name: name, Persistable: true} + ent.Attributes = []*domainmodel.Attribute{{Name: "Solo", Type: &domainmodel.StringAttributeType{}}} + if err := b.CreateEntity(dm.ID, ent); err != nil { + t.Fatalf("CreateEntity: %v", err) + } + + stored := reread(t, b, mod.ID, name) + if len(stored.Attributes) != 1 { + t.Fatalf("fixture entity has %d attributes, want exactly 1", len(stored.Attributes)) + } + // Add the validation rule now that the attribute has a persisted identity. + stored.ValidationRules = []*domainmodel.ValidationRule{{ + BaseElement: model.BaseElement{ID: model.ID(stored.Attributes[0].ID)}, + AttributeID: model.ID("MyFirstModule." + name + ".Solo"), + Type: "Required", + }} + if err := b.UpdateEntity(dm.ID, stored); err != nil { + t.Fatalf("UpdateEntity (seed rule): %v", err) + } + + stored = reread(t, b, mod.ID, name) + if len(stored.ValidationRules) != 1 { + t.Fatalf("seeded %d validation rules, want exactly 1", len(stored.ValidationRules)) + } + return b, proj, mod.ID, stored +} + +// dmIDFor returns the domain model's own ID, which UpdateEntity requires. +func dmIDFor(t *testing.T, b *Backend, modID model.ID) model.ID { + t.Helper() + dm, err := b.GetDomainModel(modID) + if err != nil || dm == nil { + t.Fatalf("GetDomainModel = %v, %v", dm, err) + } + return dm.ID +} + +func reread(t *testing.T, b *Backend, modID model.ID, name string) *domainmodel.Entity { + t.Helper() + dm, err := b.GetDomainModel(modID) + if err != nil || dm == nil { + t.Fatalf("GetDomainModel = %v, %v", dm, err) + } + for _, e := range dm.Entities { + if e.Name == name { + return e + } + } + t.Fatalf("entity %q not found", name) + return nil +} + +// reopen closes the backend and returns a fresh one, so assertions are made +// against what reached disk rather than against in-memory state. +func reopen(t *testing.T, b *Backend, proj string, modID model.ID, name string) *domainmodel.Entity { + t.Helper() + if err := b.Disconnect(); err != nil { + t.Fatalf("disconnect: %v", err) + } + b2 := New() + if err := b2.Connect(proj); err != nil { + t.Fatalf("reconnect: %v", err) + } + t.Cleanup(func() { _ = b2.Disconnect() }) + return reread(t, b2, modID, name) +} + +// TestUpdateEntityPersistsRemovalOfLastValidationRule is the regression for the +// CE1613 that DROP ATTRIBUTE produced: +// +// [error] [CE1613] "The selected attribute 'DropAttr.Account.AccountNumber' +// no longer exists." at Validation rule of entity 'DropAttr.Account' +// +// The executor removed the rule correctly; the write dropped the removal on the +// floor, because an emptied child list is "clean" and the codec then keeps the +// stored raw bytes. +func TestUpdateEntityPersistsRemovalOfLastValidationRule(t *testing.T) { + b, proj, modID, ent := emptiedListEntity(t, "EmptiedRules") + + ent.ValidationRules = nil + if err := b.UpdateEntity(dmIDFor(t, b, modID), ent); err != nil { + t.Fatalf("UpdateEntity: %v", err) + } + + got := reopen(t, b, proj, modID, "EmptiedRules") + if len(got.ValidationRules) != 0 { + t.Errorf("%d validation rule(s) survived after removing the last one — this is CE1613", len(got.ValidationRules)) + } + // The rest of the entity must be intact: a writer that clears the list by + // clearing the entity would pass the check above. + if len(got.Attributes) != 1 { + t.Errorf("attributes = %d, want 1 (untouched)", len(got.Attributes)) + } +} + +// TestUpdateEntityPersistsRemovalOfLastAttribute covers the same mechanism on +// the Attributes list, where it is worse than a dangling reference: DROP +// ATTRIBUTE reported success and wrote nothing at all. +func TestUpdateEntityPersistsRemovalOfLastAttribute(t *testing.T) { + b, proj, modID, ent := emptiedListEntity(t, "EmptiedAttrs") + + // The rule has to go with it — a rule referencing a removed attribute is + // the CE1613 above, not what this test is about. + ent.ValidationRules = nil + ent.Attributes = nil + if err := b.UpdateEntity(dmIDFor(t, b, modID), ent); err != nil { + t.Fatalf("UpdateEntity: %v", err) + } + + got := reopen(t, b, proj, modID, "EmptiedAttrs") + if len(got.Attributes) != 0 { + t.Errorf("%d attribute(s) survived after removing the last one — the drop silently did nothing", len(got.Attributes)) + } +} + +// TestUpdateEntityKeepsRemainingListMembers is the positive control. Removing +// one of TWO rules always worked, because the surviving member dirties the +// list — so this is the case that would have hidden the bug. It has to keep +// passing, or the fix has traded a stale list for a cleared one. +func TestUpdateEntityKeepsRemainingListMembers(t *testing.T) { + b, proj, modID, ent := emptiedListEntity(t, "KeptRules") + + ent.Attributes = append(ent.Attributes, &domainmodel.Attribute{Name: "Second", Type: &domainmodel.StringAttributeType{}}) + if err := b.UpdateEntity(dmIDFor(t, b, modID), ent); err != nil { + t.Fatalf("UpdateEntity (add second attribute): %v", err) + } + + got := reopen(t, b, proj, modID, "KeptRules") + if len(got.Attributes) != 2 { + t.Fatalf("attributes = %d, want 2", len(got.Attributes)) + } + if len(got.ValidationRules) != 1 { + t.Errorf("validation rules = %d, want 1 (the seeded rule must survive an unrelated update)", len(got.ValidationRules)) + } +} diff --git a/mdl/backend/modelsdk/identity_drift_test.go b/mdl/backend/modelsdk/identity_drift_test.go index e4c676543..8c3452602 100644 --- a/mdl/backend/modelsdk/identity_drift_test.go +++ b/mdl/backend/modelsdk/identity_drift_test.go @@ -37,6 +37,24 @@ import ( // re-minting it is harmless — "it looked random" is not one. var churnIsIntended = map[string][]string{} +// The guard's blind spot, which is how Workflows$*.PersistentId got in. +// +// It sees only properties minted through the codec's FreshGUIDFields +// registration. A property minted by hand in a writer is invisible to it — +// PersistentId was emitted by addFreshPersistentID (modelsdk) and a literal +// idToBsonBinary(generateUUID()) (legacy), so no registration existed and +// nothing complained while every workflow write churned it (issue #949). +// +// It is now carried by canon.CarryPersistentIDs rather than registered here, +// because it lives on nested elements: identityFields/CarryIdentity only reach +// top-level properties of the document root, and PersistentId sits on activities +// and outcomes arbitrarily deep in the flow tree. +// +// So the rule this guard encodes still holds — a fresh GUID needs a recorded +// decision — but "registered with the codec" is narrower than "minted fresh". +// When adding a writer that mints an identity by hand, either register it so +// this guard can see it, or carry it in canon and say so here. + func TestFreshGUIDFieldsHaveAnIdentityDecision(t *testing.T) { registered := codec.FreshGUIDRegistrations() if len(registered) == 0 { diff --git a/mdl/backend/modelsdk/microflow.go b/mdl/backend/modelsdk/microflow.go index 563365470..23cfc1b1f 100644 --- a/mdl/backend/modelsdk/microflow.go +++ b/mdl/backend/modelsdk/microflow.go @@ -129,6 +129,63 @@ func nanoflowFromGen(nf *genMf.Nanoflow, containerID model.ID) *microflows.Nanof return out } +// ListRules returns every Microflows$Rule document. A rule is its own doctype, +// so it is deliberately absent from ListMicroflows — SHOW MICROFLOWS lists +// microflows only, as it already does for nanoflows and workflows. +func (b *Backend) ListRules() ([]*microflows.Rule, error) { + units, err := mprread.ListUnitsWithContainer[*genMf.Rule](b.reader) + if err != nil { + return nil, err + } + out := make([]*microflows.Rule, 0, len(units)) + for _, u := range units { + out = append(out, ruleFromGen(u.Element, u.ContainerID)) + } + return out, nil +} + +func (b *Backend) GetRule(id model.ID) (*microflows.Rule, error) { + units, err := mprread.ListUnitsWithContainer[*genMf.Rule](b.reader) + if err != nil { + return nil, err + } + for _, u := range units { + if model.ID(u.Element.ID()) == id { + return ruleFromGen(u.Element, u.ContainerID), nil + } + } + return nil, nil +} + +// ruleFromGen mirrors nanoflowFromGen — a rule shares the parameter, flow-object +// and return-type structures with a microflow, so the same helpers apply. +// +// gen also declares a ReturnType string beside MicroflowReturnType. Studio Pro +// 11.13 does not write it (measured on both reference rules) and +// generated/metamodel does not list it, so it is not read here and must not be +// written: it is a pre-7 legacy property with nothing to carry through. +func ruleFromGen(r *genMf.Rule, containerID model.ID) *microflows.Rule { + out := µflows.Rule{ + ContainerID: containerID, + Name: r.Name(), + Documentation: r.Documentation(), + Excluded: r.Excluded(), + MarkAsUsed: r.MarkAsUsed(), + ApplyEntityAccess: r.ApplyEntityAccess(), + ReturnVariableName: r.ReturnVariableName(), + ReturnType: dataTypeFromGen(r.MicroflowReturnType()), + } + out.ID = model.ID(r.ID()) + params, objs := splitFlowObjects(r.ObjectCollection()) + out.Parameters = params + flows := flowsFromGen(r.FlowsItems()) + annotFlows := annotationFlowsFromGen(r.FlowsItems()) + if objs != nil || flows != nil || annotFlows != nil { + out.ObjectCollection = µflows.MicroflowObjectCollection{Objects: objs, Flows: flows, AnnotationFlows: annotFlows} + } + return out +} + func microflowFromGen(mf *genMf.Microflow, containerID model.ID) *microflows.Microflow { out := µflows.Microflow{ ContainerID: containerID, diff --git a/mdl/backend/modelsdk/microflow_rule_split_write_test.go b/mdl/backend/modelsdk/microflow_rule_split_write_test.go new file mode 100644 index 000000000..0e12e17d7 --- /dev/null +++ b/mdl/backend/modelsdk/microflow_rule_split_write_test.go @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/codec" + "github.com/mendixlabs/mxcli/sdk/microflows" + "go.mongodb.org/mongo-driver/v2/bson" +) + +// ruleSplitMicroflow is the shape upstream #939 reported: a decision whose +// condition calls a rule, which MDL writes for `if Module.SomeRule(arg = $x)`. +func ruleSplitMicroflow() *microflows.Microflow { + cond := µflows.RuleSplitCondition{ + RuleQualifiedName: "Sample.Rule_IsActive", + ParameterMappings: []*microflows.RuleCallParameterMapping{{ + ParameterName: "Sample.Rule_IsActive.IsActive", + Argument: "$IsActive", + }}, + } + cond.ID = model.ID("cond-1") + cond.ParameterMappings[0].ID = model.ID("pm-1") + + split := µflows.ExclusiveSplit{ + Caption: "Sample.Rule_IsActive(IsActive = $IsActive)", + SplitCondition: cond, + } + split.ID = model.ID("split-1") + + mf := µflows.Microflow{ + Name: "MF_UseRule", + ObjectCollection: µflows.MicroflowObjectCollection{ + Objects: []microflows.MicroflowObject{split}, + }, + } + mf.ID = model.ID("mf-1") + return mf +} + +// TestMicroflowRoundTrip_RuleSplitCondition guards upstream #939. splitConditionToGen +// had no *microflows.RuleSplitCondition case, so the condition hit `default: return +// nil` and the caller skipped SetSplitCondition entirely — the decision was stored +// with no Condition at all. Measured on mxbuild 11.13.0: CE0080 "The 'Condition' +// property is required", with the Decision's caption still showing the call, while +// the same script on the legacy engine checked clean. +// +// The read side had been implemented for #723, which is what made this silent: +// `describe microflow` rendered `if true then` and the caption kept the original +// text, so the round-trip looked like valid MDL either way. +func TestMicroflowRoundTrip_RuleSplitCondition(t *testing.T) { + got := roundTripMicroflow(t, ruleSplitMicroflow()) + + var split *microflows.ExclusiveSplit + if got.ObjectCollection != nil { + for _, obj := range got.ObjectCollection.Objects { + if s, ok := obj.(*microflows.ExclusiveSplit); ok { + split = s + } + } + } + if split == nil { + t.Fatal("ExclusiveSplit did not survive the round trip") + } + cond, ok := split.SplitCondition.(*microflows.RuleSplitCondition) + if !ok { + t.Fatalf("SplitCondition = %T, want *microflows.RuleSplitCondition — a nil condition "+ + "is mx check CE0080 and renders as `if true then`", split.SplitCondition) + } + if cond.RuleQualifiedName != "Sample.Rule_IsActive" { + t.Errorf("RuleQualifiedName = %q, want Sample.Rule_IsActive", cond.RuleQualifiedName) + } + if len(cond.ParameterMappings) != 1 { + t.Fatalf("ParameterMappings = %d, want 1", len(cond.ParameterMappings)) + } + if pm := cond.ParameterMappings[0]; pm.ParameterName != "Sample.Rule_IsActive.IsActive" || pm.Argument != "$IsActive" { + t.Errorf("parameter mapping = {%q, %q}, want {Sample.Rule_IsActive.IsActive, $IsActive}", + pm.ParameterName, pm.Argument) + } +} + +// The round trip above passes whichever key the rule name is stored under, as long +// as both sides agree — so it cannot see the second half of #939: modelsdk/gen binds +// Microflows$RuleCall.Rule to the BSON key "Rule", and Mendix stores it as +// "Microflow" (rules share the microflow namespace). generated/metamodel +// (`json:"microflow"`), the legacy writer and modelsdk/gen/keyaudit_test.go all +// agree. Written as "Rule" the reference is invisible to Mendix and the decision is +// CE0080 again, for a different reason. Assert the storage key on the raw document. +func TestRuleSplitConditionUsesMicroflowStorageKey(t *testing.T) { + raw, err := (&codec.Encoder{}).Encode(microflowToGen(ruleSplitMicroflow(), 11)) + if err != nil { + t.Fatalf("encode: %v", err) + } + + objects, err := bson.Raw(raw).LookupErr("ObjectCollection", "Objects") + if err != nil { + t.Fatalf("no ObjectCollection.Objects: %v", err) + } + vals, err := objects.Array().Values() + if err != nil { + t.Fatalf("Objects array: %v", err) + } + var ruleCall bson.Raw + for _, v := range vals { + doc, ok := v.DocumentOK() // the leading int32 typed-array marker is not a document + if !ok { + continue + } + if rc, err := doc.LookupErr("SplitCondition", "RuleCall"); err == nil { + ruleCall = rc.Document() + } + } + if ruleCall == nil { + t.Fatal("no SplitCondition.RuleCall on the encoded split") + } + + if _, err := ruleCall.LookupErr("Rule"); err == nil { + t.Error("RuleCall stores the reference under \"Rule\"; Mendix reads \"Microflow\" and " + + "reports CE0080 \"The 'Condition' property is required\"") + } + name, err := ruleCall.LookupErr("Microflow") + if err != nil { + t.Fatalf("RuleCall has no \"Microflow\" key: %v", err) + } + if s, _ := name.StringValueOK(); s != "Sample.Rule_IsActive" { + t.Errorf("Microflow = %q, want Sample.Rule_IsActive", s) + } + + // Studio Pro and the legacy writer lead the ParameterMappings list with + // typed-array marker 2, not the codec's default 3. + mappings, err := ruleCall.LookupErr("ParameterMappings") + if err != nil { + t.Fatalf("RuleCall has no ParameterMappings: %v", err) + } + entries, err := mappings.Array().Values() + if err != nil || len(entries) == 0 { + t.Fatalf("ParameterMappings array: %v", err) + } + if m, ok := entries[0].Int32OK(); !ok || m != 2 { + t.Errorf("ParameterMappings marker = %v, want int32 2", entries[0]) + } +} diff --git a/mdl/backend/modelsdk/microflow_write.go b/mdl/backend/modelsdk/microflow_write.go index 9e6693097..3144a5fb5 100644 --- a/mdl/backend/modelsdk/microflow_write.go +++ b/mdl/backend/modelsdk/microflow_write.go @@ -49,6 +49,12 @@ func init() { codec.RegisterTypeDefaults("Microflows$NanoflowCall", codec.TypeDefaults{ MandatoryListMarkers: map[string]int32{"ParameterMappings": 2}, }) + // A rule split's RuleCall follows the same shape: the ParameterMappings list + // is always emitted, with marker 2 (legacy writer, verified vs mxbuild 11.13). + codec.RegisterTypeDefaults("Microflows$RuleCall", codec.TypeDefaults{ + MandatoryListMarkers: map[string]int32{"ParameterMappings": 2}, + }) + codec.RegisterListMarker("Microflows$RuleCallParameterMapping", 2) codec.RegisterListMarker("Microflows$MicroflowCallParameterMapping", 2) codec.RegisterListMarker("Microflows$NanoflowCallParameterMapping", 2) // A JavaActionCallAction always serializes QueueSettings as null; its @@ -1179,8 +1185,15 @@ func entityRefToGen(steps []microflows.EntityRefStep) element.Element { return ref } -// splitConditionToGen builds an exclusive-split condition. Rule conditions are a -// later slice (RuleCall + parameter mappings). +// splitConditionToGen builds an exclusive-split condition — either an expression +// or a call into a rule. Inverse of splitConditionFromGen. +// +// The rule case is the one #939 reported: it used to fall through to nil and the +// caller skipped SetSplitCondition entirely, so a decision that reads +// `if Module.SomeRule(...) then` was stored with its Condition missing — mx check +// CE0080 "The 'Condition' property is required", with the Decision's caption +// still showing the call. The legacy engine had always written it +// (sdk/mpr/writer_microflow.go), which is why the same script passed there. func splitConditionToGen(sc microflows.SplitCondition) element.Element { switch c := sc.(type) { case *microflows.ExpressionSplitCondition: @@ -1188,11 +1201,34 @@ func splitConditionToGen(sc microflows.SplitCondition) element.Element { g.SetID(element.ID(c.ID)) g.SetExpression(c.Expression) return g + case *microflows.RuleSplitCondition: + g := genMf.NewRuleSplitCondition() + g.SetID(element.ID(c.ID)) + g.SetRuleCall(ruleCallToGen(c)) + return g default: return nil } } +// ruleCallToGen builds the RuleCall sub-document a RuleSplitCondition wraps. The +// rule's qualified name goes to the "Microflow" storage key (patched in +// initRuleCall); the RuleCall itself carries no semantic ID, so it gets a fresh +// one like every other synthesized part. +func ruleCallToGen(c *microflows.RuleSplitCondition) element.Element { + rc := genMf.NewRuleCall() + assignID(rc) + rc.SetRuleQualifiedName(c.RuleQualifiedName) + for _, pm := range c.ParameterMappings { + g := genMf.NewRuleCallParameterMapping() + g.SetID(element.ID(pm.ID)) + g.SetParameterQualifiedName(pm.ParameterName) + g.SetArgument(pm.Argument) + rc.AddParameterMappings(g) + } + return rc +} + // caseValueToGen renders a sequence-flow case. ExpressionCase is serialized AS an // EnumerationCase with Value = the expression ("true"/"false") — Studio Pro has // never recognised Microflows$ExpressionCase (verified vs legacy). Default NoCase. diff --git a/mdl/backend/modelsdk/move_documents_write.go b/mdl/backend/modelsdk/move_documents_write.go index 9919b7f03..0511edf0b 100644 --- a/mdl/backend/modelsdk/move_documents_write.go +++ b/mdl/backend/modelsdk/move_documents_write.go @@ -36,6 +36,13 @@ func (b *Backend) MoveNanoflow(nf *microflows.Nanoflow) error { return b.moveUnit(nf.ID, nf.ContainerID, "Nanoflow") } +func (b *Backend) MoveRule(rule *microflows.Rule) error { + if rule == nil { + return fmt.Errorf("MoveRule: nil rule") + } + return b.moveUnit(rule.ID, rule.ContainerID, "Rule") +} + func (b *Backend) MovePage(page *pages.Page) error { if page == nil { return fmt.Errorf("MovePage: nil page") diff --git a/mdl/backend/modelsdk/rule_read_test.go b/mdl/backend/modelsdk/rule_read_test.go new file mode 100644 index 000000000..de2c08a0e --- /dev/null +++ b/mdl/backend/modelsdk/rule_read_test.go @@ -0,0 +1,119 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/codec" + "github.com/mendixlabs/mxcli/modelsdk/element" + genDT "github.com/mendixlabs/mxcli/modelsdk/gen/datatypes" + genMf "github.com/mendixlabs/mxcli/modelsdk/gen/microflows" + "github.com/mendixlabs/mxcli/sdk/microflows" + "go.mongodb.org/mongo-driver/v2/bson" +) + +// studioProRule is Rules.Rule2 from the reference app (ako/TestApp, Mendix +// 11.13.0), authored in Studio Pro: an enumeration-returning rule with an entity +// parameter. It is built through gen rather than pasted as BSON so the encode +// and decode key bindings are both exercised. +// +// Two properties of the real document are load-bearing here and are asserted +// below: a rule stores NO AllowedModuleRoles (it is not independently callable), +// and its return type lives under MicroflowReturnType — gen's sibling +// ReturnType string is not written by Studio Pro and must not be read. +func studioProRule(t *testing.T) *genMf.Rule { + t.Helper() + r := genMf.NewRule() + r.SetID(element.ID("22222222-2222-2222-2222-222222222222")) + r.SetName("Rule2") + r.SetDocumentation("") + r.SetExportLevel("Hidden") + r.SetReturnVariableName("Variable") + + enum := genDT.NewEnumerationType() + enum.SetID(element.ID("33333333-3333-3333-3333-333333333333")) + enum.SetEnumerationQualifiedName("Rules.RuleResult") + r.SetMicroflowReturnType(enum) + + oc := genMf.NewMicroflowObjectCollection() + oc.SetID(element.ID("44444444-4444-4444-4444-444444444444")) + + param := genMf.NewMicroflowParameter() + param.SetID(element.ID("55555555-5555-5555-5555-555555555555")) + param.SetName("pName") + objType := genDT.NewObjectType() + objType.SetID(element.ID("66666666-6666-6666-6666-666666666666")) + objType.SetEntityQualifiedName("Pages.Bus") + param.SetParameterType(objType) + oc.AddObjects(param) + + r.SetObjectCollection(oc) + return r +} + +// A rule round-trips through the codec with its enumeration return type and its +// parameter intact. The enumeration case matters: microflows.Rule's doc comment +// used to claim the return type is "always boolean", which Rules.Rule2 disproves. +func TestRuleFromGen_EnumerationReturnAndEntityParameter(t *testing.T) { + encoded, err := (&codec.Encoder{}).Encode(studioProRule(t)) + if err != nil { + t.Fatalf("encode rule: %v", err) + } + el, err := codec.NewDecoder(codec.DefaultRegistry).Decode(bson.Raw(encoded)) + if err != nil { + t.Fatalf("decode rule: %v", err) + } + g, ok := el.(*genMf.Rule) + if !ok { + t.Fatalf("decoded %T, want *genMf.Rule", el) + } + + rule := ruleFromGen(g, model.ID("container-1")) + if rule.Name != "Rule2" { + t.Errorf("Name = %q, want Rule2", rule.Name) + } + if rule.ReturnVariableName != "Variable" { + t.Errorf("ReturnVariableName = %q, want %q — Studio Pro writes it and a rewrite must not drop it", + rule.ReturnVariableName, "Variable") + } + enum, ok := rule.ReturnType.(*microflows.EnumerationType) + if !ok { + t.Fatalf("ReturnType = %T, want *microflows.EnumerationType (a rule may return an enumeration, not only Boolean)", rule.ReturnType) + } + if enum.EnumerationQualifiedName != "Rules.RuleResult" { + t.Errorf("enumeration = %q, want Rules.RuleResult", enum.EnumerationQualifiedName) + } + if len(rule.Parameters) != 1 || rule.Parameters[0].Name != "pName" { + t.Fatalf("Parameters = %+v, want one named pName", rule.Parameters) + } +} + +// The storage keys a rule does NOT carry. Measured on both Studio Pro reference +// rules: neither AllowedModuleRoles nor ReturnType appears, where a microflow in +// the same app stores nine properties a rule has no concept of. Writing either +// would be inventing model the user does not have. +func TestEncodedRuleOmitsMicroflowOnlyKeys(t *testing.T) { + encoded, err := (&codec.Encoder{}).Encode(studioProRule(t)) + if err != nil { + t.Fatalf("encode rule: %v", err) + } + raw := bson.Raw(encoded) + for _, key := range []string{ + "AllowedModuleRoles", "ReturnType", "AllowConcurrentExecution", + "ConcurrenyErrorMessage", "ConcurrencyErrorMicroflow", "StableId", + "Url", "UrlSearchParameters", "MicroflowActionInfo", "WorkflowActionInfo", + } { + if _, err := raw.LookupErr(key); err == nil { + t.Errorf("encoded rule carries %q; Studio Pro writes only the ten rule properties", key) + } + } + // Control: the keys it must carry are present, so the loop above is not + // passing because the document is empty. + for _, key := range []string{"Name", "ObjectCollection", "MicroflowReturnType", "ReturnVariableName"} { + if _, err := raw.LookupErr(key); err != nil { + t.Errorf("encoded rule is missing %q", key) + } + } +} diff --git a/mdl/backend/modelsdk/rule_write.go b/mdl/backend/modelsdk/rule_write.go new file mode 100644 index 000000000..204bdadc3 --- /dev/null +++ b/mdl/backend/modelsdk/rule_write.go @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/codec" + "github.com/mendixlabs/mxcli/modelsdk/element" + genMf "github.com/mendixlabs/mxcli/modelsdk/gen/microflows" + mmpr "github.com/mendixlabs/mxcli/modelsdk/mpr" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +func init() { + // Studio Pro writes Flows on every rule, as the bare typed-array marker when + // the rule has none. Without this a rule whose body produced no sequence + // flows omitted the key entirely — one key short of the reference document, + // and `mx check` passes either way. + codec.RegisterTypeDefaults("Microflows$Rule", codec.TypeDefaults{ + MandatoryLists: []string{"Flows"}, + }) +} + +// CreateRule inserts a new Microflows$Rule document unit. A rule shares the +// microflow flow model (parameters + object collection + sequence flows), so it +// reuses the microflow object/flow converters; only the top-level field set +// differs, and it differs by omission — see ruleToGen. +func (b *Backend) CreateRule(rule *microflows.Rule) error { + if rule == nil { + return fmt.Errorf("CreateRule: nil rule") + } + if b.writer == nil { + return fmt.Errorf("CreateRule: not connected for writing") + } + if rule.ID == "" { + rule.ID = model.ID(mmpr.GenerateID()) + } + g := ruleToGen(rule, b.majorVersion()) + g.SetID(element.ID(rule.ID)) + assignRuleIDs(g) + contents, err := (&codec.Encoder{}).Encode(g) + if err != nil { + return fmt.Errorf("CreateRule: encode: %w", err) + } + if err := b.writer.InsertUnit(string(rule.ID), string(rule.ContainerID), "Documents", "Microflows$Rule", contents); err != nil { + return fmt.Errorf("CreateRule: insert: %w", err) + } + return nil +} + +// UpdateRule rebuilds a rule document (the CREATE OR REPLACE path). +func (b *Backend) UpdateRule(rule *microflows.Rule) error { + if rule == nil { + return fmt.Errorf("UpdateRule: nil rule") + } + if b.writer == nil { + return fmt.Errorf("UpdateRule: not connected for writing") + } + g := ruleToGen(rule, b.majorVersion()) + g.SetID(element.ID(rule.ID)) + assignRuleIDs(g) + contents, err := (&codec.Encoder{}).Encode(g) + if err != nil { + return fmt.Errorf("UpdateRule: encode: %w", err) + } + if err := b.writer.UpdateRawUnit(string(rule.ID), contents); err != nil { + return fmt.Errorf("UpdateRule: update: %w", err) + } + return nil +} + +// DeleteRule removes the rule unit. +func (b *Backend) DeleteRule(id model.ID) error { + if b.writer == nil { + return fmt.Errorf("DeleteRule: not connected for writing") + } + return b.writer.DeleteUnit(string(id)) +} + +// ruleToGen builds a gen Rule from the model. The field set is the ten +// properties a Studio Pro-authored rule stores, measured against ako/TestApp +// (Mendix 11.13.0). Three omissions are deliberate and load-bearing: +// +// - No AllowedModuleRoles. A rule is not independently callable, so it has no +// module-role security; writing an empty list would invent a property the +// document does not have. +// - No ReturnType. gen declares it beside MicroflowReturnType, but Studio Pro +// does not write it and generated/metamodel 11.6.0 does not list it — a +// pre-7 legacy sibling with nothing to carry through. +// - No concurrency group, Url, StableId or action-info slots: those are the +// nine properties that make a microflow a microflow. +func ruleToGen(rule *microflows.Rule, major int) *genMf.Rule { + out := genMf.NewRule() + out.SetName(rule.Name) + out.SetDocumentation(rule.Documentation) + // Both Studio Pro reference rules store ExportLevel "Hidden", and both + // engines already hardcode it for microflows. Omitting it was the one key + // the first authored rule was missing against the reference document. + out.SetExportLevel("Hidden") + out.SetExcluded(rule.Excluded) + out.SetMarkAsUsed(rule.MarkAsUsed) + out.SetApplyEntityAccess(rule.ApplyEntityAccess) + out.SetReturnVariableName(rule.ReturnVariableName) + if rule.ReturnType != nil { + out.SetMicroflowReturnType(microflowDataTypeToGen(rule.ReturnType)) + } + + oc := genMf.NewMicroflowObjectCollection() + for i, p := range rule.Parameters { + oc.AddObjects(microflowParameterToGen(p, i, major)) + } + if rule.ObjectCollection != nil { + for _, obj := range rule.ObjectCollection.Objects { + if g := microflowObjectToGen(obj); g != nil { + oc.AddObjects(g) + } + } + } + out.SetObjectCollection(oc) + + if rule.ObjectCollection != nil { + for _, f := range rule.ObjectCollection.Flows { + out.AddFlows(sequenceFlowToGen(f, major)) + } + } + return out +} + +// assignRuleIDs assigns fresh IDs to the rule's return type, object collection +// and sequence flows — the nanoflow counterpart. +func assignRuleIDs(r *genMf.Rule) { + if rt := r.MicroflowReturnType(); rt != nil { + assignID(rt) + } + if oc, ok := r.ObjectCollection().(*genMf.MicroflowObjectCollection); ok { + assignObjectCollectionIDs(oc) + } + for _, el := range r.FlowsItems() { + assignID(el) + if sf, ok := el.(*genMf.SequenceFlow); ok { + for _, cv := range sf.CaseValuesItems() { + assignID(cv) + } + assignID(sf.Line()) + } + } +} diff --git a/mdl/backend/modelsdk/rule_write_test.go b/mdl/backend/modelsdk/rule_write_test.go new file mode 100644 index 000000000..4170ba32b --- /dev/null +++ b/mdl/backend/modelsdk/rule_write_test.go @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "sort" + "testing" + + "github.com/mendixlabs/mxcli/modelsdk/codec" + "github.com/mendixlabs/mxcli/sdk/microflows" + "go.mongodb.org/mongo-driver/v2/bson" +) + +// studioProRuleKeys is the exact key set of a Studio Pro-authored rule document, +// measured on Rules.Rule1 and Rules.Rule2 (ako/TestApp, Mendix 11.13.0). An +// mxcli-authored rule must match it — no more (inventing microflow-only model +// the user does not have) and no less. +// +// ExportLevel is the reason this test exists: the first authored rule was +// missing exactly that one key, and mx check passed anyway. +var studioProRuleKeys = []string{ + "$ID", "$Type", + "ApplyEntityAccess", "Documentation", "Excluded", "ExportLevel", "Flows", + "MarkAsUsed", "MicroflowReturnType", "Name", "ObjectCollection", "ReturnVariableName", +} + +func TestAuthoredRuleMatchesStudioProKeySet(t *testing.T) { + rule := µflows.Rule{ + Name: "Rule_NameNotEmpty", + ReturnType: µflows.BooleanType{}, + ReturnVariableName: "Variable", + } + + encoded, err := (&codec.Encoder{}).Encode(ruleToGen(rule, 11)) + if err != nil { + t.Fatalf("encode rule: %v", err) + } + elems, err := bson.Raw(encoded).Elements() + if err != nil { + t.Fatalf("read encoded rule: %v", err) + } + + var got []string + for _, e := range elems { + got = append(got, e.Key()) + } + sort.Strings(got) + want := append([]string(nil), studioProRuleKeys...) + sort.Strings(want) + + if len(got) != len(want) { + t.Fatalf("authored rule has %d keys %v, want %d %v", len(got), got, len(want), want) + } + for i := range got { + if got[i] != want[i] { + t.Errorf("key %d = %q, want %q (full set: %v)", i, got[i], want[i], got) + } + } +} + +// ExportLevel specifically: both reference rules store "Hidden", and both engines +// already hardcode it for microflows. +func TestAuthoredRuleSetsExportLevel(t *testing.T) { + encoded, err := (&codec.Encoder{}).Encode(ruleToGen(µflows.Rule{ + Name: "R", + ReturnType: µflows.BooleanType{}, + }, 11)) + if err != nil { + t.Fatalf("encode rule: %v", err) + } + val, err := bson.Raw(encoded).LookupErr("ExportLevel") + if err != nil { + t.Fatalf("authored rule has no ExportLevel; Studio Pro writes it on every rule") + } + if s, ok := val.StringValueOK(); !ok || s != "Hidden" { + t.Errorf("ExportLevel = %v, want \"Hidden\"", val) + } +} diff --git a/mdl/backend/modelsdk/security_write.go b/mdl/backend/modelsdk/security_write.go index 55143ccbb..d8abd30ad 100644 --- a/mdl/backend/modelsdk/security_write.go +++ b/mdl/backend/modelsdk/security_write.go @@ -184,6 +184,21 @@ func (b *Backend) SetProjectDemoUsersEnabled(unitID model.ID, enabled bool) erro return b.persistUnit(unitID, ps) } +// SetProjectGuestAccess toggles anonymous (guest) access. An empty +// guestUserRole leaves the stored role alone, so turning access off and back on +// does not lose it. +func (b *Backend) SetProjectGuestAccess(unitID model.ID, enabled bool, guestUserRole string) error { + ps, err := b.loadProjectSecurityGen(unitID) + if err != nil { + return err + } + ps.SetEnableGuestAccess(enabled) + if guestUserRole != "" { + ps.SetGuestUserRoleName(guestUserRole) + } + return b.persistUnit(unitID, ps) +} + // AlterUserRoleModuleRoles adds or removes module-role mappings on a project user // role (by name). add=true unions, add=false subtracts. func (b *Backend) AlterUserRoleModuleRoles(unitID model.ID, userRoleName string, add bool, moduleRoles []string) error { diff --git a/mdl/backend/modelsdk/workflow_boundary_read_test.go b/mdl/backend/modelsdk/workflow_boundary_read_test.go new file mode 100644 index 000000000..a49ce2266 --- /dev/null +++ b/mdl/backend/modelsdk/workflow_boundary_read_test.go @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/modelsdk/codec" + genWf "github.com/mendixlabs/mxcli/modelsdk/gen/workflows" + "github.com/mendixlabs/mxcli/sdk/workflows" +) + +// Issue #948. The default engine's workflow reader had no boundary-event support +// at all, while the legacy engine's parser has had it all along. Both engines +// WRITE them, so a boundary event mxcli itself had just written read back as +// absent: DESCRIBE rendered nothing, and a describe -> edit -> re-exec round trip +// silently dropped the timer, its handler flow and the jump inside it. +func TestWorkflowRead_BoundaryEventsRoundTrip(t *testing.T) { + inner := &workflows.CallMicroflowTask{Microflow: "M.ACT_Escalate"} + inner.Name = "ACT_Escalate" + jump := &workflows.JumpToActivity{TargetActivity: "Review"} + jump.Name = "back" + + call := &workflows.CallMicroflowTask{Microflow: "M.ACT_Step"} + call.Name = "Step" + call.BoundaryEvents = []*workflows.BoundaryEvent{{ + EventType: "InterruptingTimer", + TimerDelay: "addHours([%CurrentDateTime%], 2)", + Caption: "escalate", + Flow: &workflows.Flow{Activities: []workflows.WorkflowActivity{inner, jump}}, + }} + + got := roundTripWorkflowActivity(t, call) + + rt, ok := got.(*workflows.CallMicroflowTask) + if !ok { + t.Fatalf("round-tripped to %T, want *workflows.CallMicroflowTask", got) + } + if len(rt.BoundaryEvents) != 1 { + t.Fatalf("boundary events after round trip = %d, want 1 (the reader dropped it)", len(rt.BoundaryEvents)) + } + be := rt.BoundaryEvents[0] + if be.EventType != "InterruptingTimer" { + t.Errorf("EventType = %q, want InterruptingTimer", be.EventType) + } + if be.TimerDelay != "addHours([%CurrentDateTime%], 2)" { + t.Errorf("TimerDelay = %q, want the delay expression", be.TimerDelay) + } + if be.Caption != "escalate" { + t.Errorf("Caption = %q, want escalate", be.Caption) + } + if be.Flow == nil || len(be.Flow.Activities) != 2 { + t.Fatalf("handler flow = %v, want 2 activities", be.Flow) + } + if _, ok := be.Flow.Activities[1].(*workflows.JumpToActivity); !ok { + t.Errorf("handler activity[1] = %T, want *workflows.JumpToActivity", be.Flow.Activities[1]) + } +} + +// A non-interrupting timer is a different $Type and must not be flattened onto +// the interrupting one. +func TestWorkflowRead_NonInterruptingBoundaryEvent(t *testing.T) { + call := &workflows.CallMicroflowTask{Microflow: "M.ACT_Step"} + call.Name = "Step" + call.BoundaryEvents = []*workflows.BoundaryEvent{{ + EventType: "NonInterruptingTimer", + TimerDelay: "[%CurrentDateTime%]", + }} + + rt, ok := roundTripWorkflowActivity(t, call).(*workflows.CallMicroflowTask) + if !ok || len(rt.BoundaryEvents) != 1 { + t.Fatalf("boundary event lost") + } + if got := rt.BoundaryEvents[0].EventType; got != "NonInterruptingTimer" { + t.Errorf("EventType = %q, want NonInterruptingTimer", got) + } +} + +// Control: an activity with no boundary events must not gain an empty one. +func TestWorkflowRead_NoBoundaryEventsStaysEmpty(t *testing.T) { + call := &workflows.CallMicroflowTask{Microflow: "M.ACT_Step"} + call.Name = "Step" + + rt, ok := roundTripWorkflowActivity(t, call).(*workflows.CallMicroflowTask) + if !ok { + t.Fatal("round trip failed") + } + if len(rt.BoundaryEvents) != 0 { + t.Errorf("boundary events = %d, want 0", len(rt.BoundaryEvents)) + } +} + +// roundTripWorkflowActivity encodes a semantic activity to gen through a +// workflow, runs it through the codec, and reads it back — the exact write→read +// path DESCRIBE and any CREATE OR REPLACE takes. Mirrors roundTripMicroflow. +func roundTripWorkflowActivity(t *testing.T, act workflows.WorkflowActivity) workflows.WorkflowActivity { + t.Helper() + wf := &workflows.Workflow{ + Name: "WF", + Parameter: &workflows.WorkflowParameter{EntityRef: "M.Ctx"}, + Flow: &workflows.Flow{Activities: []workflows.WorkflowActivity{act}}, + } + raw, err := (&codec.Encoder{}).Encode(workflowToGen(wf)) + if err != nil { + t.Fatalf("encode: %v", err) + } + el, err := codec.NewDecoder(codec.DefaultRegistry).Decode(raw) + if err != nil { + t.Fatalf("decode: %v", err) + } + dec, ok := el.(*genWf.Workflow) + if !ok { + t.Fatalf("decoded %T, want *genWf.Workflow", el) + } + f, ok := dec.Flow().(*genWf.Flow) + if !ok || f == nil { + t.Fatal("decoded workflow has no flow") + } + back := workflowFlowFromGen(f) + if back == nil || len(back.Activities) == 0 { + t.Fatal("round trip produced no activities") + } + return back.Activities[0] +} + +// A USER TASK carries boundary events too, and it is the shape the workflow +// roundtrip integration tests use. It is worth pinning separately because the +// wiring is per-gen-type: genWf.UserTask (the older type) has no +// BoundaryEventsItems accessor at all, so the reader can only reach these +// through SingleUserTaskActivity / MultiUserTaskActivity. This asserts the +// encoder puts a user task somewhere the reader can actually see it. +func TestWorkflowRead_UserTaskBoundaryEvents(t *testing.T) { + ut := &workflows.UserTask{Page: "M.TaskPage"} + ut.Name = "Review" + ut.Caption = "Review" + ut.Outcomes = []*workflows.UserTaskOutcome{{Value: "Approve"}} + ut.BoundaryEvents = []*workflows.BoundaryEvent{{ + EventType: "InterruptingTimer", + TimerDelay: "${PT1H}", + }} + + rt, ok := roundTripWorkflowActivity(t, ut).(*workflows.UserTask) + if !ok { + t.Fatalf("round-tripped to a different type") + } + if len(rt.BoundaryEvents) != 1 { + t.Fatalf("user task boundary events = %d, want 1", len(rt.BoundaryEvents)) + } + if got := rt.BoundaryEvents[0].TimerDelay; got != "${PT1H}" { + t.Errorf("TimerDelay = %q, want ${PT1H}", got) + } +} diff --git a/mdl/backend/modelsdk/workflow_read.go b/mdl/backend/modelsdk/workflow_read.go index 68ac04a57..c7cdec270 100644 --- a/mdl/backend/modelsdk/workflow_read.go +++ b/mdl/backend/modelsdk/workflow_read.go @@ -100,6 +100,7 @@ func workflowActivityFromGen(el element.Element) workflows.WorkflowActivity { Page: taskPageName(a.TaskPage()), } setWfBase(&t.BaseWorkflowActivity, a.ID(), a.Name(), a.Caption(), a.Annotation(), "Workflows$SingleUserTaskActivity") + t.BoundaryEvents = boundaryEventsFromGen(a.BoundaryEventsItems()) t.Outcomes = userTaskOutcomesFromGen(a.OutcomesItems()) return t case *genWf.MultiUserTaskActivity: @@ -113,17 +114,20 @@ func workflowActivityFromGen(el element.Element) workflows.WorkflowActivity { Page: taskPageName(a.TaskPage()), } setWfBase(&t.BaseWorkflowActivity, a.ID(), a.Name(), a.Caption(), a.Annotation(), "Workflows$MultiUserTaskActivity") + t.BoundaryEvents = boundaryEventsFromGen(a.BoundaryEventsItems()) t.Outcomes = userTaskOutcomesFromGen(a.OutcomesItems()) return t case *genWf.CallMicroflowTask: t := &workflows.CallMicroflowTask{Microflow: a.MicroflowQualifiedName()} setWfBase(&t.BaseWorkflowActivity, a.ID(), a.Name(), a.Caption(), a.Annotation(), "Workflows$CallMicroflowTask") + t.BoundaryEvents = boundaryEventsFromGen(a.BoundaryEventsItems()) t.Outcomes = conditionOutcomesFromGen(a.OutcomesItems()) t.ParameterMappings = microflowParamMappingsFromGen(a.ParameterMappingsItems()) return t case *genWf.CallMicroflowActivity: t := &workflows.CallMicroflowTask{Microflow: a.MicroflowQualifiedName()} setWfBase(&t.BaseWorkflowActivity, a.ID(), a.Name(), a.Caption(), a.Annotation(), "Workflows$CallMicroflowActivity") + t.BoundaryEvents = boundaryEventsFromGen(a.BoundaryEventsItems()) t.Outcomes = conditionOutcomesFromGen(a.OutcomesItems()) t.ParameterMappings = microflowParamMappingsFromGen(a.ParameterMappingsItems()) return t @@ -133,6 +137,7 @@ func workflowActivityFromGen(el element.Element) workflows.WorkflowActivity { ParameterExpression: a.ParameterExpression(), } setWfBase(&t.BaseWorkflowActivity, a.ID(), a.Name(), a.Caption(), a.Annotation(), "Workflows$CallWorkflowActivity") + t.BoundaryEvents = boundaryEventsFromGen(a.BoundaryEventsItems()) return t case *genWf.ExclusiveSplitActivity: t := &workflows.ExclusiveSplitActivity{Expression: a.Expression()} @@ -352,3 +357,62 @@ func setWfBase(a *workflows.BaseWorkflowActivity, id element.ID, name, caption s a.TypeName = typeName a.Annotation = annotationText(annotation) } + +// boundaryEventsFromGen reconstructs an activity's boundary events, including +// each one's handler flow. +// +// Both engines WRITE boundary events; only the legacy parser +// (sdk/mpr/parser_workflow.go, parseBoundaryEvents) read them back. So a boundary +// event mxcli had just written read back as absent on the default engine: +// DESCRIBE rendered nothing, and describe → edit → re-exec silently dropped the +// timer, its handler flow and the jump inside it (issue #948). +// +// The delay lives under FirstExecutionTime for every timer variant — Delay() is +// a separate gen accessor that real documents leave empty — so it is read first +// and Delay() is only a fallback, matching the legacy parser. +func boundaryEventsFromGen(items []element.Element) []*workflows.BoundaryEvent { + var out []*workflows.BoundaryEvent + for _, el := range items { + be := boundaryEventFromGen(el) + if be != nil { + out = append(out, be) + } + } + return out +} + +func boundaryEventFromGen(el element.Element) *workflows.BoundaryEvent { + // The three timer variants differ only in $Type, and gen gives each its own + // concrete type, so the shared shape is read through a small interface rather + // than repeated three times. + type timerBoundary interface { + Caption() string + FirstExecutionTime() string + Delay() string + Flow() element.Element + } + t, ok := el.(timerBoundary) + if !ok { + return nil + } + be := &workflows.BoundaryEvent{Caption: t.Caption()} + be.ID = model.ID(el.ID()) + be.TimerDelay = t.FirstExecutionTime() + if be.TimerDelay == "" { + be.TimerDelay = t.Delay() + } + switch el.(type) { + case *genWf.InterruptingTimerBoundaryEvent: + be.EventType = "InterruptingTimer" + case *genWf.NonInterruptingTimerBoundaryEvent: + be.EventType = "NonInterruptingTimer" + case *genWf.TimerBoundaryEvent: + be.EventType = "Timer" + default: + return nil // an unknown boundary-event kind is not silently mistyped + } + if f, ok := t.Flow().(*genWf.Flow); ok && f != nil { + be.Flow = workflowFlowFromGen(f) + } + return be +} diff --git a/mdl/backend/mpr/backend.go b/mdl/backend/mpr/backend.go index 354557fd7..9548b4d1c 100644 --- a/mdl/backend/mpr/backend.go +++ b/mdl/backend/mpr/backend.go @@ -275,6 +275,12 @@ func (b *MprBackend) IsRule(qualifiedName string) (bool, error) { func (b *MprBackend) ListNanoflows() ([]*microflows.Nanoflow, error) { return b.reader.ListNanoflows() } +func (b *MprBackend) ListRules() ([]*microflows.Rule, error) { + return b.reader.ListRules() +} +func (b *MprBackend) GetRule(id model.ID) (*microflows.Rule, error) { + return b.reader.GetRule(id) +} func (b *MprBackend) ParseMicroflowFromRaw(raw map[string]any, unitID, containerID model.ID) *microflows.Microflow { return mpr.ParseMicroflowFromRaw(raw, unitID, containerID) } @@ -295,6 +301,22 @@ func (b *MprBackend) MoveNanoflow(nf *microflows.Nanoflow) error { return b.writer.MoveNanoflow(nf) } +// Rule authoring is modelsdk-only, like menus: the legacy serializer has no +// serializeRule, and a rule document is close enough to a microflow that a +// half-written one would look valid. Reads are implemented above. +func (b *MprBackend) CreateRule(*microflows.Rule) error { + return errors.New("creating a rule requires the modelsdk engine — rerun without MXCLI_ENGINE=legacy") +} +func (b *MprBackend) UpdateRule(*microflows.Rule) error { + return errors.New("modifying a rule requires the modelsdk engine — rerun without MXCLI_ENGINE=legacy") +} +func (b *MprBackend) DeleteRule(model.ID) error { + return errors.New("dropping a rule requires the modelsdk engine — rerun without MXCLI_ENGINE=legacy") +} +func (b *MprBackend) MoveRule(*microflows.Rule) error { + return errors.New("moving a rule requires the modelsdk engine — rerun without MXCLI_ENGINE=legacy") +} + // --------------------------------------------------------------------------- // PageBackend // --------------------------------------------------------------------------- @@ -382,6 +404,11 @@ func (b *MprBackend) SetProjectSecurityLevel(unitID model.ID, level string) erro func (b *MprBackend) SetProjectDemoUsersEnabled(unitID model.ID, enabled bool) error { return b.writer.SetProjectDemoUsersEnabled(unitID, enabled) } + +// SetProjectGuestAccess toggles anonymous (guest) access. +func (b *MprBackend) SetProjectGuestAccess(unitID model.ID, enabled bool, guestUserRole string) error { + return b.writer.SetProjectGuestAccess(unitID, enabled, guestUserRole) +} func (b *MprBackend) AddUserRole(unitID model.ID, name string, moduleRoles []string, manageAllRoles bool) error { return b.writer.AddUserRole(unitID, name, moduleRoles, manageAllRoles) } diff --git a/mdl/backend/security.go b/mdl/backend/security.go index 1f3832496..b6a25b3f3 100644 --- a/mdl/backend/security.go +++ b/mdl/backend/security.go @@ -26,6 +26,11 @@ type ProjectSecurityBackend interface { GetProjectSecurity() (*security.ProjectSecurity, error) SetProjectSecurityLevel(unitID model.ID, level string) error SetProjectDemoUsersEnabled(unitID model.ID, enabled bool) error + // SetProjectGuestAccess toggles anonymous access. An empty guestUserRole + // leaves the stored role untouched — the caller is responsible for having + // established that a role exists, because Mendix raises CE0133 on guest + // access with no role. + SetProjectGuestAccess(unitID model.ID, enabled bool, guestUserRole string) error AddUserRole(unitID model.ID, name string, moduleRoles []string, manageAllRoles bool) error AlterUserRoleModuleRoles(unitID model.ID, userRoleName string, add bool, moduleRoles []string) error RemoveUserRole(unitID model.ID, name string) error diff --git a/mdl/backend/wfmutator/mutator.go b/mdl/backend/wfmutator/mutator.go index 297c5f406..838dec998 100644 --- a/mdl/backend/wfmutator/mutator.go +++ b/mdl/backend/wfmutator/mutator.go @@ -17,6 +17,7 @@ import ( "go.mongodb.org/mongo-driver/bson" "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" + "github.com/mendixlabs/mxcli/mdl/backend/wfnames" "github.com/mendixlabs/mxcli/mdl/bsonutil" "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/workflows" @@ -217,7 +218,7 @@ func (m *Mutator) InsertAfterActivity(activityRef string, atPos int, activities return err } - newBsonActs := m.serializeAndDedup(activities) + newBsonActs := m.serializeAndDedup(activities, "") insertIdx := idx + 1 newArr := make([]any, 0, len(acts)+len(newBsonActs)) @@ -249,7 +250,13 @@ func (m *Mutator) ReplaceActivity(activityRef string, atPos int, activities []wo return err } - newBsonActs := m.serializeAndDedup(activities) + // Free the outgoing activity's name for the replacement. It is read off the + // resolved element rather than from activityRef, which may be a caption. + var outgoing string + if old, ok := acts[idx].(bson.D); ok { + outgoing = bsonnav.DGetString(old, "Name") + } + newBsonActs := m.serializeAndDedup(activities, outgoing) newArr := make([]any, 0, len(acts)-1+len(newBsonActs)) newArr = append(newArr, acts[:idx]...) @@ -752,36 +759,24 @@ func collectNamesRecursive(flow bson.D, names map[string]bool) { } } -// deduplicateNewActivityName ensures a new activity name doesn't conflict. -func deduplicateNewActivityName(act workflows.WorkflowActivity, existingNames map[string]bool) { - name := act.GetName() - if name == "" { - return - } - if !existingNames[name] { - existingNames[name] = true - return - } - for i := 2; i < 1000; i++ { - candidate := fmt.Sprintf("%s_%d", name, i) - if !existingNames[candidate] { - act.SetName(candidate) - existingNames[candidate] = true - return - } - } -} - // --------------------------------------------------------------------------- // Internal helpers — serialization (via Deps) // --------------------------------------------------------------------------- -// serializeAndDedup serializes workflow activities to BSON, deduplicating names. -func (m *Mutator) serializeAndDedup(activities []workflows.WorkflowActivity) []any { - existingNames := m.collectAllActivityNames() - for _, act := range activities { - deduplicateNewActivityName(act, existingNames) - } +// serializeAndDedup serializes workflow activities to BSON, deduplicating their +// names against the ones already in the workflow. +// +// freeName, when non-empty, names an activity that this same operation is +// removing, so it is not really taken. REPLACE ACTIVITY needs it: the +// replacement reusing the original's name is the ordinary in-place edit, not a +// collision, and treating it as one renamed every same-name replace to Name_2 — +// compounding on each re-run (Name_2_2, Name_2_2_2, ...) so an ALTER that +// changed nothing still mutated the model (issue #944). INSERT passes "": there +// the incoming activity is genuinely additional, and every existing name stands. +func (m *Mutator) serializeAndDedup(activities []workflows.WorkflowActivity, freeName string) []any { + taken := m.collectAllActivityNames() + delete(taken, freeName) // no-op for "" + wfnames.Dedup(activities, taken) result := make([]any, 0, len(activities)) for _, act := range activities { @@ -795,10 +790,9 @@ func (m *Mutator) serializeAndDedup(activities []workflows.WorkflowActivity) []a // buildSubFlowBson builds a Workflows$Flow BSON document from activities. func (m *Mutator) buildSubFlowBson(activities []workflows.WorkflowActivity) bson.D { - existingNames := m.collectAllActivityNames() - for _, act := range activities { - deduplicateNewActivityName(act, existingNames) - } + // An outcome/path/branch/boundary-event sub-flow is always new content, so + // every existing name stands (unlike ReplaceActivity — see serializeAndDedup). + wfnames.Dedup(activities, m.collectAllActivityNames()) var subActsBson bson.A subActsBson = append(subActsBson, bsonArrayMarker) diff --git a/mdl/backend/wfmutator/replace_samename_test.go b/mdl/backend/wfmutator/replace_samename_test.go new file mode 100644 index 000000000..38607bde0 --- /dev/null +++ b/mdl/backend/wfmutator/replace_samename_test.go @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: Apache-2.0 + +package wfmutator + +import ( + "fmt" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" + "github.com/mendixlabs/mxcli/sdk/workflows" +) + +// Issue #944. REPLACE ACTIVITY edits one activity in place, so the replacement +// reusing the original's name is the normal case, not a collision — the original +// is on its way out. Deduplicating against a name pool that still contains it +// renamed every same-name replace to Name_2. +func TestWorkflowMutator_ReplaceActivity_SameNameKeepsName(t *testing.T) { + act := makeWfActivity("Workflows$UserTask", "Original Caption A", "TaskA") + other := makeWfActivity("Workflows$UserTask", "Original Caption B", "TaskB") + m := newMutator(makeWorkflowDoc(act, other)) + + rep := makeTestWorkflowActivity("TaskA", "Changed Caption A") + if err := m.ReplaceActivity("TaskA", 0, []workflows.WorkflowActivity{rep}); err != nil { + t.Fatalf("ReplaceActivity: %v", err) + } + + acts := getActivities(m.rawData) + if got := bsonnav.DGetString(acts[0], "Name"); got != "TaskA" { + t.Errorf("replaced activity renamed to %q; a same-name replace must keep the name", got) + } + if got := bsonnav.DGetString(acts[1], "Name"); got != "TaskB" { + t.Errorf("untouched sibling = %q, want TaskB", got) + } +} + +// The rename compounded: every re-run of the same script suffixed again +// (TaskA -> TaskA_2 -> TaskA_2_2 -> ...), so an ALTER that changed nothing still +// mutated the model on every run — against the idempotence ADR-0008 promises. +func TestWorkflowMutator_ReplaceActivity_SameNameIsIdempotent(t *testing.T) { + act := makeWfActivity("Workflows$UserTask", "Caption", "TaskA") + m := newMutator(makeWorkflowDoc(act)) + + for i := range 4 { + rep := makeTestWorkflowActivity("TaskA", "Caption") + if err := m.ReplaceActivity("TaskA", 0, []workflows.WorkflowActivity{rep}); err != nil { + t.Fatalf("ReplaceActivity run %d: %v", i+1, err) + } + got := bsonnav.DGetString(getActivities(m.rawData)[0], "Name") + if got != "TaskA" { + t.Fatalf("after %d run(s) the name drifted to %q; the suffix accretes", i+1, got) + } + } +} + +// Control: the exclusion must free ONLY the outgoing activity's name. A +// replacement colliding with a different, surviving activity still dedupes. +func TestWorkflowMutator_ReplaceActivity_StillDedupesAgainstOthers(t *testing.T) { + act := makeWfActivity("Workflows$UserTask", "Caption A", "TaskA") + other := makeWfActivity("Workflows$UserTask", "Caption B", "TaskB") + m := newMutator(makeWorkflowDoc(act, other)) + + // Replacing TaskA with something named TaskB collides with a survivor. + rep := makeTestWorkflowActivity("TaskB", "Caption B") + if err := m.ReplaceActivity("TaskA", 0, []workflows.WorkflowActivity{rep}); err != nil { + t.Fatalf("ReplaceActivity: %v", err) + } + acts := getActivities(m.rawData) + if got := bsonnav.DGetString(acts[0], "Name"); got != "TaskB_2" { + t.Errorf("replacement = %q, want TaskB_2 (TaskB survives, so it is still taken)", got) + } +} + +// The reference may be a caption, so the name to free has to come from the +// resolved activity rather than from the reference string. +func TestWorkflowMutator_ReplaceActivity_ResolvedByCaption(t *testing.T) { + act := makeWfActivity("Workflows$UserTask", "Review the order", "ReviewOrder") + m := newMutator(makeWorkflowDoc(act)) + + rep := makeTestWorkflowActivity("ReviewOrder", "Review the order (v2)") + if err := m.ReplaceActivity("Review the order", 0, []workflows.WorkflowActivity{rep}); err != nil { + t.Fatalf("ReplaceActivity: %v", err) + } + if got := bsonnav.DGetString(getActivities(m.rawData)[0], "Name"); got != "ReviewOrder" { + t.Errorf("resolved-by-caption replace renamed to %q, want ReviewOrder", got) + } +} + +// Multi-activity replace: the outgoing name is freed once, so the first +// replacement may take it and the rest dedupe among themselves. +func TestWorkflowMutator_ReplaceActivity_MultipleReplacements(t *testing.T) { + act := makeWfActivity("Workflows$UserTask", "Caption", "TaskA") + m := newMutator(makeWorkflowDoc(act)) + + a := makeTestWorkflowActivity("TaskA", "A") + b := makeTestWorkflowActivity("TaskA", "B") + if err := m.ReplaceActivity("TaskA", 0, []workflows.WorkflowActivity{a, b}); err != nil { + t.Fatalf("ReplaceActivity: %v", err) + } + acts := getActivities(m.rawData) + names := []string{bsonnav.DGetString(acts[0], "Name"), bsonnav.DGetString(acts[1], "Name")} + if names[0] != "TaskA" || names[1] != "TaskA_2" { + t.Errorf("names = %v, want [TaskA TaskA_2]", names) + } + if names[0] == names[1] { + t.Errorf("both replacements got %q — a duplicate name is CE0495", fmt.Sprint(names[0])) + } +} diff --git a/mdl/backend/wfnames/wfnames.go b/mdl/backend/wfnames/wfnames.go new file mode 100644 index 000000000..f363bce3e --- /dev/null +++ b/mdl/backend/wfnames/wfnames.go @@ -0,0 +1,132 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package wfnames holds the workflow activity-name uniqueness policy that every +// backend shares. +// +// Mendix requires activity names to be unique across a whole workflow — nested +// sub-flows included — and reports CE0495 "Duplicate name" when they are not. +// Activity names are frequently *derived* rather than authored (a CALL MICROFLOW +// activity is named after its target microflow), so two calls to the same +// microflow collide before anything has a chance to notice. +// +// The policy lives here rather than in a backend because the backends disagree +// on everything except the policy: the file backends walk BSON to learn which +// names are taken, while the MCP backend learns them from PED reads. Only the +// rename rule is common — and it is the part that must not drift, because a +// backend that skips it writes a model Studio Pro refuses to open cleanly while +// mxcli's own read-back looks fine (issue #945). +package wfnames + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/sdk/workflows" +) + +// maxSuffix bounds the candidate search. It is a runaway guard, not a real +// limit: a workflow with 1000 same-named activities is a bug upstream. +const maxSuffix = 1000 + +// Unique returns name if it is free in taken, otherwise the first free +// "_" (n counting from 2), and marks the result as taken. An empty +// name is returned unchanged and never recorded — Mendix generates one for +// activity types that carry no author-visible name. +// +// The taken-set is deliberately not a seen-*count*: counting collides again the +// moment a workflow already contains the suffixed name (names A, A, A_2 count +// their way to A, A_2, A_2). +func Unique(name string, taken map[string]bool) string { + if name == "" { + return name + } + if !taken[name] { + taken[name] = true + return name + } + for i := 2; i < maxSuffix; i++ { + candidate := fmt.Sprintf("%s_%d", name, i) + if !taken[candidate] { + taken[candidate] = true + return candidate + } + } + return name +} + +// Dedup renames the given activities — and every activity in the sub-flows they +// carry — so that no name collides with taken or with each other, updating taken +// as it goes. +// +// Seed taken with the names already in the target workflow (the file backends +// collect them from the stored BSON, the MCP backend from PED); an empty map +// deduplicates a fresh set against itself only. +func Dedup(activities []workflows.WorkflowActivity, taken map[string]bool) { + for _, act := range activities { + if act == nil { + continue + } + act.SetName(Unique(act.GetName(), taken)) + for _, flow := range SubFlows(act) { + Dedup(flow.Activities, taken) + } + } +} + +// SubFlows returns every sub-flow an activity carries: its outcome flows first, +// then its boundary-event flows. The order matches the depth-first, in-order +// traversal DESCRIBE and the mutators' @N positional addressing use, so a +// rename here lands on the activity the author would point at. +// +// An activity type absent from the switch simply has no sub-flows; a new type +// that gains one has to be added here, or its nested activities silently skip +// deduplication. +func SubFlows(act workflows.WorkflowActivity) []*workflows.Flow { + var out []*workflows.Flow + add := func(f *workflows.Flow) { + if f != nil { + out = append(out, f) + } + } + addConditionOutcomes := func(outcomes []workflows.ConditionOutcome) { + for _, o := range outcomes { + if o != nil { + add(o.GetFlow()) + } + } + } + addBoundaryEvents := func(events []*workflows.BoundaryEvent) { + for _, e := range events { + if e != nil { + add(e.Flow) + } + } + } + + switch a := act.(type) { + case *workflows.UserTask: + for _, o := range a.Outcomes { + if o != nil { + add(o.Flow) + } + } + addBoundaryEvents(a.BoundaryEvents) + case *workflows.SystemTask: + addConditionOutcomes(a.Outcomes) + case *workflows.CallMicroflowTask: + addConditionOutcomes(a.Outcomes) + addBoundaryEvents(a.BoundaryEvents) + case *workflows.CallWorkflowActivity: + addBoundaryEvents(a.BoundaryEvents) + case *workflows.ExclusiveSplitActivity: + addConditionOutcomes(a.Outcomes) + case *workflows.ParallelSplitActivity: + for _, o := range a.Outcomes { + if o != nil { + add(o.Flow) + } + } + case *workflows.WaitForNotificationActivity: + addBoundaryEvents(a.BoundaryEvents) + } + return out +} diff --git a/mdl/backend/wfnames/wfnames_test.go b/mdl/backend/wfnames/wfnames_test.go new file mode 100644 index 000000000..312093e43 --- /dev/null +++ b/mdl/backend/wfnames/wfnames_test.go @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: Apache-2.0 + +package wfnames + +import ( + "testing" + + "github.com/mendixlabs/mxcli/sdk/workflows" +) + +func TestUnique(t *testing.T) { + taken := map[string]bool{} + cases := []struct{ in, want string }{ + {"Approve", "Approve"}, + {"Approve", "Approve_2"}, + {"Approve", "Approve_3"}, + {"Reject", "Reject"}, + {"", ""}, // an empty name is left alone and never recorded + {"", ""}, + } + for _, c := range cases { + if got := Unique(c.in, taken); got != c.want { + t.Errorf("Unique(%q) = %q, want %q", c.in, got, c.want) + } + } + if taken[""] { + t.Error("the empty name must not be recorded as taken") + } +} + +// A seen-count would hand out a name the workflow already contains: A, A, A_2 +// counts its way to A, A_2, A_2. The taken-set must skip to A_3. +func TestUnique_SkipsNameAlreadyPresent(t *testing.T) { + taken := map[string]bool{"Approve_2": true} + if got := Unique("Approve", taken); got != "Approve" { + t.Fatalf("first Approve = %q", got) + } + if got := Unique("Approve", taken); got != "Approve_3" { + t.Errorf("second Approve = %q, want Approve_3 (Approve_2 was already taken)", got) + } +} + +func TestDedup_SeededWithExistingNames(t *testing.T) { + a := &workflows.CallMicroflowTask{Microflow: "M.ACT_Do"} + a.Name = "ACT_Do" + b := &workflows.CallMicroflowTask{Microflow: "M.ACT_Do"} + b.Name = "ACT_Do" + + Dedup([]workflows.WorkflowActivity{a, b}, map[string]bool{"ACT_Do": true}) + + if a.Name != "ACT_Do_2" || b.Name != "ACT_Do_3" { + t.Errorf("names = %q, %q; want ACT_Do_2, ACT_Do_3", a.Name, b.Name) + } +} + +// Uniqueness is workflow-wide, so an inserted activity's own sub-flows have to +// be walked too. +func TestDedup_RecursesIntoSubFlows(t *testing.T) { + nested := &workflows.CallMicroflowTask{Microflow: "M.ACT_Do"} + nested.Name = "ACT_Do" + deeper := &workflows.WaitForTimerActivity{} + deeper.Name = "ACT_Do" + nestedBoundary := &workflows.BoundaryEvent{ + Flow: &workflows.Flow{Activities: []workflows.WorkflowActivity{deeper}}, + } + task := &workflows.UserTask{ + Outcomes: []*workflows.UserTaskOutcome{ + {Value: "Approve", Flow: &workflows.Flow{Activities: []workflows.WorkflowActivity{nested}}}, + }, + BoundaryEvents: []*workflows.BoundaryEvent{nestedBoundary}, + } + task.Name = "ACT_Do" + + Dedup([]workflows.WorkflowActivity{task}, map[string]bool{}) + + if task.Name != "ACT_Do" || nested.Name != "ACT_Do_2" || deeper.Name != "ACT_Do_3" { + t.Errorf("names = %q, %q, %q; want ACT_Do, ACT_Do_2, ACT_Do_3", task.Name, nested.Name, deeper.Name) + } +} + +func TestSubFlows_OutcomesBeforeBoundaryEvents(t *testing.T) { + outFlow := &workflows.Flow{} + beFlow := &workflows.Flow{} + call := &workflows.CallMicroflowTask{ + Outcomes: []workflows.ConditionOutcome{&workflows.VoidConditionOutcome{Flow: outFlow}}, + BoundaryEvents: []*workflows.BoundaryEvent{{Flow: beFlow}}, + } + got := SubFlows(call) + if len(got) != 2 || got[0] != outFlow || got[1] != beFlow { + t.Errorf("SubFlows order = %v; want [outcome, boundaryEvent]", got) + } +} + +// A nil flow must not become a nil entry the walker then dereferences. +func TestSubFlows_SkipsAbsentFlows(t *testing.T) { + call := &workflows.CallMicroflowTask{ + Outcomes: []workflows.ConditionOutcome{&workflows.VoidConditionOutcome{}}, + BoundaryEvents: []*workflows.BoundaryEvent{{}}, + } + if got := SubFlows(call); len(got) != 0 { + t.Errorf("SubFlows = %v, want none", got) + } +} diff --git a/mdl/catalog/builder.go b/mdl/catalog/builder.go index a3340a659..eb14751e2 100644 --- a/mdl/catalog/builder.go +++ b/mdl/catalog/builder.go @@ -46,6 +46,7 @@ type CatalogReader interface { // Microflows & nanoflows ListMicroflows() ([]*microflows.Microflow, error) ListNanoflows() ([]*microflows.Nanoflow, error) + ListRules() ([]*microflows.Rule, error) // Pages, layouts & snippets ListPages() ([]*pages.Page, error) @@ -109,6 +110,7 @@ type Builder struct { // By caching results, we parse each document type exactly once. microflowCache []*microflows.Microflow nanoflowCache []*microflows.Nanoflow + ruleCache []*microflows.Rule pageCache []*pages.Page domainModelCache []*domainmodel.DomainModel enumerationCache []*model.Enumeration @@ -263,6 +265,17 @@ func (b *Builder) cachedNanoflows() ([]*microflows.Nanoflow, error) { return b.nanoflowCache, nil } +func (b *Builder) cachedRules() ([]*microflows.Rule, error) { + if b.ruleCache == nil { + var err error + b.ruleCache, err = b.reader.ListRules() + if err != nil { + return nil, err + } + } + return b.ruleCache, nil +} + func (b *Builder) cachedPages() ([]*pages.Page, error) { if b.pageCache == nil { var err error diff --git a/mdl/catalog/builder_microflows.go b/mdl/catalog/builder_microflows.go index d46395bdf..d9cf1596d 100644 --- a/mdl/catalog/builder_microflows.go +++ b/mdl/catalog/builder_microflows.go @@ -23,6 +23,15 @@ func (b *Builder) buildMicroflows() error { return err } + // Get all rules (cached). A rule is a distinct doctype and lands in + // microflows_data with MicroflowType "RULE", the way a nanoflow does — + // without a row here a rule is not an object at all, so `show callers` can + // never resolve it and GRAPH_DEAD_ASSETS cannot see it. + rules, err := b.cachedRules() + if err != nil { + return err + } + mfStmt, err := b.tx.Prepare(` INSERT INTO microflows_data (Id, Name, QualifiedName, ModuleName, Folder, MicroflowType, Description, ReturnType, ParameterCount, ActivityCount, Complexity, Excluded, @@ -63,6 +72,7 @@ func (b *Builder) buildMicroflows() error { mfCount := 0 nfCount := 0 + ruleCount := 0 paramCount := 0 // insertParams writes one row per parameter. Microflows and nanoflows share @@ -281,8 +291,69 @@ func (b *Builder) buildMicroflows() error { } } + // Process rules. Their bodies are walked for activities in full mode like + // any other flow; the reference edges out of those bodies are emitted by + // builder_references.go, and the two must land together — a rule that is an + // object but whose body is not walked reports every document it calls as + // dead, which is worse than not knowing about rules at all. + for _, rule := range rules { + moduleID := b.hierarchy.findModuleID(rule.ContainerID) + moduleName := b.hierarchy.getModuleName(moduleID) + qualifiedName := moduleName + "." + rule.Name + + returnType := "" + if rule.ReturnType != nil { + returnType = getDataTypeName(rule.ReturnType) + } + + _, err = mfStmt.Exec( + string(rule.ID), + rule.Name, + qualifiedName, + moduleName, + b.hierarchy.buildFolderPath(rule.ContainerID), + "RULE", + rule.Documentation, + returnType, + len(rule.Parameters), + countRuleActivities(rule), + calculateRuleComplexity(rule), + rule.Excluded, + projectID, snapshotID, + ) + if err != nil { + return err + } + if err := insertParams(string(rule.ID), qualifiedName, moduleName, rule.Parameters); err != nil { + return err + } + ruleCount++ + + if b.fullMode && rule.ObjectCollection != nil { + for seq, obj := range rule.ObjectCollection.Objects { + activityType := getMicroflowObjectType(obj) + activityName := activityType + actionType := "" + if act, ok := obj.(*microflows.ActionActivity); ok && act.Action != nil { + actionType = getMicroflowActionType(act.Action) + activityName = actionType + } + if _, err := actStmt.Exec( + string(obj.GetID()), activityName, "Activity", activityType, seq+1, + string(rule.ID), qualifiedName, moduleName, moduleName, + "", actionType, "", "", "", + projectID, snapshotID, + ); err != nil { + return err + } + actCount++ + } + } + } + b.report("Microflows", mfCount) b.report("Nanoflows", nfCount) + b.report("Rules", ruleCount) b.report("Flow parameters", paramCount) if b.fullMode { b.report("Activities", actCount) @@ -481,3 +552,37 @@ func calculateNanoflowComplexity(nf *microflows.Nanoflow) int { complexity += countDecisionPoints(nf.ObjectCollection.Objects) return complexity } + +// countRuleActivities counts meaningful activities in a rule, excluding +// structural elements — the nanoflow counterpart, kept beside it because the two +// differ only in the type they take. +func countRuleActivities(rule *microflows.Rule) int { + if rule.ObjectCollection == nil { + return 0 + } + count := 0 + for _, obj := range rule.ObjectCollection.Objects { + switch obj.(type) { + case *microflows.StartEvent, *microflows.EndEvent, *microflows.ExclusiveMerge: + // Structural, not activities. + default: + count++ + } + } + return count +} + +// calculateRuleComplexity calculates McCabe cyclomatic complexity for a rule. +func calculateRuleComplexity(rule *microflows.Rule) int { + if rule.ObjectCollection == nil { + return 1 + } + complexity := 1 + for _, obj := range rule.ObjectCollection.Objects { + switch obj.(type) { + case *microflows.ExclusiveSplit, *microflows.InheritanceSplit, *microflows.LoopedActivity: + complexity++ + } + } + return complexity +} diff --git a/mdl/catalog/builder_references.go b/mdl/catalog/builder_references.go index 4130fd31d..d516e88c2 100644 --- a/mdl/catalog/builder_references.go +++ b/mdl/catalog/builder_references.go @@ -56,6 +56,32 @@ func collectActionActivities(oc *microflows.MicroflowObjectCollection) []*microf return result } +// collectRuleCalls returns the qualified name of every rule a flow calls from a +// decision, recursing into LoopedActivity bodies like collectActionActivities. +// +// A rule is not an activity: Mendix can only call one from an ExclusiveSplit's +// condition, so the action walk above never sees it and a rule called from a +// decision looked uncalled — `show callers` reported none, which is how #939's +// reporter read the corruption as "the reference never resolves" even after the +// condition was stored correctly. +func collectRuleCalls(oc *microflows.MicroflowObjectCollection) []string { + if oc == nil { + return nil + } + var out []string + for _, obj := range oc.Objects { + switch o := obj.(type) { + case *microflows.ExclusiveSplit: + if rc, ok := o.SplitCondition.(*microflows.RuleSplitCondition); ok && rc.RuleQualifiedName != "" { + out = append(out, rc.RuleQualifiedName) + } + case *microflows.LoopedActivity: + out = append(out, collectRuleCalls(o.ObjectCollection)...) + } + } + return out +} + // microflowActionRef returns the cross-reference a microflow action makes to // another document: the target's catalog object type, its qualified name, and // the RefKind label. ok is false when the action references no resolvable @@ -229,6 +255,9 @@ func (b *Builder) buildReferences() error { // Intra-flow variable→entity map so change/delete (which operate on a // variable, not a named entity) can resolve their target. varEntity := buildVarEntityMap(params, acts) + for _, rule := range collectRuleCalls(oc) { + emit("RULE", rule, RefKindCall) + } for _, act := range acts { if tt, tn, rk, ok := microflowActionRef(act.Action); ok { emit(tt, tn, rk) @@ -257,6 +286,18 @@ func (b *Builder) buildReferences() error { } } + // Extract rule references. A rule's body calls microflows, retrieves and + // evaluates like any other flow, and without this walk every document a rule + // calls is invisible to the reference graph — reported dead by + // `show callers`, GRAPH_DEAD_ASSETS and lint rule QUAL004. Same shape as the + // scheduled-event gap, one layer deeper. + rules, err := b.cachedRules() + if err == nil { + for _, rule := range rules { + emitActionRefs("RULE", string(rule.ID), rule.ContainerID, rule.Name, rule.Parameters, rule.ReturnType, rule.ObjectCollection) + } + } + // Extract entity references (generalization) — using cached list dms, err := b.cachedDomainModels() if err == nil { diff --git a/mdl/catalog/builder_rule_refs_test.go b/mdl/catalog/builder_rule_refs_test.go new file mode 100644 index 000000000..87a1151b4 --- /dev/null +++ b/mdl/catalog/builder_rule_refs_test.go @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: Apache-2.0 + +package catalog + +import ( + "testing" + + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// A rule is not an activity: Mendix can only call one from a decision's +// condition, so the action walk that feeds CATALOG.REFS never saw it and a rule +// called from a decision had no callers at all. #939's reporter read that as +// "the reference never resolves" — a second symptom with its own cause, which +// survived fixing the write path. +func TestCollectRuleCalls(t *testing.T) { + split := µflows.ExclusiveSplit{ + SplitCondition: µflows.RuleSplitCondition{RuleQualifiedName: "Sample.Rule_IsActive"}, + } + nested := µflows.ExclusiveSplit{ + SplitCondition: µflows.RuleSplitCondition{RuleQualifiedName: "Sample.Rule_InLoop"}, + } + oc := µflows.MicroflowObjectCollection{ + Objects: []microflows.MicroflowObject{ + split, + µflows.LoopedActivity{ + ObjectCollection: µflows.MicroflowObjectCollection{ + Objects: []microflows.MicroflowObject{nested}, + }, + }, + // An expression split contributes nothing — it names no document. + µflows.ExclusiveSplit{ + SplitCondition: µflows.ExpressionSplitCondition{Expression: "$x = 1"}, + }, + }, + } + + got := collectRuleCalls(oc) + want := []string{"Sample.Rule_IsActive", "Sample.Rule_InLoop"} + if len(got) != len(want) { + t.Fatalf("got %v, want %v", got, want) + } + for i := range want { + if got[i] != want[i] { + t.Errorf("[%d] = %q, want %q — loop bodies are walked like collectActionActivities does", i, got[i], want[i]) + } + } + + if refs := collectRuleCalls(nil); refs != nil { + t.Errorf("nil collection = %v, want nil", refs) + } +} diff --git a/mdl/catalog/builder_source.go b/mdl/catalog/builder_source.go index 480418e8c..0a37ed52d 100644 --- a/mdl/catalog/builder_source.go +++ b/mdl/catalog/builder_source.go @@ -70,6 +70,17 @@ func (b *Builder) buildSource() error { } } + // Rules. Without this a rule's body is not in CATALOG.SOURCE, so + // `search ''` cannot find an expression that only a rule contains. + ruleList, err := b.cachedRules() + if err == nil { + for _, rule := range ruleList { + moduleID := b.hierarchy.findModuleID(rule.ContainerID) + moduleName := b.hierarchy.getModuleName(moduleID) + items = append(items, sourceItem{"RULE", moduleName + "." + rule.Name, moduleName}) + } + } + // Pages pageList, err := b.cachedPages() if err == nil { diff --git a/mdl/catalog/catalog.go b/mdl/catalog/catalog.go index 4fa61763b..009089c61 100644 --- a/mdl/catalog/catalog.go +++ b/mdl/catalog/catalog.go @@ -106,6 +106,7 @@ func (c *Catalog) Tables() []string { "CATALOG.ATTRIBUTES", "CATALOG.MICROFLOWS", "CATALOG.NANOFLOWS", + "CATALOG.RULES", "CATALOG.PAGES", "CATALOG.PAGE_TEMPLATES", "CATALOG.SNIPPETS", diff --git a/mdl/catalog/tables.go b/mdl/catalog/tables.go index 7fd917b83..d4ef34b35 100644 --- a/mdl/catalog/tables.go +++ b/mdl/catalog/tables.go @@ -183,6 +183,13 @@ func (c *Catalog) createTables() error { `CREATE VIEW IF NOT EXISTS nanoflows AS SELECT * FROM microflows WHERE MicroflowType = 'NANOFLOW'`, + // rules view (filtered subset of microflows view). A rule shares the + // storage because it shares the shape — signature, body, complexity — + // but it is a distinct doctype, so `microflows` is itself filtered to + // MicroflowType = 'MICROFLOW' wherever it is presented as microflows. + `CREATE VIEW IF NOT EXISTS rules AS + SELECT * FROM microflows WHERE MicroflowType = 'RULE'`, + // microflow_parameters: one row per parameter, for microflows and // nanoflows alike. microflows_data carries only ParameterCount, so a // caller could see that a flow takes three arguments but not what they @@ -1039,6 +1046,10 @@ func (c *Catalog) createTables() error { ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM microflows WHERE MicroflowType = 'NANOFLOW' UNION ALL + SELECT Id, 'RULE' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, + ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource + FROM microflows WHERE MicroflowType = 'RULE' + UNION ALL SELECT Id, 'PAGE' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM pages diff --git a/mdl/executor/cmd_alter_workflow.go b/mdl/executor/cmd_alter_workflow.go index 875856e2c..c1881b892 100644 --- a/mdl/executor/cmd_alter_workflow.go +++ b/mdl/executor/cmd_alter_workflow.go @@ -4,6 +4,7 @@ package executor import ( "fmt" + "strings" "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" @@ -27,6 +28,14 @@ func execAlterWorkflow(ctx *ExecContext, s *ast.AlterWorkflowStmt) error { return err } + // Same exec-side guard as CREATE WORKFLOW: ALTER had no reference validation + // at all, so an inserted activity could name a microflow that exists nowhere + // and still be written (issue #943). + if refErrors := validateAlterWorkflowRefs(ctx, s, nil); len(refErrors) > 0 { + return mdlerrors.NewValidationf("workflow '%s' has reference errors:\n - %s", + s.Name.String(), strings.Join(refErrors, "\n - ")) + } + h, err := getHierarchy(ctx) if err != nil { return mdlerrors.NewBackend("build hierarchy", err) diff --git a/mdl/executor/cmd_associations.go b/mdl/executor/cmd_associations.go index 20a83f45d..3402e9a88 100644 --- a/mdl/executor/cmd_associations.go +++ b/mdl/executor/cmd_associations.go @@ -64,18 +64,7 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error owner = domainmodel.AssociationOwnerBoth } - // Convert delete behavior - var deleteBehavior domainmodel.DeleteBehaviorType - switch s.DeleteBehavior { - case ast.DeleteKeepReferences: - deleteBehavior = domainmodel.DeleteBehaviorTypeDeleteMeButKeepReferences - case ast.DeleteCascade: - deleteBehavior = domainmodel.DeleteBehaviorTypeDeleteMeAndReferences - case ast.DeleteIfNoReferences: - deleteBehavior = domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences - default: - deleteBehavior = domainmodel.DeleteBehaviorTypeDeleteMeButKeepReferences - } + deleteBehavior := storageDeleteBehavior(s.DeleteBehavior) // Convert storage type (default: Column = foreign key in parent table) storageFormat := domainmodel.StorageFormatColumn @@ -246,7 +235,7 @@ func execAlterAssociation(ctx *ExecContext, s *ast.AlterAssociationStmt) error { switch s.Operation { case ast.AlterAssociationSetDeleteBehavior: assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ - Type: domainmodel.DeleteBehaviorType(s.DeleteBehavior.String()), + Type: storageDeleteBehavior(s.DeleteBehavior), } case ast.AlterAssociationSetOwner: assoc.Owner = domainmodel.AssociationOwner(s.Owner.String()) @@ -271,7 +260,7 @@ func execAlterAssociation(ctx *ExecContext, s *ast.AlterAssociationStmt) error { switch s.Operation { case ast.AlterAssociationSetDeleteBehavior: ca.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ - Type: domainmodel.DeleteBehaviorType(s.DeleteBehavior.String()), + Type: storageDeleteBehavior(s.DeleteBehavior), } case ast.AlterAssociationSetOwner: ca.Owner = domainmodel.AssociationOwner(s.Owner.String()) @@ -527,11 +516,17 @@ func describeAssociation(ctx *ExecContext, name ast.QualifiedName) error { fmt.Fprintf(ctx.Output, "storage column\n") } + // DELETE_AND_REFERENCES, not DELETE_CASCADE: DESCRIBE has to emit MDL the + // parser accepts, and DELETE_CASCADE is not a token — only CASCADE and the + // three canonical names are. The other two arms already spell the + // canonical name, so cascade was the odd one out and a describe → edit → + // exec loop died on it (upstream #901). The round-trip test in + // cmd_associations_delete_behavior_test.go feeds this back through the parser. deleteBehavior := "DELETE_BUT_KEEP_REFERENCES" if childDeleteBehavior != nil { switch childDeleteBehavior.Type { case domainmodel.DeleteBehaviorTypeDeleteMeAndReferences: - deleteBehavior = "DELETE_CASCADE" + deleteBehavior = "DELETE_AND_REFERENCES" case domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences: deleteBehavior = "DELETE_IF_NO_REFERENCES" case domainmodel.DeleteBehaviorTypeDeleteMeButKeepReferences: @@ -580,6 +575,30 @@ func describeAssociation(ctx *ExecContext, name ast.QualifiedName) error { return mdlerrors.NewNotFound("association", name.String()) } +// storageDeleteBehavior maps an authored delete behaviour onto the value Mendix +// stores. Mendix's DeletingBehavior admits exactly three (generated/metamodel +// enums.go); ast.DeleteBehavior has six, three of which name nothing Mendix has. +// +// This is the single conversion for every path that writes one. The ALTER path +// used to build it as DeleteBehaviorType(s.DeleteBehavior.String()) instead, and +// String() spells the prevent case "DeleteIfNoReferences" where Mendix writes +// "DeleteMeIfNoReferences" — so `ALTER ASSOCIATION ... SET DELETE_BEHAVIOR +// PREVENT` put an out-of-domain enum on disk (upstream #901). Nothing downstream +// rejects one: mxbuild is lenient about property values and Studio Pro is not, +// so the failure surfaces only when someone opens the project. +// +// String() is a display helper. Do not reintroduce it as a storage encoding. +func storageDeleteBehavior(b ast.DeleteBehavior) domainmodel.DeleteBehaviorType { + switch b { + case ast.DeleteCascade: + return domainmodel.DeleteBehaviorTypeDeleteMeAndReferences + case ast.DeleteIfNoReferences: + return domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences + default: + return domainmodel.DeleteBehaviorTypeDeleteMeButKeepReferences + } +} + // applyAnchors copies authored `@anchor(from: …, to: …)` / `SET ANCHOR` values // onto the association. // diff --git a/mdl/executor/cmd_associations_delete_behavior_test.go b/mdl/executor/cmd_associations_delete_behavior_test.go new file mode 100644 index 000000000..7d4380f02 --- /dev/null +++ b/mdl/executor/cmd_associations_delete_behavior_test.go @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// upstream #901, the half of it the visitor fix does not reach. +// +// Mendix's DeletingBehavior admits exactly three values (generated/metamodel +// enums.go). The CREATE path converts through an explicit switch and lands on +// them; ALTER ASSOCIATION SET built the stored value as +// DeleteBehaviorType(s.DeleteBehavior.String()), and ast.DeleteBehavior.String() +// returns "DeleteIfNoReferences" where Mendix writes "DeleteMeIfNoReferences". +// +// That mattered only once the visitor could produce DeleteIfNoReferences at all, +// which is why it hid behind the reported bug: measured by patching +// buildDeleteBehavior alone and running `ALTER ASSOCIATION ... SET DELETE_BEHAVIOR +// PREVENT` against a real 11.6.6 project, the string "DeleteIfNoReferences" +// reached the .mxunit on disk. An out-of-domain enum is worse than the wrong +// behaviour it replaces — mxbuild tolerates unknown property VALUES about as far +// as Studio Pro does, which is not far (CLAUDE.md, MprProperty.cs). +// +// So the assertion is on the STORED value, not on the AST and not on the command +// output. "Altered association: ..." was printed in every broken case. +func TestAssociationDeleteBehaviorReachesStorageAsAMendixValue(t *testing.T) { + cases := []struct { + name string + in ast.DeleteBehavior + want domainmodel.DeleteBehaviorType + }{ + {"keep", ast.DeleteKeepReferences, domainmodel.DeleteBehaviorTypeDeleteMeButKeepReferences}, + {"cascade", ast.DeleteCascade, domainmodel.DeleteBehaviorTypeDeleteMeAndReferences}, + {"prevent", ast.DeleteIfNoReferences, domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences}, + } + + for _, tc := range cases { + t.Run("create-or-modify/"+tc.name, func(t *testing.T) { + ctx, assoc := assocFixture(t) + assertNoError(t, execCreateAssociation(ctx, &ast.CreateAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Child_Parent"}, + Parent: ast.QualifiedName{Module: "M", Name: "Child"}, + Child: ast.QualifiedName{Module: "M", Name: "Parent"}, + Type: ast.AssocReference, + DeleteBehavior: tc.in, + CreateOrModify: true, + })) + assertStoredBehavior(t, assoc, tc.want) + }) + + t.Run("alter-set/"+tc.name, func(t *testing.T) { + ctx, assoc := assocFixture(t) + assertNoError(t, execAlterAssociation(ctx, &ast.AlterAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Child_Parent"}, + Operation: ast.AlterAssociationSetDeleteBehavior, + DeleteBehavior: tc.in, + })) + assertStoredBehavior(t, assoc, tc.want) + }) + } +} + +// DESCRIBE emitted `delete_behavior DELETE_CASCADE;` for a cascading association, +// and DELETE_CASCADE is not a token — the parser rejects it with "missing +// {DELETE_AND_REFERENCES, DELETE_BUT_KEEP_REFERENCES, DELETE_IF_NO_REFERENCES, +// CASCADE, PREVENT}". It is the very line the #901 reporter pasted as their +// starting state, so a user following the obvious describe → edit → exec loop hit +// a parse error on cascade and silent data loss on everything else. +// +// Feeding DESCRIBE's own output back through the parser is the only check that +// proves the two sides agree; asserting on a string literal would pass against a +// formatter emitting something nothing can read. +func TestDescribeAssociationDeleteBehaviorRoundTripsThroughTheParser(t *testing.T) { + for _, want := range []ast.DeleteBehavior{ + ast.DeleteKeepReferences, ast.DeleteCascade, ast.DeleteIfNoReferences, + } { + t.Run(want.String(), func(t *testing.T) { + ctx, assoc := assocFixture(t) + assertNoError(t, execCreateAssociation(ctx, &ast.CreateAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Child_Parent"}, + Parent: ast.QualifiedName{Module: "M", Name: "Child"}, + Child: ast.QualifiedName{Module: "M", Name: "Parent"}, + Type: ast.AssocReference, + DeleteBehavior: want, + CreateOrModify: true, + })) + _ = assoc + + var buf bytes.Buffer + ctx.Output = &buf + assertNoError(t, describeAssociation(ctx, ast.QualifiedName{Module: "M", Name: "Child_Parent"})) + + prog, errs := visitor.Build(buf.String()) + if len(errs) > 0 { + t.Fatalf("DESCRIBE emitted MDL the parser rejects: %v\n--- output ---\n%s", errs, buf.String()) + } + stmt, ok := prog.Statements[0].(*ast.CreateAssociationStmt) + if !ok { + t.Fatalf("got %T, want *ast.CreateAssociationStmt", prog.Statements[0]) + } + if stmt.DeleteBehavior != want { + t.Errorf("round-tripped delete behaviour = %v, want %v\n--- output ---\n%s", + stmt.DeleteBehavior, want, buf.String()) + } + }) + } +} + +// assocFixture builds M.Child_Parent stored as DELETE_CASCADE — the reporter's +// starting state — and returns the association the executor will mutate in place. +// Starting from cascade rather than the default is what makes a silent fallback +// to DeleteMeButKeepReferences visible instead of indistinguishable from success. +func assocFixture(t *testing.T) (*ExecContext, *domainmodel.Association) { + t.Helper() + mod := mkModule("M") + child := mkEntity(mod.ID, "Child") + parent := mkEntity(mod.ID, "Parent") + assoc := mkAssociation(mod.ID, "Child_Parent", child.ID, parent.ID) + assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorTypeDeleteMeAndReferences, + } + + dm := &domainmodel.DomainModel{ + BaseElement: model.BaseElement{ID: nextID("dm")}, + ContainerID: mod.ID, + Entities: []*domainmodel.Entity{child, parent}, + Associations: []*domainmodel.Association{assoc}, + } + h := mkHierarchy(mod) + withContainer(h, dm.ID, mod.ID) + withContainer(h, child.ContainerID, dm.ID) + withContainer(h, parent.ContainerID, dm.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListDomainModelsFunc: func() ([]*domainmodel.DomainModel, error) { return []*domainmodel.DomainModel{dm}, nil }, + GetDomainModelFunc: func(model.ID) (*domainmodel.DomainModel, error) { return dm, nil }, + UpdateDomainModelFunc: func(*domainmodel.DomainModel) error { return nil }, + ReconcileMemberAccessesFunc: func(model.ID, string) (int, error) { return 0, nil }, + } + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + return ctx, assoc +} + +func assertStoredBehavior(t *testing.T, assoc *domainmodel.Association, want domainmodel.DeleteBehaviorType) { + t.Helper() + if assoc.ChildDeleteBehavior == nil { + t.Fatalf("no delete behaviour stored, want %q", want) + } + got := assoc.ChildDeleteBehavior.Type + switch got { + case domainmodel.DeleteBehaviorTypeDeleteMeAndReferences, + domainmodel.DeleteBehaviorTypeDeleteMeButKeepReferences, + domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences: + default: + t.Fatalf("stored %q, which is not one of Mendix's three DeletingBehavior values — "+ + "this is an unopenable model, not merely a wrong one", got) + } + if got != want { + t.Errorf("stored %q, want %q", got, want) + } +} diff --git a/mdl/executor/cmd_diff_mdl.go b/mdl/executor/cmd_diff_mdl.go index 72cbc2c19..9dfef16ab 100644 --- a/mdl/executor/cmd_diff_mdl.go +++ b/mdl/executor/cmd_diff_mdl.go @@ -184,7 +184,7 @@ func associationStmtToMDL(ctx *ExecContext, s *ast.CreateAssociationStmt) string deleteBehavior := "DELETE_BUT_KEEP_REFERENCES" switch s.DeleteBehavior { case ast.DeleteCascade: - deleteBehavior = "DELETE_CASCADE" + deleteBehavior = "DELETE_AND_REFERENCES" case ast.DeleteIfNoReferences: deleteBehavior = "DELETE_IF_NO_REFERENCES" } @@ -753,7 +753,7 @@ func associationToMDL(ctx *ExecContext, moduleName string, assoc *domainmodel.As if assoc.ChildDeleteBehavior != nil { switch assoc.ChildDeleteBehavior.Type { case domainmodel.DeleteBehaviorTypeDeleteMeAndReferences: - deleteBehavior = "DELETE_CASCADE" + deleteBehavior = "DELETE_AND_REFERENCES" case domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences: deleteBehavior = "DELETE_IF_NO_REFERENCES" } diff --git a/mdl/executor/cmd_entities.go b/mdl/executor/cmd_entities.go index 6591d8927..fc6e56318 100644 --- a/mdl/executor/cmd_entities.go +++ b/mdl/executor/cmd_entities.go @@ -994,8 +994,30 @@ func execAlterEntity(ctx *ExecContext, s *ast.AlterEntityStmt) error { } return mdlerrors.NewNotFoundMsg("attribute", s.AttributeName, fmt.Sprintf("attribute '%s' not found on entity %s", s.AttributeName, s.Name)) } - // Clean up entity-level references to the dropped attribute + // Clean up entity-level references to the dropped attribute. + // + // Two reference forms have to be matched, not one. Mendix stores an + // index's column as an element ID (AttributePointer) but a validation + // rule's Attribute and an access rule's MemberAccess as a BY_NAME + // qualified name string. Both come back in the same model.ID field, so + // comparing against the element ID alone matched nothing and left the + // rules behind — CE1613 "The selected attribute ... no longer exists." + // serializeValidationRule documents the same duality on the way out. droppedID := entity.Attributes[idx].ID + droppedQName := fmt.Sprintf("%s.%s.%s", module.Name, entity.Name, entity.Attributes[idx].Name) + // Which FIELD carries the reference also varies by engine: the legacy + // backend fills a MemberAccess's AttributeID and AttributeName both, while + // the modelsdk one leaves AttributeID empty and fills only AttributeName + // (domainmodel.go: memberAccessFromGen). Both spell it qualified, so check + // every field that could hold it rather than picking one. + refersToDropped := func(refs ...string) bool { + for _, r := range refs { + if r != "" && (r == string(droppedID) || r == droppedQName) { + return true + } + } + return false + } // Track what gets cleaned up for reporting origValidationCount := len(entity.ValidationRules) @@ -1004,7 +1026,7 @@ func execAlterEntity(ctx *ExecContext, s *ast.AlterEntityStmt) error { // Remove validation rules that reference this attribute var keepRules []*domainmodel.ValidationRule for _, vr := range entity.ValidationRules { - if vr.AttributeID != droppedID { + if !refersToDropped(string(vr.AttributeID)) { keepRules = append(keepRules, vr) } } @@ -1015,7 +1037,7 @@ func execAlterEntity(ctx *ExecContext, s *ast.AlterEntityStmt) error { for _, rule := range entity.AccessRules { var keepMembers []*domainmodel.MemberAccess for _, ma := range rule.MemberAccesses { - if ma.AttributeID != droppedID { + if !refersToDropped(string(ma.AttributeID), ma.AttributeName) { keepMembers = append(keepMembers, ma) } else { removedMemberAccess++ @@ -1029,7 +1051,7 @@ func execAlterEntity(ctx *ExecContext, s *ast.AlterEntityStmt) error { for _, idx := range entity.Indexes { var keepAttrs []*domainmodel.IndexAttribute for _, ia := range idx.Attributes { - if ia.AttributeID != droppedID { + if !refersToDropped(string(ia.AttributeID)) { keepAttrs = append(keepAttrs, ia) } } @@ -1037,7 +1059,7 @@ func execAlterEntity(ctx *ExecContext, s *ast.AlterEntityStmt) error { var keepIDs []model.ID for _, id := range idx.AttributeIDs { - if id != droppedID { + if !refersToDropped(string(id)) { keepIDs = append(keepIDs, id) } } diff --git a/mdl/executor/cmd_microflows_builder.go b/mdl/executor/cmd_microflows_builder.go index f776a6875..19e8e9ff0 100644 --- a/mdl/executor/cmd_microflows_builder.go +++ b/mdl/executor/cmd_microflows_builder.go @@ -607,6 +607,20 @@ func (fb *flowBuilder) buildSplitCondition(expr ast.Expression, fallbackExpressi if ruleCond := fb.tryBuildRuleSplitCondition(expr); ruleCond != nil { return ruleCond } + // A qualified call that did NOT resolve to a rule cannot become a condition + // either: Mendix expressions have no user-callable functions, so falling back + // to an ExpressionSplitCondition here writes text the build rejects with + // CE0117. MDL066 catches the same mistake in value positions without needing a + // project; this is the decision position, where telling a rule from a + // microflow needs the backend. Refuse rather than write it (#939). + if call := unwrapParenCall(expr); call != nil && strings.Contains(call.Name, ".") { + fb.addErrorWithExample(fmt.Sprintf( + "if condition calls '%s(...)', which is not a rule in this project — a decision "+ + "can only call a rule, and a Mendix expression cannot call anything, so the "+ + "build fails CE0117 \"Error(s) in expression\"", call.Name), + " $Result = CALL MICROFLOW "+call.Name+" (...);\n"+ + " if $Result then ... end if;") + } return µflows.ExpressionSplitCondition{ BaseElement: model.BaseElement{ID: model.ID(types.GenerateID())}, Expression: fallbackExpression, diff --git a/mdl/executor/cmd_microflows_builder_validate.go b/mdl/executor/cmd_microflows_builder_validate.go index 6bc3e86ff..edef1c2b2 100644 --- a/mdl/executor/cmd_microflows_builder_validate.go +++ b/mdl/executor/cmd_microflows_builder_validate.go @@ -383,3 +383,11 @@ func (fb *flowBuilder) validateOutputVariable(varName, statement string) { fb.addError("duplicate variable name '$%s' — %s output variable is already declared in this scope (CE0111)", varName, statement) } } + +// ValidateRuleBody validates a rule body for semantic errors (undeclared +// variables and the like) without building objects — the check-command +// counterpart of ValidateMicroflowBody. What a rule may not *contain* is a +// separate question, answered by validateRule. +func ValidateRuleBody(s *ast.CreateRuleStmt) []string { + return validateFlowBody(s.Parameters, s.Body) +} diff --git a/mdl/executor/cmd_microflows_show.go b/mdl/executor/cmd_microflows_show.go index efee72c45..ec1dad140 100644 --- a/mdl/executor/cmd_microflows_show.go +++ b/mdl/executor/cmd_microflows_show.go @@ -1348,3 +1348,237 @@ func irreducibleGraphWarnings(oc *microflows.MicroflowObjectCollection) []string } return out } + +// listRules renders SHOW / LIST RULES. A rule is its own doctype, so it has its +// own listing: SHOW MICROFLOWS lists microflows only, as it already does for +// nanoflows and workflows. +// +// The columns mirror the nanoflow listing minus "Excluded"'s neighbours that a +// rule has no concept of — a rule stores no AllowedModuleRoles, so there is +// nothing to grant and nothing to show. +func listRules(ctx *ExecContext, moduleName string) error { + h, err := getHierarchy(ctx) + if err != nil { + return mdlerrors.NewBackend("build hierarchy", err) + } + + if moduleName != "" { + if _, err := findModule(ctx, moduleName); err != nil { + return err + } + } + + rules, err := ctx.Backend.ListRules() + if err != nil { + return mdlerrors.NewBackend("list rules", err) + } + + type row struct { + qualifiedName string + module string + name string + excluded bool + folderPath string + params int + activities int + complexity int + returnType string + } + var rows []row + + for _, rule := range rules { + modID := h.FindModuleID(rule.ContainerID) + modName := h.GetModuleName(modID) + if moduleName != "" && modName != moduleName { + continue + } + returnType := "" + if rule.ReturnType != nil { + returnType = rule.ReturnType.GetTypeName() + } + rows = append(rows, row{ + qualifiedName: modName + "." + rule.Name, + module: modName, + name: rule.Name, + excluded: rule.Excluded, + folderPath: h.BuildFolderPath(rule.ContainerID), + params: len(rule.Parameters), + activities: countRuleActivities(rule), + complexity: calculateRuleComplexity(rule), + returnType: returnType, + }) + } + + sort.Slice(rows, func(i, j int) bool { + return strings.ToLower(rows[i].qualifiedName) < strings.ToLower(rows[j].qualifiedName) + }) + + result := &TableResult{ + Columns: []string{"Qualified Name", "Module", "Name", "Excluded", "Folder", "Params", "Actions", "McCabe", "Returns"}, + Summary: fmt.Sprintf("(%d rules)", len(rows)), + } + for _, r := range rows { + result.Rows = append(result.Rows, []any{r.qualifiedName, r.module, r.name, r.excluded, r.folderPath, r.params, r.activities, r.complexity, r.returnType}) + } + return writeResult(ctx, result) +} + +// countRuleActivities counts meaningful activities in a rule. +func countRuleActivities(rule *microflows.Rule) int { + if rule.ObjectCollection == nil { + return 0 + } + count := 0 + for _, obj := range rule.ObjectCollection.Objects { + switch obj.(type) { + case *microflows.StartEvent, *microflows.EndEvent, *microflows.ExclusiveMerge: + // Structural, not activities. + default: + count++ + } + } + return count +} + +// calculateRuleComplexity calculates McCabe cyclomatic complexity for a rule. +func calculateRuleComplexity(rule *microflows.Rule) int { + if rule.ObjectCollection == nil { + return 1 + } + complexity := 1 + for _, obj := range rule.ObjectCollection.Objects { + switch obj.(type) { + case *microflows.ExclusiveSplit, *microflows.InheritanceSplit, *microflows.LoopedActivity: + complexity++ + } + } + return complexity +} + +// describeRule renders DESCRIBE RULE as re-executable MDL. It mirrors +// describeNanoflow: a rule shares a microflow's body, so the body is rendered by +// wrapping it in a Microflow and reusing formatMicroflowActivities. +// +// Two rule-specific omissions, both because the document has no such property: +// no `grant execute` line (a rule stores no AllowedModuleRoles) and no +// concurrency or URL options. +func describeRule(ctx *ExecContext, name ast.QualifiedName) error { + h, err := getHierarchy(ctx) + if err != nil { + return mdlerrors.NewBackend("build hierarchy", err) + } + + entityNames := make(map[model.ID]string) + domainModels, err := ctx.Backend.ListDomainModels() + if err != nil { + return mdlerrors.NewBackend("list domain models", err) + } + for _, dm := range domainModels { + modName := h.GetModuleName(dm.ContainerID) + for _, entity := range dm.Entities { + entityNames[entity.ID] = modName + "." + entity.Name + } + } + + // A rule's body can call microflows, so the call-target lookup is the same + // one the microflow describer builds. + microflowNames := make(map[model.ID]string) + allMicroflows, err := ctx.Backend.ListMicroflows() + if err != nil { + return mdlerrors.NewBackend("list microflows", err) + } + for _, mf := range allMicroflows { + microflowNames[mf.ID] = h.GetQualifiedName(mf.ContainerID, mf.Name) + } + + allRules, err := ctx.Backend.ListRules() + if err != nil { + return mdlerrors.NewBackend("list rules", err) + } + for _, r := range allRules { + microflowNames[r.ID] = h.GetQualifiedName(r.ContainerID, r.Name) + } + + // Describe the live rule, not an excluded twin of the same name (#914). + target, _ := pickLive(allRules, + func(r *microflows.Rule) bool { + return h.GetModuleName(h.FindModuleID(r.ContainerID)) == name.Module && r.Name == name.Name + }, + func(r *microflows.Rule) bool { return r.Excluded }, + ) + if target == nil { + return mdlerrors.NewNotFound("rule", name.String()) + } + + var lines []string + + if target.Documentation != "" { + lines = append(lines, "/**") + for docLine := range strings.SplitSeq(target.Documentation, "\n") { + lines = append(lines, " * "+docLine) + } + lines = append(lines, " */") + } + if target.Excluded { + lines = append(lines, "@excluded") + } + + qualifiedName := name.Module + "." + name.Name + if len(target.Parameters) > 0 { + lines = append(lines, fmt.Sprintf("create or modify rule %s (", qualifiedName)) + for i, param := range target.Parameters { + paramType := "Object" + if param.Type != nil { + paramType = formatMicroflowDataType(ctx, param.Type, entityNames) + } + comma := "," + if i == len(target.Parameters)-1 { + comma = "" + } + lines = append(lines, fmt.Sprintf(" $%s: %s%s", param.Name, paramType, comma)) + } + lines = append(lines, ")") + } else { + lines = append(lines, fmt.Sprintf("create or modify rule %s ()", qualifiedName)) + } + + // A rule always returns Boolean or an enumeration, so unlike a microflow the + // return type is never legitimately absent — render whatever is stored and + // let the validator complain about a rule that has none. + if target.ReturnType != nil { + returnType := formatMicroflowDataType(ctx, target.ReturnType, entityNames) + if returnType != "Void" && returnType != "" { + lines = append(lines, fmt.Sprintf("returns %s", returnType)) + } + } + + if folderPath := h.BuildFolderPath(target.ContainerID); folderPath != "" { + lines = append(lines, fmt.Sprintf("folder %s", mdlQuote(folderPath))) + } + + lines = append(lines, "begin") + + wrapperMf := µflows.Microflow{ + ReturnType: target.ReturnType, + ObjectCollection: target.ObjectCollection, + } + prevDescribingReturnValue := ctx.DescribingMicroflowHasReturnValue + ctx.DescribingMicroflowHasReturnValue = microflowHasReturnValue(wrapperMf) + defer func() { + ctx.DescribingMicroflowHasReturnValue = prevDescribingReturnValue + }() + + if target.ObjectCollection != nil && len(target.ObjectCollection.Objects) > 0 { + for _, line := range formatMicroflowActivities(ctx, wrapperMf, entityNames, microflowNames) { + lines = append(lines, " "+line) + } + } else { + lines = append(lines, " -- No activities") + } + + lines = append(lines, "end;") + lines = append(lines, "/") + + fmt.Fprintln(ctx.Output, strings.Join(lines, "\n")) + return nil +} diff --git a/mdl/executor/cmd_misc.go b/mdl/executor/cmd_misc.go index 42d6ca7be..3022d4fd2 100644 --- a/mdl/executor/cmd_misc.go +++ b/mdl/executor/cmd_misc.go @@ -118,7 +118,7 @@ Domain Model - Associations: to Module.Child type Reference|ReferenceSet [owner Default|Both|Parent|Child] - [delete_behavior DELETE_BUT_KEEP_REFERENCES|DELETE_CASCADE]; + [delete_behavior DELETE_BUT_KEEP_REFERENCES|DELETE_AND_REFERENCES|DELETE_IF_NO_REFERENCES]; / drop association Module.Name; @@ -242,6 +242,7 @@ Security - Access Control: Security - Project Settings: alter project security level off|prototype|production; alter project security demo users on|off; + alter project security guest access on role |off; create demo user 'name' password 'pass' (UserRole [, ...]); drop demo user 'name'; diff --git a/mdl/executor/cmd_move.go b/mdl/executor/cmd_move.go index c783845e9..99ab6458e 100644 --- a/mdl/executor/cmd_move.go +++ b/mdl/executor/cmd_move.go @@ -71,6 +71,10 @@ func execMove(ctx *ExecContext, s *ast.MoveStmt) error { if err := moveNanoflow(ctx, s.Name, targetContainerID); err != nil { return err } + case ast.DocumentTypeRule: + if err := moveRule(ctx, s.Name, targetContainerID); err != nil { + return err + } case ast.DocumentTypeEnumeration: return moveEnumeration(ctx, s.Name, targetContainerID, targetModule.Name) case ast.DocumentTypeConstant: @@ -227,6 +231,36 @@ func moveSnippet(ctx *ExecContext, name ast.QualifiedName, targetContainerID mod } // moveNanoflow moves a nanoflow to a new container. +// moveRule reparents a rule unit. A rule's placement is ordinary — the unit row +// is ContainmentName "Documents" with the folder as its container, identical to +// a microflow (measured on ako/TestApp, where both rules sit in a folder). +func moveRule(ctx *ExecContext, name ast.QualifiedName, targetContainerID model.ID) error { + rules, err := ctx.Backend.ListRules() + if err != nil { + return mdlerrors.NewBackend("list rules", err) + } + + h, err := getHierarchy(ctx) + if err != nil { + return mdlerrors.NewBackend("build hierarchy", err) + } + + for _, rule := range rules { + modID := h.FindModuleID(rule.ContainerID) + if h.GetModuleName(modID) != name.Module || rule.Name != name.Name { + continue + } + rule.ContainerID = targetContainerID + if err := ctx.Backend.MoveRule(rule); err != nil { + return mdlerrors.NewBackend("move rule", err) + } + fmt.Fprintf(ctx.Output, "Moved rule %s to new location\n", name.String()) + return nil + } + + return mdlerrors.NewNotFound("rule", name.String()) +} + func moveNanoflow(ctx *ExecContext, name ast.QualifiedName, targetContainerID model.ID) error { // Find the nanoflow nfs, err := ctx.Backend.ListNanoflows() diff --git a/mdl/executor/cmd_pages_builder.go b/mdl/executor/cmd_pages_builder.go index 1a440f8b5..a528a4af8 100644 --- a/mdl/executor/cmd_pages_builder.go +++ b/mdl/executor/cmd_pages_builder.go @@ -51,6 +51,19 @@ type pageBuilder struct { // Entity context for resolving short attribute names inside DataViews entityContext string // Qualified entity name (e.g., "Module.Entity") + // Name of the variable the enclosing data widget is bound to, without the "$" + // (e.g. "Car" for `dataview dv (DataSource: $Car)`). Empty when the context + // object has no name of its own — a database/association/microflow source + // supplies a row object addressable only as $currentObject. Used to tell a + // SHOW_PAGE argument that names the context object from one that names + // something else, which mxcli cannot store; see cmd_pages_showpage_args.go. + contextVarName string + + // True once the walk has entered a data-bound widget, so contextVarName is + // meaningful. False means the context object is unknown (ALTER PAGE builds an + // action without traversing the stored page), not that it has no name. + contextKnown bool + // Local page/snippet variables (Variables: { $name: Type = 'default' }). // Used to distinguish a $localVar reference from a page parameter when // resolving TextTemplate parameters — local variables must be stored as diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index 6cf46fa4d..966229429 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -1237,6 +1237,20 @@ func (pb *pageBuilder) buildClientActionV3(action *ast.ActionV3) (pages.ClientAc // Build parameter mappings from Args for _, arg := range action.Args { + // mxcli stores this action with an empty ParameterMappings array and + // lets Mendix infer the argument from the enclosing widget's context + // object — required, because an explicit mapping is rejected as CE0115 + // (#296). An argument naming anything else therefore cannot be honoured, + // and was previously dropped in silence: the button opened the page with + // the context object, `mx check` reported 0 errors, and DESCRIBE printed + // the inferred mapping. Refuse instead of re-pointing the argument. + if strVal, ok := arg.Value.(string); ok && !pageArgumentBindsContextObject(strVal, pb.contextVarName, pb.contextKnown) { + return nil, mdlerrors.NewValidationf( + "show_page %s: argument %s: %s cannot be stored — a widget's page argument is always the enclosing context object, which mxcli records by leaving the mapping empty (an explicit one is rejected as CE0115). Writing %s here would silently open the page with %s instead. Use $currentObject%s, or call a microflow that shows the page with the object you want [MDL-PAGEARG01]", + action.Target, arg.Name, strVal, strVal, pb.describeContextObject(), + pb.contextVarAlternative()) + } + mapping := &pages.PageClientParameterMapping{ BaseElement: model.BaseElement{ ID: model.ID(types.GenerateID()), diff --git a/mdl/executor/cmd_pages_builder_v3_widgets.go b/mdl/executor/cmd_pages_builder_v3_widgets.go index 80eced6e4..13c17891d 100644 --- a/mdl/executor/cmd_pages_builder_v3_widgets.go +++ b/mdl/executor/cmd_pages_builder_v3_widgets.go @@ -83,8 +83,16 @@ func (pb *pageBuilder) buildDataViewV3(w *ast.WidgetV3) (*pages.DataView, error) // Save and restore entity context so nested DataViews work correctly oldContext := pb.entityContext + oldContextVar := pb.contextVarName + oldContextKnown := pb.contextKnown pb.entityContext = entityName - defer func() { pb.entityContext = oldContext }() + pb.contextVarName = contextVarFor(ds) + pb.contextKnown = true + defer func() { + pb.entityContext = oldContext + pb.contextVarName = oldContextVar + pb.contextKnown = oldContextKnown + }() // Register the widget name with its entity so template params like $dvOrder.Attr // can be resolved to Entity.Attr @@ -285,8 +293,16 @@ func (pb *pageBuilder) buildListViewV3(w *ast.WidgetV3) (*pages.ListView, error) // Save and restore entity context so nested containers work correctly oldContext := pb.entityContext + oldContextVar := pb.contextVarName + oldContextKnown := pb.contextKnown pb.entityContext = entityName - defer func() { pb.entityContext = oldContext }() + pb.contextVarName = contextVarFor(ds) + pb.contextKnown = true + defer func() { + pb.entityContext = oldContext + pb.contextVarName = oldContextVar + pb.contextKnown = oldContextKnown + }() // Register widget name with entity for SELECTION datasource lookup if w.Name != "" && entityName != "" { diff --git a/mdl/executor/cmd_pages_showpage_args.go b/mdl/executor/cmd_pages_showpage_args.go new file mode 100644 index 000000000..5b09763df --- /dev/null +++ b/mdl/executor/cmd_pages_showpage_args.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// contextVarFor returns the name a data source gives its context object, without +// the "$". Only a parameter/variable source names it; a database, association, +// microflow or selection source yields a row object addressable only as +// $currentObject, so the name is empty. +func contextVarFor(ds *ast.DataSourceV3) string { + if ds == nil || ds.Type != "parameter" { + return "" + } + return strings.TrimPrefix(ds.Reference, "$") +} + +// pageArgumentBindsContextObject reports whether the argument `value`, written on +// a SHOW_PAGE widget action, denotes the object Mendix will actually pass to the +// target page. +// +// mxcli stores a widget's show-page action with an EMPTY ParameterMappings array +// and lets Mendix infer the argument from the enclosing widget's context object. +// That is deliberate and twice-confirmed: an explicit Forms$PageParameterMapping +// whose Argument is "$currentObject" makes Studio Pro report CE0115 "parameters do +// not match", because a widget's current-row object is an inferred WidgetValue and +// not an Argument expression (issue #296, re-confirmed against mxbuild 11.12.1 for +// mxcli-formula1 §56). See the comment on *pages.PageClientAction in +// sdk/mpr/writer_widgets_action.go. +// +// The half that was missing is what happens when the author names something that +// is NOT the context object. `SHOW_PAGE Detail(Car: $Other)` inside a data view +// bound to $Car stored the same empty array, so the button opened Detail with +// $Car. The argument was not rejected, not warned about, and not visible +// afterwards: DESCRIBE prints the mapping Mendix infers, and `mx check` reports 0 +// errors, because an inferred mapping is perfectly valid. The model is a valid +// model of a different app than the one the author wrote — the exact trap §39's +// reporter spent three cycles in while distrusting a button that was correct. +// +// So the argument is honoured only when it names the context object, either as +// $currentObject or by the name of the variable the enclosing data widget is bound +// to. Anything else is refused by the caller rather than silently re-pointed. +// +// Arguments that are not a $-reference (a literal or an expression) are left +// alone: they cannot be checked this way, and refusing them would be guesswork. +// validateShowPageArguments is the check-time mirror of the executor guard, so +// `mxcli check` reports the ignored argument without needing a project — the same +// pairing as MDL-WIDGET09. contextVar is the variable the nearest enclosing data +// widget is bound to, "" when the context object has no name of its own. +func validateShowPageArguments(w *ast.WidgetV3, contextVar string, contextKnown bool, locationPrefix string) []linter.Violation { + if w == nil { + return nil + } + action := w.GetAction() + if action == nil || action.Type != "showPage" { + return nil + } + var out []linter.Violation + for _, arg := range action.Args { + strVal, ok := arg.Value.(string) + if !ok || pageArgumentBindsContextObject(strVal, contextVar, contextKnown) { + continue + } + bound := "the enclosing widget's context object" + if contextVar != "" { + bound = "$" + contextVar + } + out = append(out, linter.Violation{ + RuleID: "MDL-PAGEARG01", + Severity: linter.SeverityError, + Message: fmt.Sprintf( + "%s: widget `%s`: show_page %s argument `%s: %s` cannot be stored — a widget's page argument is always the enclosing context object, so the page would open with %s instead. Use $currentObject, or call a microflow that shows the page with the object you want", + locationPrefix, w.Name, action.Target, arg.Name, strVal, bound, + ), + }) + } + return out +} + +// describeContextObject names the object Mendix will actually pass, for the +// refusal message. +func (pb *pageBuilder) describeContextObject() string { + if pb.contextVarName != "" { + return "$" + pb.contextVarName + } + if pb.entityContext != "" { + return "the row object of the enclosing widget (" + pb.entityContext + ")" + } + return "the enclosing context object" +} + +// contextVarAlternative offers the context variable by name when it has one, so +// the message names a spelling that works rather than only one that does not. +func (pb *pageBuilder) contextVarAlternative() string { + if pb.contextVarName == "" { + return "" + } + return " (or $" + pb.contextVarName + ")" +} + +// contextKnown is false when the caller did not walk in through a data widget and +// so cannot say what the context object is — ALTER PAGE's SET/INSERT build an +// action against a stored page this pass never traverses. The guard then allows +// the argument: it only ever refuses what it can prove is discarded, and an +// unprovable case must behave exactly as it did before the guard existed. +func pageArgumentBindsContextObject(value, contextVar string, contextKnown bool) bool { + if !contextKnown { + return true + } + if !strings.HasPrefix(value, "$") { + return true + } + name := strings.TrimPrefix(value, "$") + // A path expression ($obj/Module.Assoc) is not a plain variable reference. + if strings.ContainsAny(name, "/.") { + return true + } + if strings.EqualFold(name, "currentObject") { + return true + } + return contextVar != "" && strings.EqualFold(name, contextVar) +} diff --git a/mdl/executor/cmd_pages_showpage_args_test.go b/mdl/executor/cmd_pages_showpage_args_test.go new file mode 100644 index 000000000..f116ca23a --- /dev/null +++ b/mdl/executor/cmd_pages_showpage_args_test.go @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// mxcli-formula1 §39 (adjacent): a SHOW_PAGE argument naming anything other than +// the enclosing widget's context object was discarded in silence. The page opened +// with the context object instead, `mx check` reported 0 errors, and DESCRIBE +// printed the inferred mapping — so nothing anywhere said the written argument had +// been ignored. +func TestPageArgumentBindsContextObject(t *testing.T) { + cases := []struct { + name string + value string + contextVar string + contextKnown bool + want bool + }{ + // The context object, under either spelling. + {"currentObject in a database-backed list", "$currentObject", "", true, true}, + {"currentObject inside a parameter data view", "$currentObject", "Car", true, true}, + {"the context variable by its own name", "$Car", "Car", true, true}, + {"case-insensitive, as MDL identifiers are", "$car", "Car", true, true}, + + // The bug: a different variable, silently re-pointed at the context object. + {"another page parameter", "$Other", "Car", true, false}, + {"any variable in a database-backed list", "$Other", "", true, false}, + + // Not a plain variable reference — not checkable this way, so left alone. + {"an association path", "$currentObject/Sales.Order_Customer", "Car", true, true}, + {"a literal", "'Sales'", "Car", true, true}, + {"an expression", "1 + 2", "Car", true, true}, + + // ALTER PAGE builds an action without traversing the stored page, so the + // context object is unknown, not absent. The guard must stay quiet — refusing + // here would reject `SET Action = SHOW_PAGE P(Car: $Car) ON btnGo`, which is + // correct code. + {"context unknown (ALTER PAGE)", "$Car", "", false, true}, + {"context unknown, any variable", "$Other", "", false, true}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := pageArgumentBindsContextObject(tc.value, tc.contextVar, tc.contextKnown); got != tc.want { + t.Errorf("pageArgumentBindsContextObject(%q, %q, %v) = %v, want %v", + tc.value, tc.contextVar, tc.contextKnown, got, tc.want) + } + }) + } +} + +// The check-time mirror must see the context variable of the nearest enclosing +// data widget, not of the widget carrying the action — a button has no data +// source of its own. +func TestValidateShowPageArguments_ThroughTheWidgetTree(t *testing.T) { + button := func(arg string) *ast.WidgetV3 { + return &ast.WidgetV3{ + Type: "actionbutton", + Name: "btnGo", + Properties: map[string]any{ + "Action": &ast.ActionV3{ + Type: "showPage", + Target: "Mod.Detail", + Args: []ast.FlowArgV3{{Name: "Car", Value: arg}}, + }, + }, + } + } + dataviewOn := func(ref string, child *ast.WidgetV3) *ast.WidgetV3 { + return &ast.WidgetV3{ + Type: "dataview", + Name: "dv1", + Properties: map[string]any{ + "DataSource": &ast.DataSourceV3{Type: "parameter", Reference: ref}, + }, + Children: []*ast.WidgetV3{child}, + } + } + + cases := []struct { + name string + tree []*ast.WidgetV3 + want int + }{ + {"argument is the context variable", []*ast.WidgetV3{dataviewOn("$Car", button("$Car"))}, 0}, + {"argument is $currentObject", []*ast.WidgetV3{dataviewOn("$Car", button("$currentObject"))}, 0}, + {"argument is another variable", []*ast.WidgetV3{dataviewOn("$Car", button("$Other"))}, 1}, + } + + // The tree walk resolves every widget against the registry, so a real one is + // required — the callers all pass one (ValidateWidgetProperties bails when it + // cannot be built). + registry := LoadWidgetRegistry("") + if registry == nil { + t.Fatal("LoadWidgetRegistry returned nil") + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := validateWidgetTree(tc.tree, registry, "page Mod.P") + var n int + for _, v := range got { + if v.RuleID == "MDL-PAGEARG01" { + n++ + } + } + if n != tc.want { + t.Errorf("MDL-PAGEARG01 violations = %d, want %d (all: %+v)", n, tc.want, got) + } + }) + } +} diff --git a/mdl/executor/cmd_rules_create.go b/mdl/executor/cmd_rules_create.go new file mode 100644 index 000000000..0bf5135f4 --- /dev/null +++ b/mdl/executor/cmd_rules_create.go @@ -0,0 +1,266 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package executor - CREATE RULE command +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// execCreateRule handles CREATE RULE statements. + +// Mirrors execCreateNanoflow — a rule shares a microflow's parameters, return +// type and body — with three differences that follow from the document shape +// (measured on ako/TestApp, Mendix 11.13.0): +// +// - No AllowedModuleRoles. A rule stores none, because it is not +// independently callable, so there is nothing to preserve across a rewrite +// and no default grant to apply. +// - The return type is mandatory and must be Boolean or an enumeration. +// - The flow builder runs with neither isNanoflow nor a rule flag: a rule's +// body is a server-side microflow body, and what it may NOT contain is +// refused by validateRule before the graph is built. +func execCreateRule(ctx *ExecContext, s *ast.CreateRuleStmt) error { + if !ctx.ConnectedForWrite() { + return mdlerrors.NewNotConnectedWrite() + } + + // Validate name is not empty + if strings.TrimSpace(s.Name.Name) == "" { + return mdlerrors.NewValidation("rule name must not be empty") + } + + // Find or auto-create module + module, err := findOrCreateModule(ctx, s.Name.Module) + if err != nil { + return err + } + + // Resolve folder if specified + containerID := module.ID + if s.Folder != "" { + folderID, err := resolveFolder(ctx, module.ID, s.Folder) + if err != nil { + return mdlerrors.NewBackend("resolve folder "+s.Folder, err) + } + containerID = folderID + } + + // Check whether a rule of this name already exists in the module. + var existingID model.ID + var existingContainerID model.ID + // Excluded is model state, not script state: an absent @excluded must not + // clear a stored exclusion (#914). + existingExcluded := false + // Studio Pro's rule editor writes "Variable" here on both reference rules, so + // a rule mxcli creates matches rather than storing an empty name that Studio + // Pro would fill in on first edit. + existingReturnVariableName := "Variable" + existingRules, err := ctx.Backend.ListRules() + if err != nil { + return mdlerrors.NewBackend("check existing rules", err) + } + // A module may hold several rules with this name as long as all but one are + // excluded, so target the live one rather than whichever comes first. + if existing, ok := pickLive(existingRules, + func(r *microflows.Rule) bool { + return r.Name == s.Name.Name && getModuleID(ctx, r.ContainerID) == module.ID + }, + func(r *microflows.Rule) bool { return r.Excluded }, + ); ok { + if !s.CreateOrModify { + return mdlerrors.NewAlreadyExistsMsg("rule", s.Name.Module+"."+s.Name.Name, "rule '"+s.Name.Module+"."+s.Name.Name+"' already exists (use create or modify to overwrite)") + } + existingID = existing.ID + existingContainerID = existing.ContainerID + existingExcluded = existing.Excluded + // MDL has no surface for ReturnVariableName, and Studio Pro writes one + // ("Variable" on both reference rules), so carry the stored value rather + // than blanking it on every rewrite. + existingReturnVariableName = existing.ReturnVariableName + } + + // For CREATE OR REPLACE/MODIFY, reuse the existing ID to preserve references + qualifiedName := s.Name.Module + "." + s.Name.Name + ruleID := model.ID(types.GenerateID()) + if existingID != "" { + ruleID = existingID + if s.Folder == "" { + containerID = existingContainerID + } + } + + // Build the rule. No AllowedModuleRoles: a rule document stores none. + rule := µflows.Rule{ + BaseElement: model.BaseElement{ + ID: ruleID, + }, + ContainerID: containerID, + Name: s.Name.Name, + Documentation: s.Documentation, + MarkAsUsed: false, + Excluded: s.Excluded || existingExcluded, + ReturnVariableName: existingReturnVariableName, + } + + // Load metadata needed by the entity resolver up front so backend read + // failures are returned as actionable errors instead of being treated as + // "entity not found". + dms, err := ctx.Backend.ListDomainModels() + if err != nil { + return mdlerrors.NewBackend("list domain models", err) + } + modules, err := ctx.Backend.ListModules() + if err != nil { + return mdlerrors.NewBackend("list modules", err) + } + moduleNames := make(map[model.ID]string) + for _, m := range modules { + moduleNames[m.ID] = m.Name + } + + // Build entity resolver function for parameter/return types + entityResolver := func(qn ast.QualifiedName) model.ID { + for _, dm := range dms { + modName := moduleNames[dm.ContainerID] + if modName != qn.Module { + continue + } + for _, ent := range dm.Entities { + if ent.Name == qn.Name { + return ent.ID + } + } + } + return "" + } + + // Validate and add parameters + for i, p := range s.Parameters { + if p.Type.EntityRef != nil && !isBuiltinModuleEntity(p.Type.EntityRef.Module) { + entityID := entityResolver(*p.Type.EntityRef) + if entityID == "" { + // Bare qualified name in microflow context is treated as TypeEntity by the + // visitor, but it may actually be an enumeration. Try enum lookup before failing. + if found := findEnumeration(ctx, p.Type.EntityRef.Module, p.Type.EntityRef.Name); found != nil { + s.Parameters[i].Type = ast.DataType{Kind: ast.TypeEnumeration, EnumRef: p.Type.EntityRef} + p = s.Parameters[i] + } else { + return mdlerrors.NewNotFoundMsg("entity", p.Type.EntityRef.Module+"."+p.Type.EntityRef.Name, + fmt.Sprintf("entity '%s.%s' not found for parameter '%s'", p.Type.EntityRef.Module, p.Type.EntityRef.Name, p.Name)) + } + } + } + if p.Type.Kind == ast.TypeEnumeration && p.Type.EnumRef != nil { + if found := findEnumeration(ctx, p.Type.EnumRef.Module, p.Type.EnumRef.Name); found == nil { + return mdlerrors.NewNotFoundMsg("enumeration", p.Type.EnumRef.Module+"."+p.Type.EnumRef.Name, + fmt.Sprintf("enumeration '%s.%s' not found for parameter '%s'", p.Type.EnumRef.Module, p.Type.EnumRef.Name, p.Name)) + } + } + param := µflows.MicroflowParameter{ + BaseElement: model.BaseElement{ + ID: model.ID(types.GenerateID()), + }, + ContainerID: rule.ID, + Name: p.Name, + Type: convertASTToMicroflowDataType(p.Type, entityResolver), + } + rule.Parameters = append(rule.Parameters, param) + } + + // Validate and set return type + if s.ReturnType != nil { + if s.ReturnType.Type.EntityRef != nil && !isBuiltinModuleEntity(s.ReturnType.Type.EntityRef.Module) { + entityID := entityResolver(*s.ReturnType.Type.EntityRef) + if entityID == "" { + return mdlerrors.NewNotFoundMsg("entity", s.ReturnType.Type.EntityRef.Module+"."+s.ReturnType.Type.EntityRef.Name, + fmt.Sprintf("entity '%s.%s' not found for return type", s.ReturnType.Type.EntityRef.Module, s.ReturnType.Type.EntityRef.Name)) + } + } + if s.ReturnType.Type.Kind == ast.TypeEnumeration && s.ReturnType.Type.EnumRef != nil { + if found := findEnumeration(ctx, s.ReturnType.Type.EnumRef.Module, s.ReturnType.Type.EnumRef.Name); found == nil { + return mdlerrors.NewNotFoundMsg("enumeration", s.ReturnType.Type.EnumRef.Module+"."+s.ReturnType.Type.EnumRef.Name, + fmt.Sprintf("enumeration '%s.%s' not found for return type", s.ReturnType.Type.EnumRef.Module, s.ReturnType.Type.EnumRef.Name)) + } + } + rule.ReturnType = convertASTToMicroflowDataType(s.ReturnType.Type, entityResolver) + } + + // Validate rule-specific constraints before building the flow graph. This + // covers the mandatory Boolean/enumeration return type, so rule.ReturnType is + // never left nil past this point. + if errMsg := validateRule(qualifiedName, s.Body, s.ReturnType); errMsg != "" { + return fmt.Errorf("%s", errMsg) + } + + // Build flow graph from body statements + varTypes := make(map[string]string) + declaredVars := make(map[string]string) + + for _, p := range s.Parameters { + if p.Type.EntityRef != nil { + entityQN := p.Type.EntityRef.Module + "." + p.Type.EntityRef.Name + if p.Type.Kind == ast.TypeListOf { + varTypes[p.Name] = "List of " + entityQN + } else { + varTypes[p.Name] = entityQN + } + } else { + declaredVars[p.Name] = p.Type.Kind.String() + } + } + + hierarchy, _ := getHierarchy(ctx) // best-effort: builder works without hierarchy + restServices, _ := loadRestServices(ctx) // best-effort: builder works without REST services + + builder := &flowBuilder{ + posX: 200, + posY: 200, + baseY: 200, + spacing: HorizontalSpacing, + varTypes: varTypes, + declaredVars: declaredVars, + measurer: &layoutMeasurer{varTypes: varTypes}, + backend: ctx.Backend, + hierarchy: hierarchy, + restServices: restServices, + } + + rule.ObjectCollection = builder.buildFlowGraph(s.Body, s.ReturnType) + + // Check for validation errors + if errors := builder.GetErrors(); len(errors) > 0 { + var errMsg strings.Builder + errMsg.WriteString(fmt.Sprintf("rule '%s.%s' has validation errors:\n", s.Name.Module, s.Name.Name)) + for _, err := range errors { + errMsg.WriteString(fmt.Sprintf(" - %s\n", err)) + } + return fmt.Errorf("%s", errMsg.String()) + } + + // Create or update the rule + if existingID != "" { + if err := ctx.Backend.UpdateRule(rule); err != nil { + return mdlerrors.NewBackend("update rule", err) + } + if _, err := applyDocumentFolder(ctx, rule.ID, existingContainerID, containerID); err != nil { + return err + } + ctx.ReportMutation("Replaced", "rule: %s.%s", s.Name.Module, s.Name.Name) + } else { + if err := ctx.Backend.CreateRule(rule); err != nil { + return mdlerrors.NewBackend("create rule", err) + } + fmt.Fprintf(ctx.Output, "Created rule: %s.%s\n", s.Name.Module, s.Name.Name) + } + + invalidateHierarchy(ctx) + return nil +} diff --git a/mdl/executor/cmd_rules_drop.go b/mdl/executor/cmd_rules_drop.go new file mode 100644 index 000000000..afbc81558 --- /dev/null +++ b/mdl/executor/cmd_rules_drop.go @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package executor - DROP RULE command +package executor + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" +) + +// execDropRule handles DROP RULE statements. Mirrors execDropNanoflow without +// the dropped-document memo: that memo exists to carry AllowedModuleRoles across +// a drop-then-recreate, and a rule stores none. +func execDropRule(ctx *ExecContext, s *ast.DropRuleStmt) error { + if !ctx.ConnectedForWrite() { + return mdlerrors.NewNotConnectedWrite() + } + + h, err := getHierarchy(ctx) + if err != nil { + return mdlerrors.NewBackend("build hierarchy", err) + } + + rules, err := ctx.Backend.ListRules() + if err != nil { + return mdlerrors.NewBackend("list rules", err) + } + + for _, rule := range rules { + modID := h.FindModuleID(rule.ContainerID) + if h.GetModuleName(modID) != s.Name.Module || rule.Name != s.Name.Name { + continue + } + if err := ctx.Backend.DeleteRule(rule.ID); err != nil { + return mdlerrors.NewBackend("delete rule", err) + } + invalidateHierarchy(ctx) + fmt.Fprintf(ctx.Output, "Dropped rule: %s.%s\n", s.Name.Module, s.Name.Name) + return nil + } + + return mdlerrors.NewNotFound("rule", s.Name.Module+"."+s.Name.Name) +} diff --git a/mdl/executor/cmd_security_guest_access_test.go b/mdl/executor/cmd_security_guest_access_test.go new file mode 100644 index 000000000..459ef6b5a --- /dev/null +++ b/mdl/executor/cmd_security_guest_access_test.go @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/security" +) + +// guestCall records what reached the backend, so a test can tell "refused" +// from "written with the wrong arguments". +type guestCall struct { + called bool + enabled bool + role string +} + +func guestMock(stored *security.ProjectSecurity, rec *guestCall) *mock.MockBackend { + return &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + GetProjectSecurityFunc: func() (*security.ProjectSecurity, error) { return stored, nil }, + SetProjectGuestAccessFunc: func(_ model.ID, enabled bool, role string) error { + rec.called, rec.enabled, rec.role = true, enabled, role + return nil + }, + } +} + +func guestStmt(on bool, role string) *ast.AlterProjectSecurityStmt { + return &ast.AlterProjectSecurityStmt{GuestAccessEnabled: &on, GuestUserRole: role} +} + +func projectWithRoles(guestRole string) *security.ProjectSecurity { + return &security.ProjectSecurity{ + GuestUserRole: guestRole, + UserRoles: []*security.UserRole{{Name: "Administrator"}, {Name: "Anonymous"}}, + } +} + +func TestAlterProjectSecurityGuestAccess(t *testing.T) { + t.Run("on with a known role writes it", func(t *testing.T) { + var rec guestCall + ctx, buf := newMockCtx(t, withBackend(guestMock(projectWithRoles(""), &rec))) + + assertNoError(t, execAlterProjectSecurity(ctx, guestStmt(true, "Anonymous"))) + + if !rec.called || !rec.enabled || rec.role != "Anonymous" { + t.Errorf("backend got called=%v enabled=%v role=%q; want true/true/Anonymous", + rec.called, rec.enabled, rec.role) + } + assertContainsStr(t, buf.String(), "Anonymous") + }) + + t.Run("role is stored under its declared casing", func(t *testing.T) { + var rec guestCall + ctx, _ := newMockCtx(t, withBackend(guestMock(projectWithRoles(""), &rec))) + + assertNoError(t, execAlterProjectSecurity(ctx, guestStmt(true, "anonymous"))) + + if rec.role != "Anonymous" { + t.Errorf("stored role %q, want Anonymous — a reference stored with the caller's "+ + "casing does not match the role it names", rec.role) + } + }) + + // mxbuild raises CE0133 on guest access with no role, so ON must be refused + // rather than producing a project that will not build. + t.Run("on with no role and none stored is refused", func(t *testing.T) { + var rec guestCall + ctx, _ := newMockCtx(t, withBackend(guestMock(projectWithRoles(""), &rec))) + + err := execAlterProjectSecurity(ctx, guestStmt(true, "")) + if err == nil { + t.Fatal("expected a refusal; enabling guest access with no role builds as CE0133") + } + if !strings.Contains(err.Error(), "CE0133") { + t.Errorf("error does not name the build error it prevents: %v", err) + } + if rec.called { + t.Error("backend was written to despite the refusal") + } + }) + + // The counterpart: re-enabling a project that already has one should not + // force the operator to retype it. This is why ROLE is optional. + t.Run("on with no role but one stored is allowed", func(t *testing.T) { + var rec guestCall + ctx, buf := newMockCtx(t, withBackend(guestMock(projectWithRoles("Anonymous"), &rec))) + + assertNoError(t, execAlterProjectSecurity(ctx, guestStmt(true, ""))) + + if !rec.called || !rec.enabled { + t.Fatalf("backend got called=%v enabled=%v; want true/true", rec.called, rec.enabled) + } + if rec.role != "" { + t.Errorf("role %q was rewritten; an empty role means keep the stored one", rec.role) + } + assertContainsStr(t, buf.String(), "Anonymous") + }) + + // mxbuild does NOT check this reference — a nonexistent role builds with the + // same error count as a valid one — so a typo is silent unless caught here. + t.Run("unknown role is refused", func(t *testing.T) { + var rec guestCall + ctx, _ := newMockCtx(t, withBackend(guestMock(projectWithRoles(""), &rec))) + + err := execAlterProjectSecurity(ctx, guestStmt(true, "Anonymus")) + if err == nil { + t.Fatal("expected a refusal for a role the project does not have") + } + if !strings.Contains(err.Error(), "Administrator") { + t.Errorf("error does not list the roles that do exist: %v", err) + } + if rec.called { + t.Error("backend was written to despite the refusal") + } + }) + + // OFF with a role set is valid Mendix (measured: same error count as guest + // off with no role), so the stored role is kept rather than cleared. + t.Run("off keeps the stored role", func(t *testing.T) { + var rec guestCall + ctx, buf := newMockCtx(t, withBackend(guestMock(projectWithRoles("Anonymous"), &rec))) + + assertNoError(t, execAlterProjectSecurity(ctx, guestStmt(false, ""))) + + if !rec.called || rec.enabled { + t.Fatalf("backend got called=%v enabled=%v; want true/false", rec.called, rec.enabled) + } + if rec.role != "" { + t.Errorf("role %q was written on OFF; the stored one must be left alone", rec.role) + } + assertContainsStr(t, buf.String(), "disabled") + }) + + // The three ALTER PROJECT SECURITY forms share one AST node. + t.Run("a level statement does not touch guest access", func(t *testing.T) { + var rec guestCall + ctx, _ := newMockCtx(t, withBackend(guestMock(projectWithRoles("Anonymous"), &rec))) + + assertNoError(t, execAlterProjectSecurity(ctx, &ast.AlterProjectSecurityStmt{SecurityLevel: "Production"})) + + if rec.called { + t.Error("ALTER PROJECT SECURITY LEVEL wrote guest access") + } + }) +} diff --git a/mdl/executor/cmd_security_write.go b/mdl/executor/cmd_security_write.go index 7d298ca25..9c26f14ea 100644 --- a/mdl/executor/cmd_security_write.go +++ b/mdl/executor/cmd_security_write.go @@ -1058,7 +1058,7 @@ func validateModuleRole(ctx *ExecContext, role ast.QualifiedName) error { return mdlerrors.NewNotFound("module role", role.Module+"."+role.Name) } -// execAlterProjectSecurity handles ALTER PROJECT SECURITY LEVEL/DEMO USERS. +// execAlterProjectSecurity handles ALTER PROJECT SECURITY LEVEL/DEMO USERS/GUEST ACCESS. func execAlterProjectSecurity(ctx *ExecContext, s *ast.AlterProjectSecurityStmt) error { if !ctx.ConnectedForWrite() { return mdlerrors.NewNotConnectedWrite() @@ -1100,6 +1100,69 @@ func execAlterProjectSecurity(ctx *ExecContext, s *ast.AlterProjectSecurityStmt) fmt.Fprintf(ctx.Output, "Demo users %s\n", state) } + if s.GuestAccessEnabled != nil { + if err := applyGuestAccess(ctx, ps, s); err != nil { + return err + } + } + + return nil +} + +// applyGuestAccess handles ALTER PROJECT SECURITY GUEST ACCESS ON|OFF [ROLE r]. +// +// Two things Mendix does not do for us, and one it does: +// +// - mxbuild raises CE0133 ("No user role for anonymous users selected even +// though the feature anonymous users is enabled") when access is on with no +// role, so ON is refused here rather than producing a project that will not +// build. A stored role satisfies it, which is why ROLE is optional. +// - mxbuild does NOT check that the role exists — a nonexistent one builds +// with the same error count as a valid one — so a typo would otherwise be a +// silently broken anonymous configuration. Validate it here. +// - OFF leaves the stored role in place. Guest access off with a role set is +// valid, and dropping it would lose the operator's choice on a toggle. +func applyGuestAccess(ctx *ExecContext, ps *security.ProjectSecurity, s *ast.AlterProjectSecurityStmt) error { + enabled := *s.GuestAccessEnabled + role := s.GuestUserRole + + if role != "" { + known := make([]string, 0, len(ps.UserRoles)) + var match string + for _, ur := range ps.UserRoles { + known = append(known, ur.Name) + if strings.EqualFold(ur.Name, role) { + match = ur.Name + } + } + if match == "" { + return mdlerrors.NewNotFoundMsg("user role", role, fmt.Sprintf( + "user role not found: %s (project user roles: %s). Mendix does not validate "+ + "this reference, so an unknown role would build cleanly and leave anonymous "+ + "visitors with no access", + role, strings.Join(known, ", "))) + } + // Store the role under its declared casing, not the caller's. + role = match + } else if enabled && ps.GuestUserRole == "" { + return mdlerrors.NewValidation( + "GUEST ACCESS ON requires a role: no anonymous user role is configured, and Mendix " + + "rejects anonymous access without one (CE0133). Use ALTER PROJECT SECURITY " + + "GUEST ACCESS ON ROLE ") + } + + if err := ctx.Backend.SetProjectGuestAccess(ps.ID, enabled, role); err != nil { + return mdlerrors.NewBackend("set guest access", err) + } + + if !enabled { + fmt.Fprintf(ctx.Output, "Guest access disabled\n") + return nil + } + if role == "" { + role = ps.GuestUserRole + } + fmt.Fprintf(ctx.Output, "Guest access enabled for user role %s\n", role) return nil } diff --git a/mdl/executor/cmd_workflows.go b/mdl/executor/cmd_workflows.go index 24b14c50c..d9b1083f8 100644 --- a/mdl/executor/cmd_workflows.go +++ b/mdl/executor/cmd_workflows.go @@ -528,11 +528,13 @@ func formatCallMicroflowTask(a *workflows.CallMicroflowTask, indent string) []st lines = append(lines, fmt.Sprintf("%scall microflow %s -- %s", indent, mf, caption)) } - // BoundaryEvents - lines = append(lines, formatBoundaryEvents(a.BoundaryEvents, indent+" ")...) - - // Outcomes + // Outcomes, then boundary events — the order the grammar requires + // (workflowCallMicroflowStmt: … OUTCOMES? BOUNDARY EVENT?). Emitting them + // the other way round produced DESCRIBE output that would not re-parse: + // "mismatched input 'outcomes' expecting ';'" (issue #948). It only showed + // once the default engine could read boundary events back at all. lines = append(lines, formatConditionOutcomes(a.Outcomes, indent)...) + lines = append(lines, formatBoundaryEvents(a.BoundaryEvents, indent+" ")...) return lines } diff --git a/mdl/executor/cmd_workflows_write.go b/mdl/executor/cmd_workflows_write.go index 7da427519..afc53a8ea 100644 --- a/mdl/executor/cmd_workflows_write.go +++ b/mdl/executor/cmd_workflows_write.go @@ -36,6 +36,20 @@ func execCreateWorkflow(ctx *ExecContext, s *ast.CreateWorkflowStmt) error { "remove it, or keep the note as an MDL comment (`-- ...`) [MDL-WF04]") } + // Refuse a broken reference here as well as at check time. `check + // --references` reports these, but exec runs a different pass and wrote the + // workflow anyway, so a script that skipped check produced a model the build + // rejects with CE1613 (issue #943). Same placement and reasoning as the + // microflow handler's validateMicroflowRules call (issue #833). + // + // Note this runs BEFORE findOrCreateModule, which auto-creates a module on + // demand: without it a typo'd module name silently produced a new module + // rather than an error. + if refErrors := validateWorkflowStatementRefs(ctx, s, nil); len(refErrors) > 0 { + return mdlerrors.NewValidationf("workflow '%s' has reference errors:\n - %s", + s.Name.String(), strings.Join(refErrors, "\n - ")) + } + module, err := findOrCreateModule(ctx, s.Name.Module) if err != nil { return err @@ -70,6 +84,12 @@ func execCreateWorkflow(ctx *ExecContext, s *ast.CreateWorkflowStmt) error { existingID = existing.ID existingExcluded = existing.Excluded existingContainer = existing.ContainerID + + // Refuse a rewrite that would delete a stored construct this statement + // does not restate (guard-don't-drop, ADR-0005) — issue #948. + if err := checkNoDroppedWorkflowConstructs(ctx, existingID, s.Name.String(), s); err != nil { + return err + } } containerID, err := containerForDocument(ctx, module.ID, s.Folder, existingContainer) diff --git a/mdl/executor/describe_auto.go b/mdl/executor/describe_auto.go index 10fc816c6..cd294650e 100644 --- a/mdl/executor/describe_auto.go +++ b/mdl/executor/describe_auto.go @@ -23,6 +23,7 @@ var objectTypeToDescribeKind = map[string]ast.DescribeObjectType{ "ASSOCIATION": ast.DescribeAssociation, "MICROFLOW": ast.DescribeMicroflow, "NANOFLOW": ast.DescribeNanoflow, + "RULE": ast.DescribeRule, "PAGE": ast.DescribePage, "SNIPPET": ast.DescribeSnippet, "BUILDING_BLOCK": ast.DescribeBuildingBlock, diff --git a/mdl/executor/drop_attribute_cleanup_test.go b/mdl/executor/drop_attribute_cleanup_test.go new file mode 100644 index 000000000..e654efaec --- /dev/null +++ b/mdl/executor/drop_attribute_cleanup_test.go @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// dropCleanupCtx builds an ExecContext over a Bank.Account entity whose +// validation rules and access-rule members reference their attribute the way a +// backend actually hands them back: by QUALIFIED NAME, not by element ID. +// +// Both engines do this — sdk/mpr's parseValidationRule stores the "Attribute" +// string verbatim, and mdl/backend/modelsdk/domainmodel.go says so in a comment +// ("qualified name; ruleInfoToGen handles it"). The indexes deliberately use +// element IDs, because AttributePointer really is one: they are the control +// showing the drop path itself works when the reference form matches. +func dropCleanupCtx(t *testing.T) (*ExecContext, **domainmodel.Entity) { + t.Helper() + mod := mkModule("Bank") + acct := &domainmodel.Entity{ + BaseElement: model.BaseElement{ID: nextID("ent")}, + ContainerID: nextID("dm"), + Name: "Account", + Persistable: true, + } + + qn := func(attr string) model.ID { return model.ID(fmt.Sprintf("Bank.Account.%s", attr)) } + for _, name := range []string{"AccountNumber", "Balance"} { + acct.Attributes = append(acct.Attributes, &domainmodel.Attribute{ + BaseElement: model.BaseElement{ID: nextID("attr")}, + Name: name, + }) + } + numberID := acct.Attributes[0].ID + + acct.ValidationRules = []*domainmodel.ValidationRule{ + {BaseElement: model.BaseElement{ID: nextID("vr")}, AttributeID: qn("AccountNumber"), Type: "Required"}, + {BaseElement: model.BaseElement{ID: nextID("vr")}, AttributeID: qn("AccountNumber"), Type: "Unique"}, + {BaseElement: model.BaseElement{ID: nextID("vr")}, AttributeID: qn("Balance"), Type: "Required"}, + } + // AttributeName is QUALIFIED in both engines, and which field is populated + // differs between them: the legacy backend fills AttributeID and + // AttributeName, while modelsdk (memberAccessFromGen) fills only + // AttributeName. One member of each shape, so the cleanup cannot pass by + // looking at one field. + acct.AccessRules = []*domainmodel.AccessRule{{ + BaseElement: model.BaseElement{ID: nextID("ar")}, + MemberAccesses: []*domainmodel.MemberAccess{ + {AttributeName: string(qn("AccountNumber"))}, // modelsdk shape + {AttributeName: string(qn("Balance")), AttributeID: qn("Balance")}, // legacy shape + }, + }} + acct.Indexes = []*domainmodel.Index{{ + BaseElement: model.BaseElement{ID: nextID("idx")}, + Name: "Idx_AccountNumber", + Attributes: []*domainmodel.IndexAttribute{{AttributeID: numberID}}, + AttributeIDs: []model.ID{numberID}, + }} + + dm := &domainmodel.DomainModel{ + BaseElement: model.BaseElement{ID: nextID("dm")}, + ContainerID: mod.ID, + Entities: []*domainmodel.Entity{acct}, + } + h := mkHierarchy(mod) + withContainer(h, dm.ID, mod.ID) + withContainer(h, acct.ContainerID, dm.ID) + + var saved *domainmodel.Entity + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListDomainModelsFunc: func() ([]*domainmodel.DomainModel, error) { return []*domainmodel.DomainModel{dm}, nil }, + GetDomainModelFunc: func(id model.ID) (*domainmodel.DomainModel, error) { return dm, nil }, + UpdateEntityFunc: func(dmID model.ID, e *domainmodel.Entity) error { saved = e; return nil }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + return ctx, &saved +} + +func dropAccountNumber(t *testing.T, ctx *ExecContext) { + t.Helper() + err := execAlterEntity(ctx, &ast.AlterEntityStmt{ + Name: ast.QualifiedName{Module: "Bank", Name: "Account"}, + Operation: ast.AlterEntityDropAttribute, + AttributeName: "AccountNumber", + }) + assertNoError(t, err) +} + +// TestDropAttributeRemovesValidationRulesReferencedByQualifiedName is the +// regression for the CE1613 this fixes: +// +// [error] [CE1613] "The selected attribute 'Bank.Account.AccountNumber' no +// longer exists." at Validation rule of entity 'Bank.Account' +// +// The rule outlived its attribute because the cleanup compared a qualified name +// against an element ID, so it never matched anything and quietly kept every +// rule. Measured on a real 11.13.0 project before the fix. +func TestDropAttributeRemovesValidationRulesReferencedByQualifiedName(t *testing.T) { + ctx, saved := dropCleanupCtx(t) + dropAccountNumber(t, ctx) + + if *saved == nil { + t.Fatal("expected the drop to write the entity") + } + for _, vr := range (*saved).ValidationRules { + if strings.HasSuffix(string(vr.AttributeID), ".AccountNumber") { + t.Errorf("validation rule %q outlived its attribute — this is CE1613", vr.AttributeID) + } + } + // The surviving rule must be exactly Balance's: a cleanup that drops + // everything would pass the check above while corrupting the entity. + if got := len((*saved).ValidationRules); got != 1 { + t.Fatalf("kept %d validation rules, want 1 (Balance's)", got) + } + if got := (*saved).ValidationRules[0].AttributeID; got != "Bank.Account.Balance" { + t.Errorf("kept the wrong rule: %q", got) + } +} + +// TestDropAttributeRemovesMemberAccessesReferencedByQualifiedName covers the +// same mismatch in the access-rule cleanup. A post-execution pass +// (ReconcileMemberAccesses) already repairs this downstream, so it is not a +// user-visible defect on its own — but the cleanup here reports a count it +// could never produce, and the reconcile does not run on every path. +func TestDropAttributeRemovesMemberAccessesReferencedByQualifiedName(t *testing.T) { + ctx, saved := dropCleanupCtx(t) + dropAccountNumber(t, ctx) + + members := (*saved).AccessRules[0].MemberAccesses + for _, ma := range members { + if strings.HasSuffix(string(ma.AttributeID), ".AccountNumber") || strings.HasSuffix(ma.AttributeName, ".AccountNumber") { + t.Errorf("member access %q/%q outlived its attribute", ma.AttributeID, ma.AttributeName) + } + } + if got := len(members); got != 1 { + t.Fatalf("kept %d member accesses, want 1 (Balance's)", got) + } + + out := ctx.Output.(interface{ String() string }).String() + if !strings.Contains(out, "access rule member reference") { + t.Errorf("the cleanup removed a member but did not report it; output was:\n%s", out) + } +} + +// TestDropAttributeStillCleansIndexes is the positive control. Index +// references are element IDs, so this path always worked — it proves the test +// harness drives a real drop, and that the fix did not trade one reference form +// for the other. +func TestDropAttributeStillCleansIndexes(t *testing.T) { + ctx, saved := dropCleanupCtx(t) + dropAccountNumber(t, ctx) + + if got := len((*saved).Indexes); got != 0 { + t.Errorf("kept %d indexes, want 0 (the only column was dropped)", got) + } + out := ctx.Output.(interface{ String() string }).String() + if !strings.Contains(out, "Removed 1 index(es)") { + t.Errorf("expected the index removal to be reported; output was:\n%s", out) + } +} diff --git a/mdl/executor/executor_query.go b/mdl/executor/executor_query.go index ed7653212..f5125fa53 100644 --- a/mdl/executor/executor_query.go +++ b/mdl/executor/executor_query.go @@ -35,6 +35,8 @@ func execShow(ctx *ExecContext, s *ast.ShowStmt) error { return listMicroflows(ctx, s.InModule) case ast.ShowNanoflows: return listNanoflows(ctx, s.InModule) + case ast.ShowRules: + return listRules(ctx, s.InModule) case ast.ShowPages: return listPages(ctx, s.InModule) case ast.ShowSnippets: @@ -193,6 +195,8 @@ func execDescribe(ctx *ExecContext, s *ast.DescribeStmt) error { return describeMicroflow(ctx, s.Name) case ast.DescribeNanoflow: return describeNanoflow(ctx, s.Name) + case ast.DescribeRule: + return describeRule(ctx, s.Name) case ast.DescribeModule: return describeModule(ctx, s.Name.Module, s.WithAll) case ast.DescribePage: @@ -298,6 +302,8 @@ func describeObjectTypeLabel(t ast.DescribeObjectType) string { return "microflow" case ast.DescribeNanoflow: return "nanoflow" + case ast.DescribeRule: + return "rule" case ast.DescribeModule: return "module" case ast.DescribePage: diff --git a/mdl/executor/helpers.go b/mdl/executor/helpers.go index 7c6a0f52c..61770c113 100644 --- a/mdl/executor/helpers.go +++ b/mdl/executor/helpers.go @@ -662,3 +662,22 @@ func getAttributeTypeName(at domainmodel.AttributeType) string { func formatAttributeType(at domainmodel.AttributeType) string { return getAttributeTypeName(at) } + +// buildWorkflowQualifiedNames returns a set of all workflow qualified names in +// the project. Mirrors buildPageQualifiedNames; needed so a workflow's +// `call workflow` target can be resolved (issue #943). +func buildWorkflowQualifiedNames(ctx *ExecContext) map[string]bool { + result := make(map[string]bool) + h, err := getHierarchy(ctx) + if err != nil { + return result + } + wfs, err := ctx.Backend.ListWorkflows() + if err != nil { + return result + } + for _, w := range wfs { + result[h.GetQualifiedName(w.ContainerID, w.Name)] = true + } + return result +} diff --git a/mdl/executor/register_stubs.go b/mdl/executor/register_stubs.go index 22aff1b0f..fadbd4ba8 100644 --- a/mdl/executor/register_stubs.go +++ b/mdl/executor/register_stubs.go @@ -100,6 +100,12 @@ func registerMicroflowAndNanoflowHandlers(r *Registry) { r.Register(&ast.DropNanoflowStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { return execDropNanoflow(ctx, stmt.(*ast.DropNanoflowStmt)) }) + r.Register(&ast.CreateRuleStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execCreateRule(ctx, stmt.(*ast.CreateRuleStmt)) + }) + r.Register(&ast.DropRuleStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execDropRule(ctx, stmt.(*ast.DropRuleStmt)) + }) } func registerPageHandlers(r *Registry) { diff --git a/mdl/executor/registry_test.go b/mdl/executor/registry_test.go index 671a4d5ea..2f6c6c455 100644 --- a/mdl/executor/registry_test.go +++ b/mdl/executor/registry_test.go @@ -201,6 +201,7 @@ func allKnownStatements() []ast.Statement { &ast.CreateKnowledgeBaseStmt{}, &ast.CreateMicroflowStmt{}, &ast.CreateNanoflowStmt{}, + &ast.CreateRuleStmt{}, &ast.CreateModelStmt{}, &ast.CreateModuleRoleStmt{}, &ast.CreateModuleStmt{}, @@ -247,6 +248,7 @@ func allKnownStatements() []ast.Statement { &ast.DropKnowledgeBaseStmt{}, &ast.DropMicroflowStmt{}, &ast.DropNanoflowStmt{}, + &ast.DropRuleStmt{}, &ast.DropModelStmt{}, &ast.DropModuleRoleStmt{}, &ast.DropModuleStmt{}, diff --git a/mdl/executor/roundtrip_doctype_test.go b/mdl/executor/roundtrip_doctype_test.go index 89f3ffbc8..d3434df42 100644 --- a/mdl/executor/roundtrip_doctype_test.go +++ b/mdl/executor/roundtrip_doctype_test.go @@ -57,6 +57,11 @@ var engineScriptSkip = map[string]string{ // the legacy backend refuses create/modify/drop — so the script cannot pass // there and the refusal is the intended behaviour. "legacy/26-menu-examples.mdl": "menu authoring is modelsdk-only by design; the legacy backend refuses it", + // Same shape as the menu skip: rule authoring is modelsdk-only. sdk/mpr has + // no serializeRule, and a rule document is close enough to a microflow that + // a half-written one would look valid, so the legacy backend refuses + // create/modify/drop rather than emitting one. Reads work on both engines. + "legacy/rules.mdl": "rule authoring is modelsdk-only by design; the legacy backend refuses it", // The legacy widget builder has no `linechart` template either, so the OL08 // LineChart object-list example (added in 6b837ad7) fails page build // ("template not found: linechart"). Passes on modelsdk. Same class as the diff --git a/mdl/executor/roundtrip_workflow_test.go b/mdl/executor/roundtrip_workflow_test.go index 4bc2d64a8..c9a629a79 100644 --- a/mdl/executor/roundtrip_workflow_test.go +++ b/mdl/executor/roundtrip_workflow_test.go @@ -22,6 +22,30 @@ import ( // - WAIT FOR NOTIFICATION (with BOUNDARY EVENT NON INTERRUPTING TIMER) // - JUMP TO (inside DECISION outcome) // - CALL WORKFLOW (sub-workflow with parameter expression) +// +// createTaskPages creates the pages a user task's `page` clause points at. +// +// These tests used to name pages they never created. `check --references` and +// exec now resolve a workflow's page references (issue #943), so the missing +// page is reported rather than written into the model — which is the point of +// that check: Mendix rejects the same workflow with CE1613. +func createTaskPages(t *testing.T, env *testEnv, mod string, names ...string) { + t.Helper() + for _, name := range names { + mdl := `create page ` + mod + `.` + name + ` ( + Title: '` + name + `', + Layout: Atlas_Core.Atlas_Default + ) { + layoutgrid g { row r { column c (DesktopWidth: 12) { + dynamictext dt (Content: '` + name + `') + } } } + }` + if err := env.executeMDL(mdl); err != nil { + t.Fatalf("create page %s.%s: %v", mod, name, err) + } + } +} + func TestRoundtripWorkflow_Comprehensive(t *testing.T) { env := setupTestEnv(t) defer env.teardown() @@ -53,6 +77,9 @@ func TestRoundtripWorkflow_Comprehensive(t *testing.T) { t.Fatalf("create ScoreCalc: %v", err) } + // Task pages named by the user tasks below. + createTaskPages(t, env, mod, "SubPage", "ReviewPage", "MultiReviewPage", "ApprovePage") + // Sub-workflow for CALL WORKFLOW if err := env.executeMDL(`create workflow ` + mod + `.SubApprovalFlow parameter $WorkflowContext: ` + mod + `.WfCtxEntity @@ -162,8 +189,20 @@ end workflow;` } } +// Run on BOTH engines. setupTestEnv defaults to legacy, and legacy could always +// read boundary events — which is exactly why the default (modelsdk) engine +// having no boundary-event reader at all stayed invisible: DESCRIBE emitted +// nothing on the engine users actually run, so describe → exec silently dropped +// the timer and its handler flow, while this test stayed green (issue #948). +// Same shape as the TableMappings gap in roundtrip_dbconnection_test.go. func TestRoundtripWorkflow_BoundaryEventInterrupting(t *testing.T) { - env := setupTestEnv(t) + for _, eng := range gateEngines { + t.Run(eng.name, func(t *testing.T) { testRoundtripWorkflowBoundaryInterrupting(t, eng) }) + } +} + +func testRoundtripWorkflowBoundaryInterrupting(t *testing.T, eng gateEngine) { + env := setupTestEnvWithBackend(t, eng.factory) defer env.teardown() createMDL := `create workflow ` + testModule + `.WfBoundaryInt @@ -179,6 +218,7 @@ end workflow;` if err := env.executeMDL(`create or modify persistent entity ` + testModule + `.TestEntitySimple (Name: String(100));`); err != nil { t.Fatalf("Failed to create entity: %v", err) } + createTaskPages(t, env, testModule, "ReviewPage") if err := env.executeMDL(createMDL); err != nil { t.Fatalf("Failed to create workflow: %v", err) @@ -194,8 +234,15 @@ end workflow;` } } +// Both engines, for the reason on the interrupting variant above. func TestRoundtripWorkflow_BoundaryEventNonInterrupting(t *testing.T) { - env := setupTestEnv(t) + for _, eng := range gateEngines { + t.Run(eng.name, func(t *testing.T) { testRoundtripWorkflowBoundaryNonInterrupting(t, eng) }) + } +} + +func testRoundtripWorkflowBoundaryNonInterrupting(t *testing.T, eng gateEngine) { + env := setupTestEnvWithBackend(t, eng.factory) defer env.teardown() createMDL := `create workflow ` + testModule + `.WfBoundaryNonInt @@ -211,6 +258,7 @@ end workflow;` if err := env.executeMDL(`create or modify persistent entity ` + testModule + `.TestEntitySimple2 (Name: String(100));`); err != nil { t.Fatalf("Failed to create entity: %v", err) } + createTaskPages(t, env, testModule, "ReviewPage") if err := env.executeMDL(createMDL); err != nil { t.Fatalf("Failed to create workflow: %v", err) @@ -242,6 +290,7 @@ end workflow;` if err := env.executeMDL(`create or modify persistent entity ` + testModule + `.TestEntityMulti (Name: String(100));`); err != nil { t.Fatalf("Failed to create entity: %v", err) } + createTaskPages(t, env, testModule, "ReviewPage") if err := env.executeMDL(createMDL); err != nil { t.Fatalf("Failed to create workflow: %v", err) diff --git a/mdl/executor/rule_catalog_integration_test.go b/mdl/executor/rule_catalog_integration_test.go new file mode 100644 index 000000000..a1d4c8537 --- /dev/null +++ b/mdl/executor/rule_catalog_integration_test.go @@ -0,0 +1,169 @@ +// SPDX-License-Identifier: Apache-2.0 + +//go:build integration + +package executor + +import ( + "fmt" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/modelsdk/mpr" + "go.mongodb.org/mongo-driver/bson" +) + +// ruleKeys is Microflows$Rule's property list. A rule is a microflow minus nine +// properties, so the fixture below is built by keeping exactly these. +var ruleKeys = []string{ + "Name", "Documentation", "Excluded", "ExportLevel", "ObjectCollection", + "Flows", "MicroflowReturnType", "MarkAsUsed", "ReturnVariableName", "ApplyEntityAccess", +} + +// convertMicroflowUnitToRule rewrites an mxcli-created microflow document as a +// Microflows$Rule, in place. MDL cannot yet author a rule (that is slice 3), and +// no module in a blank Mendix app ships one, so this is how the test gets a rule +// whose body calls something. +func convertMicroflowUnitToRule(t *testing.T, projectPath, unitID string) { + t.Helper() + + r, err := mpr.Open(projectPath) + if err != nil { + t.Fatalf("open project: %v", err) + } + raw, err := r.GetRawUnitBytes(unitID) + if err != nil { + t.Fatalf("read unit: %v", err) + } + var containerBlob []byte + if err := r.DB().QueryRow(`SELECT ContainerID FROM Unit WHERE UnitID = ?`, + mpr.IDToBsonBinary(unitID).Data).Scan(&containerBlob); err != nil { + t.Fatalf("read container: %v", err) + } + containerID := mpr.BlobToUUID(containerBlob) + if err := r.Close(); err != nil { + t.Fatalf("close reader: %v", err) + } + + var doc bson.D + if err := bson.Unmarshal(raw, &doc); err != nil { + t.Fatalf("unmarshal unit: %v", err) + } + get := func(k string) (any, bool) { + for _, e := range doc { + if e.Key == k { + return e.Value, true + } + } + return nil, false + } + + id, _ := get("$ID") + out := bson.D{{Key: "$ID", Value: id}, {Key: "$Type", Value: "Microflows$Rule"}} + for _, k := range ruleKeys { + if v, ok := get(k); ok { + out = append(out, bson.E{Key: k, Value: v}) + } + } + contents, err := bson.Marshal(out) + if err != nil { + t.Fatalf("marshal rule: %v", err) + } + + w, err := mpr.NewWriter(projectPath) + if err != nil { + t.Fatalf("open writer: %v", err) + } + defer w.Close() + if err := w.DeleteUnit(unitID); err != nil { + t.Fatalf("delete microflow unit: %v", err) + } + if err := w.InsertUnit(unitID, containerID, "Documents", "Microflows$Rule", contents); err != nil { + t.Fatalf("insert rule unit: %v", err) + } +} + +// A microflow called only from inside a rule's body must not read as dead code. +// +// Before rules were catalog objects whose bodies are walked, the reference graph +// never entered a rule, so every document a rule called was invisible: `show +// callers` reported none and GRAPH_DEAD_ASSETS listed it. Measured on the +// reference app before the fix, Rules.OnlyCalledByRule was reported dead. +// +// The unreferenced microflow is the control: it must stay dead, so the test +// cannot pass by the dead-asset view simply going empty. +func TestRuleBodyReferencesAreNotDeadCode(t *testing.T) { + env := setupTestEnv(t) + + if err := env.executeMDL(` +create or modify microflow TestModule.CalledFromRule () returns Boolean +begin + return true; +end +/ +create or modify microflow TestModule.NeverCalled () returns Boolean +begin + return true; +end +/ +create or modify microflow TestModule.WillBecomeRule () returns Boolean +begin + $Ok = call microflow TestModule.CalledFromRule (); + return $Ok; +end +/ +`); err != nil { + t.Fatalf("create fixtures: %v", err) + } + + unitID := microflowUnitID(t, env, "WillBecomeRule") + convertMicroflowUnitToRule(t, env.projectPath, unitID) + + // Reconnect so the executor's reader sees the rewritten unit, then rebuild. + if err := env.executor.Execute(&ast.ConnectStmt{Path: env.projectPath}); err != nil { + t.Fatalf("reconnect: %v", err) + } + buildCatalogFull(t, env) + + if n := countRefs(t, env, "TestModule.WillBecomeRule", "TestModule.CalledFromRule", "call"); n != 1 { + t.Errorf("call edge from the rule to CalledFromRule = %d, want 1 — a rule's body must be walked for references", n) + } + + dead := deadAssetNames(t, env) + if dead["TestModule.CalledFromRule"] { + t.Error("CalledFromRule is reported dead, but a rule calls it") + } + if !dead["TestModule.NeverCalled"] { + t.Error("control failed: NeverCalled should be dead, so the assertion above is not passing vacuously") + } +} + +// microflowUnitID resolves a microflow's unit id by name. +func microflowUnitID(t *testing.T, env *testEnv, name string) string { + t.Helper() + mfs, err := env.executor.Backend().ListMicroflows() + if err != nil { + t.Fatalf("list microflows: %v", err) + } + for _, mf := range mfs { + if mf.Name == name { + return string(mf.ID) + } + } + t.Fatalf("microflow %s not found", name) + return "" +} + +// deadAssetNames returns the qualified names GRAPH_DEAD_ASSETS reports. +func deadAssetNames(t *testing.T, env *testEnv) map[string]bool { + t.Helper() + result, err := env.executor.catalog.Query("SELECT QualifiedName FROM graph_dead_assets") + if err != nil { + t.Fatalf("dead-asset query failed: %v", err) + } + out := map[string]bool{} + for _, row := range result.Rows { + out[fmt.Sprintf("%v", row[0])] = true + } + return out +} diff --git a/mdl/executor/rule_validation.go b/mdl/executor/rule_validation.go new file mode 100644 index 000000000..7baa5b1d0 --- /dev/null +++ b/mdl/executor/rule_validation.go @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// Rule restrictions, from Mendix's own reference (docs.mendix.com/refguide/rules): +// a rule "is a special kind of microflow" whose output must be a Boolean or an +// enumeration, may only be used from a decision, and +// +// - cannot create, delete, modify or roll back database objects; +// - cannot show a page, close a window, show a message, send validation +// feedback, or start a file download; +// - cannot call a web service, generate a document, or import/export XML. +// +// The denylist mirrors checkDisallowedNanoflowAction, and carries the same +// maintenance property: an action type not listed here is implicitly allowed, so +// a new AST statement that a rule may not contain needs a case adding. +// +// mxbuild enforces all of this, so these refusals are measured rather than +// derived from the documentation. Verified on mxbuild 11.13.0 by converting a +// microflow unit into a Microflows$Rule to get past this validator: +// +// create object in a rule → CE0009 "This action is not supported in rules." +// void or String return → CE0103 "The return type should be one of the +// following types: Boolean or Enumeration." and +// CE0139 "The return type of a rule must be either +// Boolean or an enumeration type." +// +// They are refused at check time rather than left to the build because the +// grammar deliberately shares microflowBody: restricting the body in the grammar +// would turn every violation into a parse error naming a token instead of the +// activity. +func checkDisallowedRuleAction(stmt ast.MicroflowStatement) string { + switch stmt.(type) { + // Data changes — the restriction that makes a rule cheaper than a microflow. + case *ast.CreateObjectStmt: + return "a rule cannot create objects" + case *ast.ChangeObjectStmt: + return "a rule cannot change objects" + case *ast.DeleteObjectStmt: + return "a rule cannot delete objects" + case *ast.MfCommitStmt: + return "a rule cannot commit objects" + case *ast.RollbackStmt: + return "a rule cannot roll back objects" + + // Client interaction — a rule has no client to talk to. + case *ast.ShowPageStmt: + return "a rule cannot show a page" + case *ast.ClosePageStmt: + return "a rule cannot close a page" + case *ast.ShowMessageStmt: + return "a rule cannot show a message" + case *ast.ValidationFeedbackStmt: + return "a rule cannot send validation feedback" + case *ast.ShowHomePageStmt: + return "a rule cannot show the home page" + case *ast.DownloadFileStmt: + return "a rule cannot start a file download" + + // Integration. (Mendix also bans document generation from a rule; mxcli has + // no GENERATE DOCUMENT statement, so there is nothing to refuse yet.) + case *ast.CallWebServiceStmt: + return "a rule cannot call a web service" + case *ast.ImportFromMappingStmt: + return "a rule cannot import XML/JSON via a mapping" + case *ast.ExportToMappingStmt: + return "a rule cannot export XML/JSON via a mapping" + } + return "" +} + +// validateRuleBody walks the body — including branch, loop and error-handler +// bodies — collecting every disallowed activity. +func validateRuleBody(body []ast.MicroflowStatement) []string { + var errors []string + validateRuleStatements(body, &errors) + return errors +} + +func validateRuleStatements(stmts []ast.MicroflowStatement, errors *[]string) { + for _, stmt := range stmts { + if reason := checkDisallowedRuleAction(stmt); reason != "" { + *errors = append(*errors, reason) + continue + } + switch s := stmt.(type) { + case *ast.IfStmt: + validateRuleStatements(s.ThenBody, errors) + validateRuleStatements(s.ElseBody, errors) + case *ast.LoopStmt: + validateRuleStatements(s.Body, errors) + case *ast.WhileStmt: + validateRuleStatements(s.Body, errors) + } + if eh := getErrorHandling(stmt); eh != nil && eh.Body != nil { + validateRuleStatements(eh.Body, errors) + } + } +} + +// validateRuleReturnType enforces the one restriction that is about the +// signature rather than the body: a rule returns a Boolean or an enumeration. +// +// A missing return type is an error too. For a microflow, void is a legitimate +// choice; for a rule it means the decision that calls it has nothing to branch +// on, so silently writing a void rule produces a document no decision can use. +func validateRuleReturnType(retType *ast.MicroflowReturnType) string { + if retType == nil { + return "a rule must return a Boolean or an enumeration — add `returns Boolean` or `returns enum Module.Enum`" + } + switch retType.Type.Kind { + case ast.TypeBoolean, ast.TypeEnumeration: + return "" + case ast.TypeVoid: + return "a rule must return a Boolean or an enumeration, not void" + default: + return fmt.Sprintf("a rule must return a Boolean or an enumeration, not %s", retType.Type.Kind.String()) + } +} + +// validateRule returns a formatted error message, or "" when the rule is valid. +// Called by both `mxcli check` and exec, so the two cannot disagree. +func validateRule(name string, body []ast.MicroflowStatement, retType *ast.MicroflowReturnType) string { + var allErrors []string + + if msg := validateRuleReturnType(retType); msg != "" { + allErrors = append(allErrors, msg) + } + allErrors = append(allErrors, validateRuleBody(body)...) + + if len(allErrors) == 0 { + return "" + } + + var errMsg strings.Builder + errMsg.WriteString(fmt.Sprintf("rule '%s' has validation errors:\n", name)) + for _, e := range allErrors { + errMsg.WriteString(fmt.Sprintf(" - %s\n", e)) + } + return errMsg.String() +} diff --git a/mdl/executor/rule_validation_test.go b/mdl/executor/rule_validation_test.go new file mode 100644 index 000000000..45d18ff17 --- /dev/null +++ b/mdl/executor/rule_validation_test.go @@ -0,0 +1,116 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +func boolReturn() *ast.MicroflowReturnType { + return &ast.MicroflowReturnType{Type: ast.DataType{Kind: ast.TypeBoolean}} +} + +// A rule's return type is not a style preference: mxbuild rejects anything but +// Boolean or an enumeration with CE0103 + CE0139, measured on 11.13.0 by +// converting a microflow unit into a rule to get past this validator. +func TestRuleReturnTypeMustBeBooleanOrEnumeration(t *testing.T) { + cases := []struct { + name string + ret *ast.MicroflowReturnType + wantErr bool + }{ + {"boolean", boolReturn(), false}, + {"enumeration", &ast.MicroflowReturnType{Type: ast.DataType{Kind: ast.TypeEnumeration}}, false}, + {"absent", nil, true}, + {"void", &ast.MicroflowReturnType{Type: ast.DataType{Kind: ast.TypeVoid}}, true}, + {"string", &ast.MicroflowReturnType{Type: ast.DataType{Kind: ast.TypeString}}, true}, + {"integer", &ast.MicroflowReturnType{Type: ast.DataType{Kind: ast.TypeInteger}}, true}, + } + for _, c := range cases { + msg := validateRuleReturnType(c.ret) + if (msg != "") != c.wantErr { + t.Errorf("%s: got %q, wantErr=%v", c.name, msg, c.wantErr) + } + } +} + +// The activities Mendix forbids in a rule. mxbuild reports CE0009 "This action +// is not supported in rules." for each; a rule that reached the build carrying +// one is a document mxcli should never have written. +func TestRuleBodyRefusesDisallowedActivities(t *testing.T) { + cases := []struct { + name string + stmt ast.MicroflowStatement + want string + }{ + {"create", &ast.CreateObjectStmt{}, "cannot create objects"}, + {"change", &ast.ChangeObjectStmt{}, "cannot change objects"}, + {"delete", &ast.DeleteObjectStmt{}, "cannot delete objects"}, + {"commit", &ast.MfCommitStmt{}, "cannot commit objects"}, + {"rollback", &ast.RollbackStmt{}, "cannot roll back objects"}, + {"show page", &ast.ShowPageStmt{}, "cannot show a page"}, + {"close page", &ast.ClosePageStmt{}, "cannot close a page"}, + {"show message", &ast.ShowMessageStmt{}, "cannot show a message"}, + {"validation feedback", &ast.ValidationFeedbackStmt{}, "cannot send validation feedback"}, + {"download", &ast.DownloadFileStmt{}, "cannot start a file download"}, + {"web service", &ast.CallWebServiceStmt{}, "cannot call a web service"}, + } + for _, c := range cases { + errs := validateRuleBody([]ast.MicroflowStatement{c.stmt}) + if len(errs) != 1 || !strings.Contains(errs[0], c.want) { + t.Errorf("%s: got %v, want one error containing %q", c.name, errs, c.want) + } + } +} + +// A rule may do everything a rule is FOR — retrieve, branch, call a microflow, +// compute. Without this control the denylist could reject the whole language and +// the test above would still pass. +func TestRuleBodyAllowsWhatARuleIsFor(t *testing.T) { + allowed := []ast.MicroflowStatement{ + &ast.RetrieveStmt{}, + &ast.CallMicroflowStmt{}, + &ast.DeclareStmt{}, + &ast.ReturnStmt{}, + } + if errs := validateRuleBody(allowed); len(errs) != 0 { + t.Errorf("a rule must be allowed to retrieve, call and compute; got %v", errs) + } +} + +// A disallowed activity nested inside a branch or a loop is still disallowed — +// hiding a create inside an `if` must not get it past the check. +func TestRuleBodyWalksNestedBodies(t *testing.T) { + nested := []ast.MicroflowStatement{ + &ast.IfStmt{ + ThenBody: []ast.MicroflowStatement{&ast.CreateObjectStmt{}}, + ElseBody: []ast.MicroflowStatement{ + &ast.LoopStmt{Body: []ast.MicroflowStatement{&ast.ShowMessageStmt{}}}, + }, + }, + } + errs := validateRuleBody(nested) + if len(errs) != 2 { + t.Fatalf("got %d errors %v, want 2 (one per nested body)", len(errs), errs) + } +} + +// validateRule is what both `check` and `exec` call, so the two cannot disagree +// about what a rule may contain. It reports every problem at once rather than +// the first. +func TestValidateRuleReportsEveryProblem(t *testing.T) { + msg := validateRule("Mod.R", + []ast.MicroflowStatement{&ast.CreateObjectStmt{}, &ast.ShowPageStmt{}}, + &ast.MicroflowReturnType{Type: ast.DataType{Kind: ast.TypeString}}) + for _, want := range []string{"Mod.R", "not String", "cannot create objects", "cannot show a page"} { + if !strings.Contains(msg, want) { + t.Errorf("message missing %q; got:\n%s", want, msg) + } + } + if validateRule("Mod.R", []ast.MicroflowStatement{&ast.ReturnStmt{}}, boolReturn()) != "" { + t.Error("a valid rule must produce no message") + } +} diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index 8292b972c..b81196cda 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -26,6 +26,7 @@ type scriptContext struct { nanoflows map[string]bool // Nanoflows created (Module.Nanoflow) pages map[string]bool // Pages created (Module.Page) snippets map[string]bool // Snippets created (Module.Snippet) + workflows map[string]bool // Workflows created (Module.Workflow) // Java/JavaScript actions created in the script, mapped to their declared // parameter names. A bool would be enough to stop the false "not found", @@ -44,6 +45,7 @@ func newScriptContext() *scriptContext { microflows: make(map[string]bool), nanoflows: make(map[string]bool), pages: make(map[string]bool), + workflows: make(map[string]bool), snippets: make(map[string]bool), javaActions: make(map[string][]string), @@ -99,6 +101,10 @@ func (sc *scriptContext) collectDefinitions(prog *ast.Program) { if s.Name.Module != "" { sc.snippets[s.Name.String()] = true } + case *ast.CreateWorkflowStmt: + if s.Name.Module != "" { + sc.workflows[s.Name.String()] = true + } case *ast.CreateJavaActionStmt: if s.Name.Module != "" { sc.javaActions[s.Name.String()] = codeActionParamNames(s.Parameters) @@ -148,6 +154,10 @@ func (sc *scriptContext) collectSingle(stmt ast.Statement) { if s.Name.Module != "" { sc.snippets[s.Name.String()] = true } + case *ast.CreateWorkflowStmt: + if s.Name.Module != "" { + sc.workflows[s.Name.String()] = true + } case *ast.CreateJavaActionStmt: if s.Name.Module != "" { sc.javaActions[s.Name.String()] = codeActionParamNames(s.Parameters) @@ -180,6 +190,9 @@ func (sc *scriptContext) allNames() []string { for n := range sc.snippets { names = append(names, n) } + for n := range sc.workflows { + names = append(names, n) + } for n := range sc.javaActions { names = append(names, n) } @@ -446,6 +459,27 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext return mdlerrors.NewValidationf("microflow '%s' has reference errors:\n - %s", s.Name.String(), strings.Join(refErrors, "\n - ")) } + case *ast.CreateRuleStmt: + if s.Name.Module != "" && !sc.modules[s.Name.Module] { + if _, err := findModule(ctx, s.Name.Module); err != nil { + return mdlerrors.NewNotFound("module", s.Name.Module) + } + } + // The same validateRule the executor calls, so `check` and `exec` cannot + // disagree about what a rule may contain. + if errMsg := validateRule(s.Name.String(), s.Body, s.ReturnType); errMsg != "" { + return mdlerrors.NewValidationf("%s", strings.TrimRight(errMsg, "\n")) + } + if validationErrors := ValidateRuleBody(s); len(validationErrors) > 0 { + return mdlerrors.NewValidationf("rule '%s' has validation errors:\n - %s", + s.Name.String(), strings.Join(validationErrors, "\n - ")) + } + if !s.Excluded { + if refErrors := validateFlowBodyReferences(ctx, s.Body, sc); len(refErrors) > 0 { + return mdlerrors.NewValidationf("rule '%s' has reference errors:\n - %s", + s.Name.String(), strings.Join(refErrors, "\n - ")) + } + } case *ast.CreateNanoflowStmt: if s.Name.Module != "" && !sc.modules[s.Name.Module] { if _, err := findModule(ctx, s.Name.Module); err != nil { @@ -497,10 +531,21 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext s.Name.String(), strings.Join(ctxErrors, "\n - ")) } case *ast.CreateWorkflowStmt: - // Reference check: every workflow call-microflow must map all of its - // target microflow's parameters (FINDINGS #40). Syntax-only workflow checks - // (MDL-WF01/02/03) run separately in the no-project phase. - if refErrors := validateWorkflowParameterMappings(ctx, s, sc); len(refErrors) > 0 { + // Two reference passes. Missing targets first: a name that resolves to + // nothing is the more basic error, and reporting "parameter not mapped" + // for a microflow that does not exist would be actively misleading. + // Syntax-only workflow checks (MDL-WF01/02/03) run separately in the + // no-project phase. + refErrors := validateWorkflowStatementRefs(ctx, s, sc) + // Then, for targets that do resolve, that every parameter is mapped + // (FINDINGS #40). + refErrors = append(refErrors, validateWorkflowParameterMappings(ctx, s, sc)...) + if len(refErrors) > 0 { + return mdlerrors.NewValidationf("workflow '%s' has reference errors:\n - %s", + s.Name.String(), strings.Join(refErrors, "\n - ")) + } + case *ast.AlterWorkflowStmt: + if refErrors := validateAlterWorkflowRefs(ctx, s, sc); len(refErrors) > 0 { return mdlerrors.NewValidationf("workflow '%s' has reference errors:\n - %s", s.Name.String(), strings.Join(refErrors, "\n - ")) } diff --git a/mdl/executor/validate_duplicates.go b/mdl/executor/validate_duplicates.go index 2d58eaa05..5eaad1309 100644 --- a/mdl/executor/validate_duplicates.go +++ b/mdl/executor/validate_duplicates.go @@ -102,6 +102,8 @@ func stmtCreateInfo(stmt ast.Statement) (docType, name string, idempotent bool) return "microflow", s.Name.String(), s.CreateOrModify case *ast.CreateNanoflowStmt: return "nanoflow", s.Name.String(), s.CreateOrModify + case *ast.CreateRuleStmt: + return "rule", s.Name.String(), s.CreateOrModify case *ast.CreatePageStmtV3: return "page", s.Name.String(), s.IsModify || s.IsReplace case *ast.CreateSnippetStmtV3: diff --git a/mdl/executor/validate_microflow.go b/mdl/executor/validate_microflow.go index c1c84092d..8f5c3a2d2 100644 --- a/mdl/executor/validate_microflow.go +++ b/mdl/executor/validate_microflow.go @@ -175,6 +175,7 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { case *ast.ReturnStmt: v.checkReturn(stmt) v.checkExprFunctions("return", stmt.Value) + v.checkQualifiedCallInExpression("return", stmt.Value) v.checkDivisionSlash("return", stmt.Value) v.checkDateTimeLiterals("return", stmt.Value) case *ast.IfStmt: @@ -270,6 +271,7 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { // #893 item 1: a Create Variable activity requires a value (CE0038). v.checkDeclareHasValue(stmt) v.checkExprFunctions(fmt.Sprintf("declare '$%s'", stmt.Variable), stmt.InitialValue) + v.checkQualifiedCallInExpression(fmt.Sprintf("declare '$%s'", stmt.Variable), stmt.InitialValue) v.checkDivisionSlash(fmt.Sprintf("declare '$%s'", stmt.Variable), stmt.InitialValue) v.checkDateTimeLiterals(fmt.Sprintf("declare '$%s'", stmt.Variable), stmt.InitialValue) case *ast.MfSetStmt: @@ -281,6 +283,7 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { } } v.checkExprFunctions(fmt.Sprintf("set '%s'", stmt.Target), stmt.Value) + v.checkQualifiedCallInExpression(fmt.Sprintf("set '%s'", stmt.Target), stmt.Value) v.checkDivisionSlash(fmt.Sprintf("set '%s'", stmt.Target), stmt.Value) v.checkDateTimeLiterals(fmt.Sprintf("set '%s'", stmt.Target), stmt.Value) case *ast.RetrieveStmt: @@ -294,6 +297,13 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { } case *ast.SynchronizeStmt: v.checkSynchronizeIsNanoflowOnly() + case *ast.WhileStmt: + // A while condition is a plain boolean expression — Mendix has no rule + // split for a loop, so unlike an IF condition a qualified call here is + // never legal. The body was not walked at all before, so nothing inside + // a while was checked. + v.checkQualifiedCallInExpression("while condition", stmt.Condition) + v.walkBody(stmt.Body) case *ast.CallMicroflowStmt: v.checkAssociationObjectArgs("microflow "+stmt.MicroflowName.String(), stmt.Arguments) case *ast.CallNanoflowStmt: @@ -344,10 +354,12 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { // but check previously only inspected return/if/declare/set (FINDINGS #17). for _, ch := range stmt.Changes { v.checkExprFunctions(fmt.Sprintf("create %s attribute '%s'", stmt.EntityType.String(), ch.Attribute), ch.Value) + v.checkQualifiedCallInExpression(fmt.Sprintf("create %s attribute '%s'", stmt.EntityType.String(), ch.Attribute), ch.Value) } case *ast.ChangeObjectStmt: for _, ch := range stmt.Changes { v.checkExprFunctions(fmt.Sprintf("change '%s' attribute '%s'", stmt.Variable, ch.Attribute), ch.Value) + v.checkQualifiedCallInExpression(fmt.Sprintf("change '%s' attribute '%s'", stmt.Variable, ch.Attribute), ch.Value) } } // Check error handling inside loops diff --git a/mdl/executor/validate_qualified_call.go b/mdl/executor/validate_qualified_call.go new file mode 100644 index 000000000..c83b9c54a --- /dev/null +++ b/mdl/executor/validate_qualified_call.go @@ -0,0 +1,81 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// A Mendix expression has no user-callable functions. Its function library is +// built-in and unqualified (`length`, `toString`, `contains`, …); microflows, +// rules and Java actions are called by ACTIVITIES, never from an expression. +// +// So `Module.Name(arg = $x)` in a value position is not a Mendix expression at +// all, and mxcli used to write it as literal expression text on BOTH engines. +// Measured on mxbuild 11.13.0 (upstream #939): +// +// declare $b Boolean = Sample.Rule_IsActive(IsActive = $IsActive); +// → [CE0117] "Error(s) in expression." at Create variable activity +// declare $n Integer = Sample.MF_Callee(N = $N); +// → [CE0117] — a microflow is no more callable there than a rule +// +// Both have a working spelling, which is what makes this worth a diagnostic +// rather than a shrug: a microflow or Java action is `$r = CALL MICROFLOW …`, +// and a rule belongs in a decision (`if Module.SomeRule(…) then`). +// +// The one legal home for a bare qualified call is that decision condition, +// which mxcli serializes as a Microflows$RuleSplitCondition — so the IF +// condition is not walked here. Whether the name in that position resolves to a +// real rule needs the project, and is checked by the flow builder +// (tryBuildRuleSplitCondition) where the backend is at hand. + +// checkQualifiedCallInExpression flags a qualified call in a value position. +// label describes the site (e.g. "declare '$r'"), matching checkExprFunctions. +func (v *microflowValidator) checkQualifiedCallInExpression(label string, expr ast.Expression) { + for _, name := range qualifiedCallNames(expr) { + v.addViolation("MDL066", linter.SeverityError, + fmt.Sprintf("%s calls '%s(...)', but a Mendix expression cannot call a microflow, "+ + "rule or Java action — the build fails CE0117 \"Error(s) in expression\"", label, name), + fmt.Sprintf("Use an activity: $Result = CALL MICROFLOW %s (...); "+ + "a rule can only be called from a decision — if %s (...) then ... end if;", name, name)) + } +} + +// qualifiedCallNames returns the names of every qualified function call in expr, +// in source order. A name is qualified when it contains a '.', which is what +// separates `Module.Name(...)` from the built-in library — no built-in Mendix +// expression function has a dot in its name. +func qualifiedCallNames(expr ast.Expression) []string { + var out []string + var walk func(ast.Expression) + walk = func(e ast.Expression) { + switch n := e.(type) { + case *ast.FunctionCallExpr: + if strings.Contains(n.Name, ".") { + out = append(out, n.Name) + } + for _, arg := range n.Arguments { + walk(arg) + } + case *ast.BinaryExpr: + walk(n.Left) + walk(n.Right) + case *ast.UnaryExpr: + walk(n.Operand) + case *ast.ParenExpr: + walk(n.Inner) + case *ast.IfThenElseExpr: + walk(n.Condition) + walk(n.ThenExpr) + walk(n.ElseExpr) + case *ast.SourceExpr: + walk(n.Expression) + } + } + walk(expr) + return out +} diff --git a/mdl/executor/validate_qualified_call_test.go b/mdl/executor/validate_qualified_call_test.go new file mode 100644 index 000000000..37f9a4caa --- /dev/null +++ b/mdl/executor/validate_qualified_call_test.go @@ -0,0 +1,185 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/linter" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +func qualifiedCall(name string) ast.Expression { + return &ast.FunctionCallExpr{ + Name: name, + Arguments: []ast.Expression{&ast.BinaryExpr{ + Left: &ast.IdentifierExpr{Name: "IsActive"}, + Operator: "=", + Right: &ast.VariableExpr{Name: "IsActive"}, + }}, + } +} + +func mdl066(vs []linter.Violation) []linter.Violation { + var out []linter.Violation + for _, v := range vs { + if v.RuleID == "MDL066" { + out = append(out, v) + } + } + return out +} + +// A rule (or microflow, or Java action) call in a VALUE position is CE0117 +// "Error(s) in expression" on mxbuild — measured on 11.13.0 for both a rule and +// a microflow, and on both engines, because a Mendix expression has no +// user-callable functions at all. mxcli used to write the literal text and say +// nothing (upstream #939). +func TestQualifiedCallInValuePositionIsAnError(t *testing.T) { + stmt := &ast.CreateMicroflowStmt{ + Name: ast.QualifiedName{Module: "Sample", Name: "MF"}, + Body: []ast.MicroflowStatement{ + &ast.DeclareStmt{ + Variable: "Active", + Type: ast.DataType{Kind: ast.TypeBoolean}, + InitialValue: qualifiedCall("Sample.Rule_IsActive"), + }, + }, + } + vs := mdl066(ValidateMicroflow(stmt)) + if len(vs) != 1 { + t.Fatalf("got %d MDL066 violations, want 1: %+v", len(vs), vs) + } + if vs[0].Severity != linter.SeverityError { + t.Errorf("severity = %v, want error — the build fails", vs[0].Severity) + } + if !strings.Contains(vs[0].Message, "CE0117") || !strings.Contains(vs[0].Message, "Sample.Rule_IsActive") { + t.Errorf("message should name the call and the build error, got %q", vs[0].Message) + } + if !strings.Contains(vs[0].Suggestion, "CALL MICROFLOW") { + t.Errorf("suggestion should name the working spelling, got %q", vs[0].Suggestion) + } +} + +// The one position where a bare qualified call IS valid MDL: a decision, which +// mxcli stores as a Microflows$RuleSplitCondition. Flagging it here would reject +// the form the fix for #939 exists to write. +func TestQualifiedCallInIfConditionIsNotFlagged(t *testing.T) { + stmt := &ast.CreateMicroflowStmt{ + Name: ast.QualifiedName{Module: "Sample", Name: "MF"}, + Body: []ast.MicroflowStatement{ + &ast.IfStmt{ + Condition: qualifiedCall("Sample.Rule_IsActive"), + ThenBody: []ast.MicroflowStatement{&ast.ReturnStmt{}}, + ElseBody: []ast.MicroflowStatement{&ast.ReturnStmt{}}, + }, + }, + } + if vs := mdl066(ValidateMicroflow(stmt)); len(vs) != 0 { + t.Errorf("an if condition is the legal position for a rule call, got %+v", vs) + } +} + +// A while condition has no rule-split form — Mendix loops take a boolean +// expression or a list — so the same call there is not legal. The while body was +// not walked at all before, so this also covers value positions inside one. +func TestQualifiedCallInWhileIsAnError(t *testing.T) { + stmt := &ast.CreateMicroflowStmt{ + Name: ast.QualifiedName{Module: "Sample", Name: "MF"}, + Body: []ast.MicroflowStatement{ + &ast.WhileStmt{ + Condition: qualifiedCall("Sample.Rule_IsActive"), + Body: []ast.MicroflowStatement{ + &ast.DeclareStmt{ + Variable: "N", + Type: ast.DataType{Kind: ast.TypeInteger}, + InitialValue: qualifiedCall("Sample.MF_Callee"), + }, + }, + }, + }, + } + vs := mdl066(ValidateMicroflow(stmt)) + if len(vs) != 2 { + t.Fatalf("got %d MDL066 violations, want 2 (the condition and the body), %+v", len(vs), vs) + } +} + +// A built-in call is unqualified and must stay untouched — no Mendix expression +// function has a dot in its name, which is what makes the rule project-less. +func TestBuiltinCallIsNotFlagged(t *testing.T) { + stmt := &ast.CreateMicroflowStmt{ + Name: ast.QualifiedName{Module: "Sample", Name: "MF"}, + Body: []ast.MicroflowStatement{ + &ast.DeclareStmt{ + Variable: "N", + Type: ast.DataType{Kind: ast.TypeInteger}, + InitialValue: &ast.FunctionCallExpr{ + Name: "length", + Arguments: []ast.Expression{&ast.VariableExpr{Name: "Name"}}, + }, + }, + }, + } + if vs := mdl066(ValidateMicroflow(stmt)); len(vs) != 0 { + t.Errorf("length() is a built-in, got %+v", vs) + } +} + +// The decision position needs the project to judge: only a RULE can be called +// there. When the name resolves to something else the builder used to fall back +// to an ExpressionSplitCondition holding the call text — valid-looking MDL that +// is CE0117 on mxbuild — so it refuses instead. +func TestIfConditionCallingANonRuleIsRefused(t *testing.T) { + mb := &mock.MockBackend{ + IsRuleFunc: func(string) (bool, error) { return false, nil }, + } + fb := &flowBuilder{ + posX: 100, posY: 100, spacing: HorizontalSpacing, backend: mb, + varTypes: map[string]string{}, declaredVars: map[string]string{}, + } + fb.buildFlowGraph([]ast.MicroflowStatement{&ast.IfStmt{ + Condition: qualifiedCall("Sample.MF_Callee"), + ThenBody: []ast.MicroflowStatement{&ast.ReturnStmt{}}, + ElseBody: []ast.MicroflowStatement{&ast.ReturnStmt{}}, + }}, nil) + + errs := fb.GetErrors() + if len(errs) == 0 { + t.Fatal("a non-rule call in a decision was accepted; it is written as an " + + "expression and fails the build with CE0117") + } + if !strings.Contains(errs[0], "Sample.MF_Callee") || !strings.Contains(errs[0], "CE0117") { + t.Errorf("error should name the call and the build failure, got %q", errs[0]) + } + + // Control: the same shape with a real rule builds a RuleSplitCondition and + // reports nothing. + ok := &mock.MockBackend{IsRuleFunc: func(string) (bool, error) { return true, nil }} + fb2 := &flowBuilder{ + posX: 100, posY: 100, spacing: HorizontalSpacing, backend: ok, + varTypes: map[string]string{}, declaredVars: map[string]string{}, + } + fb2.buildFlowGraph([]ast.MicroflowStatement{&ast.IfStmt{ + Condition: qualifiedCall("Sample.Rule_IsActive"), + ThenBody: []ast.MicroflowStatement{&ast.ReturnStmt{}}, + ElseBody: []ast.MicroflowStatement{&ast.ReturnStmt{}}, + }}, nil) + if errs := fb2.GetErrors(); len(errs) != 0 { + t.Fatalf("a real rule call must be accepted, got %v", errs) + } + var found bool + for _, obj := range fb2.objects { + if sp, ok := obj.(*microflows.ExclusiveSplit); ok { + if _, isRule := sp.SplitCondition.(*microflows.RuleSplitCondition); isRule { + found = true + } + } + } + if !found { + t.Error("control: the rule call did not produce a RuleSplitCondition") + } +} diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index b02f7e293..1ab911a21 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -97,7 +97,7 @@ func ValidateWidgetPropertiesForStatement(stmt ast.Statement, registry *WidgetRe // validateWidgetTree recursively walks the AST widget tree and validates // pluggable widgets it encounters. func validateWidgetTree(widgets []*ast.WidgetV3, registry *WidgetRegistry, locationPrefix string) []linter.Violation { - return validateWidgetTreeIn(widgets, registry, locationPrefix, nil, nil) + return validateWidgetTreeIn(widgets, registry, locationPrefix, nil, nil, "", false) } // validateWidgetTreeIn is validateWidgetTree with the *parent* widget's @@ -108,7 +108,7 @@ func validateWidgetTree(widgets []*ast.WidgetV3, registry *WidgetRegistry, locat // must be exempt from the MDL-WIDGET07 "unrecognized property, silently dropped" // warning. When the parent mapping is known, the child's enumeration // sub-properties are validated against their member keys (MDL-WIDGET08). (9a) -func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, locationPrefix string, parentObjectLists map[string]*ObjectListMapping, parent *ast.WidgetV3) []linter.Violation { +func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, locationPrefix string, parentObjectLists map[string]*ObjectListMapping, parent *ast.WidgetV3, contextVar string, contextKnown bool) []linter.Violation { var out []linter.Violation for _, w := range widgets { if w == nil { @@ -126,6 +126,8 @@ func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, loc out = append(out, validateDynamicTextFormatting(w, locationPrefix)...) out = append(out, validateDatasourceXPathAssociationEmpty(w, locationPrefix)...) out = append(out, validateComboBoxAssociation(w, locationPrefix)...) + // A show_page argument naming anything but the context object is dropped. + out = append(out, validateShowPageArguments(w, contextVar, contextKnown, locationPrefix)...) // Unknown-property warning applies only to built-in widgets; pluggable // widgets get the stricter def.json check (MDL-WIDGET01) above, and // object-list items are validated by the object-list engine. @@ -146,7 +148,12 @@ func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, loc // Reported once per grid, not once per column — see the rule's comment. out = append(out, validateDataGrid2ColumnNames(w, locationPrefix)...) if len(w.Children) > 0 { - out = append(out, validateWidgetTreeIn(w.Children, registry, locationPrefix, objectListMappingSet(def), w)...) + // A data-bound widget renames the context object for everything below it. + childContextVar, childContextKnown := contextVar, contextKnown + if ds := w.GetDataSource(); ds != nil { + childContextVar, childContextKnown = contextVarFor(ds), true + } + out = append(out, validateWidgetTreeIn(w.Children, registry, locationPrefix, objectListMappingSet(def), w, childContextVar, childContextKnown)...) } } out = append(out, validateConsecutiveDynamicText(widgets, locationPrefix)...) diff --git a/mdl/executor/validate_workflow_refs.go b/mdl/executor/validate_workflow_refs.go index 8a5d02dc0..ab6f5ff18 100644 --- a/mdl/executor/validate_workflow_refs.go +++ b/mdl/executor/validate_workflow_refs.go @@ -66,3 +66,164 @@ func validateWorkflowParameterMappings(ctx *ExecContext, s *ast.CreateWorkflowSt }) return errs } + +// validateWorkflowReferences checks every qualified name a workflow's activities +// point at against the project (and against what the same script creates). +// +// This is the "missing-reference check" validateWorkflowParameterMappings above +// says it defers to, and which did not exist: a workflow calling a microflow +// that was nowhere in the project passed `check --references` with "All +// references valid" and was written by exec, leaving the Mendix validator +// (CE1613) as the only thing that noticed. The identical mistake inside a plain +// microflow body was caught, because validateFlowBodyReferences runs for +// microflows and nanoflows only (issue #943). +// +// The lookups and message wording deliberately match validateFlowBodyReferences, +// so the same mistake reads the same way wherever it is made. +func validateWorkflowReferences(ctx *ExecContext, activities []ast.WorkflowActivityNode, sc *scriptContext) []string { + if ctx == nil || !ctx.Connected() || len(activities) == 0 { + return nil + } + + // The lookups are built lazily: most workflows reference only microflows, and + // each build is a full backend list. + var microflowNames, workflowNames, pageNames map[string]bool + knownMicroflow := func(qn string) bool { + if microflowNames == nil { + microflowNames = buildMicroflowQualifiedNames(ctx) + } + return microflowNames[qn] || (sc != nil && sc.microflows[qn]) + } + knownWorkflow := func(qn string) bool { + if workflowNames == nil { + workflowNames = buildWorkflowQualifiedNames(ctx) + } + return workflowNames[qn] || (sc != nil && sc.workflows[qn]) + } + knownPage := func(qn string) bool { + if pageNames == nil { + pageNames = buildPageQualifiedNames(ctx) + } + return pageNames[qn] || (sc != nil && sc.pages[qn]) + } + + var errs []string + seen := map[string]bool{} // one report per distinct reference + report := func(kind, qn, via string) { + if qn == "" || seen[kind+qn] { + return + } + // A System.* target is provided by the runtime and never appears in the + // project, the same exemption validateFlowBodyReferences makes for Java + // actions. Reporting it would be a guaranteed false positive. + if isBuiltinModuleEntity(qualifiedNameModule(qn)) { + return + } + seen[kind+qn] = true + errs = append(errs, fmt.Sprintf("%s not found: %s (referenced by %s)", kind, qn, via)) + } + + walkWorkflowActivities(activities, func(act ast.WorkflowActivityNode) { + switch n := act.(type) { + case *ast.WorkflowCallMicroflowNode: + if qn := n.Microflow.String(); qn != "." && !knownMicroflow(qn) { + report("microflow", qn, "call microflow") + } + case *ast.WorkflowCallWorkflowNode: + if qn := n.Workflow.String(); qn != "." && !knownWorkflow(qn) { + report("workflow", qn, "call workflow") + } + case *ast.WorkflowUserTaskNode: + if qn := n.Page.String(); qn != "." && qn != "" && !knownPage(qn) { + report("page", qn, "user task page") + } + // Targeting by microflow, for users or for groups; the XPath variants + // carry no qualified name. + if qn := n.Targeting.Microflow.String(); qn != "." && qn != "" && !knownMicroflow(qn) { + report("microflow", qn, "user task targeting") + } + } + }) + return errs +} + +// validateWorkflowStatementRefs is the CREATE WORKFLOW entry point: the context +// entity and the module the workflow is being created in, plus every activity +// reference. +// +// The module check matters more than it looks. exec creates a module on demand, +// so without it a typo'd module name silently produced a new module rather than +// an error — `check` is the only thing standing in the way, and it did this for +// a microflow but not for a workflow. +func validateWorkflowStatementRefs(ctx *ExecContext, s *ast.CreateWorkflowStmt, sc *scriptContext) []string { + if ctx == nil || !ctx.Connected() { + return nil + } + var errs []string + if s.Name.Module != "" && (sc == nil || !sc.modules[s.Name.Module]) { + if _, err := findModule(ctx, s.Name.Module); err != nil { + errs = append(errs, fmt.Sprintf("module not found: %s", s.Name.Module)) + } + } + if qn := s.ParameterEntity.String(); qn != "" && qn != "." { + if !isBuiltinModuleEntity(s.ParameterEntity.Module) { + known := buildEntityQualifiedNames(ctx) + if !known[qn] && (sc == nil || !sc.entities[qn]) { + errs = append(errs, fmt.Sprintf("entity not found: %s (referenced by workflow parameter)", qn)) + } + } + } + return append(errs, validateWorkflowReferences(ctx, s.Activities, sc)...) +} + +// validateAlterWorkflowRefs validates ALTER WORKFLOW, which had no case in the +// validation switch at all and so fell through to "skip validation" — it got +// nothing, not even a check that the workflow it targets exists. +func validateAlterWorkflowRefs(ctx *ExecContext, s *ast.AlterWorkflowStmt, sc *scriptContext) []string { + if ctx == nil || !ctx.Connected() { + return nil + } + var errs []string + qn := s.Name.String() + if qn != "" && qn != "." && !isBuiltinModuleEntity(s.Name.Module) { + known := buildWorkflowQualifiedNames(ctx) + if !known[qn] && (sc == nil || !sc.workflows[qn]) { + errs = append(errs, fmt.Sprintf("workflow not found: %s", qn)) + } + } + + // Every op that introduces activities gets the same reference check a CREATE + // body gets. Ops that only remove or rename something introduce no reference. + var added []ast.WorkflowActivityNode + for _, op := range s.Operations { + switch o := op.(type) { + case *ast.InsertAfterOp: + added = append(added, o.NewActivity) + case *ast.ReplaceActivityOp: + added = append(added, o.NewActivity) + case *ast.InsertOutcomeOp: + added = append(added, o.Activities...) + case *ast.InsertPathOp: + added = append(added, o.Activities...) + case *ast.InsertBranchOp: + added = append(added, o.Activities...) + case *ast.InsertBoundaryEventOp: + added = append(added, o.Activities...) + case *ast.SetActivityPropertyOp: + // SET PAGE / SET TARGETING MICROFLOW name something that must exist. + if p := o.PageName.String(); p != "" && p != "." && !isBuiltinModuleEntity(o.PageName.Module) { + known := buildPageQualifiedNames(ctx) + if !known[p] && (sc == nil || !sc.pages[p]) { + errs = append(errs, fmt.Sprintf("page not found: %s (referenced by set activity page)", p)) + } + } + if m := o.Microflow.String(); m != "" && m != "." && !isBuiltinModuleEntity(o.Microflow.Module) { + known := buildMicroflowQualifiedNames(ctx) + if !known[m] && (sc == nil || !sc.microflows[m]) { + errs = append(errs, fmt.Sprintf("microflow not found: %s (referenced by set activity targeting)", m)) + } + } + } + } + return append(errs, validateWorkflowReferences(ctx, added, sc)...) +} diff --git a/mdl/executor/validate_workflow_refs_missing_test.go b/mdl/executor/validate_workflow_refs_missing_test.go new file mode 100644 index 000000000..2e71391a7 --- /dev/null +++ b/mdl/executor/validate_workflow_refs_missing_test.go @@ -0,0 +1,183 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" + "github.com/mendixlabs/mxcli/sdk/microflows" + "github.com/mendixlabs/mxcli/sdk/pages" + "github.com/mendixlabs/mxcli/sdk/workflows" +) + +// wfRefCtx returns a context for a project holding exactly one module M with one +// entity, one microflow, one page and one workflow — so anything else a script +// names has to be a genuine missing reference. +func wfRefCtx(t *testing.T) *ExecContext { + t.Helper() + mod := &model.Module{Name: "M"} + mod.ID = "mod1" + ent := &domainmodel.Entity{Name: "Ctx"} + ent.ContainerID = "dm1" + dm := &domainmodel.DomainModel{Entities: []*domainmodel.Entity{ent}} + dm.ID = "dm1" + dm.ContainerID = "mod1" + mf := µflows.Microflow{Name: "ACT_Real"} + mf.ContainerID = "mod1" + pg := &pages.Page{Name: "RealPage"} + pg.ContainerID = "mod1" + wf := &workflows.Workflow{Name: "RealWF"} + wf.ContainerID = "mod1" + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListDomainModelsFunc: func() ([]*domainmodel.DomainModel, error) { return []*domainmodel.DomainModel{dm}, nil }, + ListMicroflowsFunc: func() ([]*microflows.Microflow, error) { return []*microflows.Microflow{mf}, nil }, + ListPagesFunc: func() ([]*pages.Page, error) { return []*pages.Page{pg}, nil }, + ListWorkflowsFunc: func() ([]*workflows.Workflow, error) { return []*workflows.Workflow{wf}, nil }, + } + ctx, _ := newMockCtx(t, withBackend(mb)) + return ctx +} + +// checkScript runs the reference validation the way `check --references` does and +// returns the combined error text ("" when the script validates clean). +func checkScript(t *testing.T, ctx *ExecContext, src string) string { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + var out []string + sc := newScriptContext() + sc.collectDefinitions(prog) + for _, stmt := range prog.Statements { + if err := validateWithContext(ctx, stmt, sc); err != nil { + out = append(out, err.Error()) + } + } + return strings.Join(out, "\n") +} + +// Issue #943. validateWorkflowParameterMappings explicitly defers a target that +// is not in the project to "the missing-reference check", which was never +// written — so every reference inside a workflow went unvalidated while the same +// mistake inside a plain microflow was caught. +func TestWorkflowRefs_MissingReferencesAreReported(t *testing.T) { + cases := []struct{ name, src, want string }{ + { + "call microflow", + `create workflow M.W parameter $C: M.Ctx begin call microflow M.Nope; end workflow;`, + "M.Nope", + }, + { + "call workflow", + `create workflow M.W parameter $C: M.Ctx begin call workflow M.NopeWF; end workflow;`, + "M.NopeWF", + }, + { + "user task page", + `create workflow M.W parameter $C: M.Ctx begin user task T 'c' page M.NopePage outcomes 'A' { } 'B' { }; end workflow;`, + "M.NopePage", + }, + { + "targeting users microflow", + `create workflow M.W parameter $C: M.Ctx begin user task T 'c' page M.RealPage targeting users microflow M.NopeMF outcomes 'A' { } 'B' { }; end workflow;`, + "M.NopeMF", + }, + { + "context entity", + `create workflow M.W parameter $C: M.NopeEntity begin call microflow M.ACT_Real; end workflow;`, + "M.NopeEntity", + }, + { + "nested in an outcome", + `create workflow M.W parameter $C: M.Ctx begin user task T 'c' page M.RealPage outcomes 'A' { call microflow M.NestedNope; } 'B' { }; end workflow;`, + "M.NestedNope", + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + got := checkScript(t, wfRefCtx(t), c.src) + if !strings.Contains(got, c.want) { + t.Errorf("missing reference %s was not reported.\ngot: %s", c.want, got) + } + }) + } +} + +// The workflow's own module is checked for a microflow but was not for a +// workflow, so a typo'd module name reached exec and silently created one. +func TestWorkflowRefs_MissingModuleIsReported(t *testing.T) { + got := checkScript(t, wfRefCtx(t), + `create workflow NoSuchMod.W parameter $C: M.Ctx begin call microflow M.ACT_Real; end workflow;`) + if !strings.Contains(got, "NoSuchMod") { + t.Errorf("missing module was not reported.\ngot: %s", got) + } +} + +// ALTER WORKFLOW had no case in the switch at all, so it fell through to +// "skip validation" and got nothing — not even a check that the workflow exists. +func TestWorkflowRefs_AlterIsValidated(t *testing.T) { + cases := []struct{ name, src, want string }{ + { + "inserted activity's target", + `alter workflow M.RealWF insert after Step call microflow M.Nope;`, + "M.Nope", + }, + { + "replaced activity's target", + `alter workflow M.RealWF replace activity Step with call microflow M.Nope;`, + "M.Nope", + }, + { + "the workflow itself", + `alter workflow M.NoSuchWorkflow insert after Step call microflow M.ACT_Real;`, + "M.NoSuchWorkflow", + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + got := checkScript(t, wfRefCtx(t), c.src) + if !strings.Contains(got, c.want) { + t.Errorf("%s was not reported.\ngot: %s", c.want, got) + } + }) + } +} + +// Controls. Every reference below resolves, so the script must validate clean — +// a check that reports everything is as useless as one that reports nothing. +func TestWorkflowRefs_ValidReferencesPass(t *testing.T) { + cases := []struct{ name, src string }{ + {"all targets exist", `create workflow M.W parameter $C: M.Ctx begin user task T 'c' page M.RealPage targeting users microflow M.ACT_Real outcomes 'A' { call microflow M.ACT_Real; } 'B' { }; end workflow;`}, + {"call an existing workflow", `create workflow M.W parameter $C: M.Ctx begin call workflow M.RealWF; end workflow;`}, + {"alter an existing workflow", `alter workflow M.RealWF insert after Step call microflow M.ACT_Real;`}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + if got := checkScript(t, wfRefCtx(t), c.src); got != "" { + t.Errorf("valid references were reported as errors: %s", got) + } + }) + } +} + +// A workflow may target something the same script creates later. Those are not +// in the project yet, so the script context has to exempt them — including +// workflows, which it did not track at all. +func TestWorkflowRefs_CreatedInSameScriptIsExempt(t *testing.T) { + src := `create microflow M.ACT_New () begin return; end; +create page M.NewPage (title: 'p', layout: Atlas_Core.Atlas_Default) { dynamictext dt (content: 'x') } +create workflow M.W1 parameter $C: M.Ctx begin call microflow M.ACT_New; end workflow; +create workflow M.W2 parameter $C: M.Ctx begin call workflow M.W1; end workflow;` + if got := checkScript(t, wfRefCtx(t), src); got != "" { + t.Errorf("references created in the same script were reported: %s", got) + } +} diff --git a/mdl/executor/validate_workflow_rewrite.go b/mdl/executor/validate_workflow_rewrite.go new file mode 100644 index 000000000..f421071fd --- /dev/null +++ b/mdl/executor/validate_workflow_rewrite.go @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "sort" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/model" +) + +// checkNoDroppedWorkflowConstructs refuses a CREATE OR REPLACE/MODIFY WORKFLOW +// that would delete a stored construct the statement does not restate. +// +// A rewrite rebuilds the workflow from the statement, so anything the script +// does not mention is gone, and nothing signals the loss afterwards: measured on +// the v1 fixture, a stored interrupting timer boundary event and its whole +// handler flow went 1 -> 0 while exec reported "Created workflow" and exit 0 +// (issue #948). That is the same shape as a dropped queue binding +// (checkNoQueuedCalls) — guard-don't-drop, ADR-0005. +// +// Boundary events ARE authorable (`boundary event interrupting timer '…' { … }`), +// so a script that restates them is allowed straight through — that is the normal +// way to edit a workflow that has one. Event sub-processes are not authorable in +// MDL at all, so any stored one refuses the rewrite outright. +// +// The stored side is read from the raw unit rather than through the semantic +// model deliberately: the reader is what was blind here in the first place, and a +// guard that shares the reader's blind spot cannot see what it is meant to +// protect. Reading the BSON also covers constructs no engine models yet. +func checkNoDroppedWorkflowConstructs(ctx *ExecContext, workflowID model.ID, qualifiedName string, stmt *ast.CreateWorkflowStmt) error { + if ctx == nil || ctx.Backend == nil || workflowID == "" { + return nil + } + raw, err := ctx.Backend.GetRawUnit(workflowID) + if err != nil { + // An unreadable stored unit is not this guard's business; the rewrite + // path reports its own errors. + return nil + } + + if n := countRawWorkflowNodes(raw, "EventSubProcess"); n > 0 { + return mdlerrors.NewUnsupported(fmt.Sprintf( + "workflow %s has %d event sub-process(es), and MDL cannot express one — "+ + "rewriting the workflow would delete it.\n"+ + " Edit the workflow in Studio Pro, or use ALTER WORKFLOW to change one activity at a time.", + qualifiedName, n)) + } + + storedBE := countRawWorkflowNodes(raw, "BoundaryEvent") + if storedBE == 0 { + return nil + } + authored := countAuthoredBoundaryEvents(stmt.Activities) + if authored >= storedBE { + return nil + } + return mdlerrors.NewUnsupported(fmt.Sprintf( + "workflow %s has %d stored boundary event(s) but this statement declares %d — "+ + "rewriting it would delete the difference, along with each one's handler flow.\n"+ + " Restate them (`boundary event interrupting timer '' { … }`), which "+ + "`describe workflow %s` now emits, or use ALTER WORKFLOW to change one activity at a time.", + qualifiedName, storedBE, authored, qualifiedName)) +} + +// countRawWorkflowNodes counts BSON sub-documents whose $Type contains the given +// marker. Matching on a substring rather than an exact type is deliberate: the +// three timer boundary-event variants and the two event-sub-process start +// activities all differ by prefix, and a variant added later should be caught by +// the guard rather than slip past it. +func countRawWorkflowNodes(v any, marker string) int { + switch t := v.(type) { + case map[string]any: + n := 0 + if s, ok := t["$Type"].(string); ok && strings.Contains(s, marker) { + n++ + } + keys := make([]string, 0, len(t)) + for k := range t { + keys = append(keys, k) + } + sort.Strings(keys) // deterministic traversal; the count itself is order-free + for _, k := range keys { + if k == "$Type" { + continue + } + n += countRawWorkflowNodes(t[k], marker) + } + return n + case []any: + n := 0 + for _, e := range t { + n += countRawWorkflowNodes(e, marker) + } + return n + } + return 0 +} + +// countAuthoredBoundaryEvents counts the boundary events the statement declares, +// walking nested flows the same way the rest of workflow validation does. +func countAuthoredBoundaryEvents(activities []ast.WorkflowActivityNode) int { + n := 0 + walkWorkflowActivities(activities, func(act ast.WorkflowActivityNode) { + switch a := act.(type) { + case *ast.WorkflowUserTaskNode: + n += len(a.BoundaryEvents) + case *ast.WorkflowCallMicroflowNode: + n += len(a.BoundaryEvents) + case *ast.WorkflowWaitForNotificationNode: + n += len(a.BoundaryEvents) + } + }) + return n +} diff --git a/mdl/executor/validate_workflow_rewrite_test.go b/mdl/executor/validate_workflow_rewrite_test.go new file mode 100644 index 000000000..edf5cef61 --- /dev/null +++ b/mdl/executor/validate_workflow_rewrite_test.go @@ -0,0 +1,174 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" +) + +// storedWorkflowCtx returns a context whose raw stored workflow unit carries the +// given nested $Type nodes, so the guard has something to find. +func storedWorkflowCtx(t *testing.T, nested ...string) *ExecContext { + t.Helper() + acts := make([]any, 0, len(nested)) + for _, ty := range nested { + acts = append(acts, map[string]any{"$Type": ty}) + } + raw := map[string]any{ + "$Type": "Workflows$Workflow", + "Flow": map[string]any{ + "$Type": "Workflows$Flow", + "Activities": []any{ + map[string]any{ + "$Type": "Workflows$CallMicroflowTask", + "BoundaryEvents": acts, + }, + }, + }, + } + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + GetRawUnitFunc: func(model.ID) (map[string]any, error) { return raw, nil }, + } + ctx, _ := newMockCtx(t, withBackend(mb)) + return ctx +} + +func parseWorkflowStmt(t *testing.T, src string) *ast.CreateWorkflowStmt { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + s, ok := prog.Statements[0].(*ast.CreateWorkflowStmt) + if !ok { + t.Fatalf("statement is %T", prog.Statements[0]) + } + return s +} + +const wfNoBoundary = `create or replace workflow M.W parameter $C: M.Ctx +begin + call microflow M.ACT_Step comment 'Step'; +end workflow;` + +const wfWithBoundary = `create or replace workflow M.W parameter $C: M.Ctx +begin + call microflow M.ACT_Step comment 'Step' + boundary event interrupting timer 'addHours([%CurrentDateTime%], 2)' { + call microflow M.ACT_Escalate; + }; +end workflow;` + +// Issue #948. A rewrite rebuilds the workflow from the statement, so a stored +// boundary event the script does not restate is deleted along with its handler +// flow — measured 1 -> 0 while exec reported success. +func TestWorkflowRewrite_RefusesDroppingBoundaryEvent(t *testing.T) { + ctx := storedWorkflowCtx(t, "Workflows$InterruptingTimerBoundaryEvent") + err := checkNoDroppedWorkflowConstructs(ctx, "wf1", "M.W", parseWorkflowStmt(t, wfNoBoundary)) + if err == nil { + t.Fatal("a rewrite that drops a stored boundary event was allowed") + } + if !strings.Contains(err.Error(), "boundary event") { + t.Errorf("error should name the construct: %v", err) + } +} + +// Control: boundary events ARE authorable, so restating them is the normal way +// to edit such a workflow and must pass. +func TestWorkflowRewrite_AllowsRestatedBoundaryEvent(t *testing.T) { + ctx := storedWorkflowCtx(t, "Workflows$InterruptingTimerBoundaryEvent") + if err := checkNoDroppedWorkflowConstructs(ctx, "wf1", "M.W", parseWorkflowStmt(t, wfWithBoundary)); err != nil { + t.Errorf("a rewrite that restates the boundary event must be allowed: %v", err) + } +} + +// Control: a workflow with nothing to lose is never blocked. +func TestWorkflowRewrite_AllowsWhenNothingStored(t *testing.T) { + ctx := storedWorkflowCtx(t) + if err := checkNoDroppedWorkflowConstructs(ctx, "wf1", "M.W", parseWorkflowStmt(t, wfNoBoundary)); err != nil { + t.Errorf("rewrite of a workflow with no stored constructs must be allowed: %v", err) + } +} + +// An event sub-process cannot be expressed in MDL at all, so restating is not an +// option and any stored one refuses the rewrite outright. +func TestWorkflowRewrite_RefusesEventSubProcessUnconditionally(t *testing.T) { + ctx := storedWorkflowCtx(t, "Workflows$InterruptingNotificationEventSubProcessStartActivity") + for name, src := range map[string]string{"restated": wfWithBoundary, "not restated": wfNoBoundary} { + err := checkNoDroppedWorkflowConstructs(ctx, "wf1", "M.W", parseWorkflowStmt(t, src)) + if err == nil { + t.Errorf("%s: a stored event sub-process must refuse the rewrite", name) + } else if !strings.Contains(err.Error(), "event sub-process") { + t.Errorf("%s: error should name the construct: %v", name, err) + } + } +} + +// The guard must not depend on the semantic reader: that reader is what was +// blind here, and one sharing its blind spot cannot see what it protects. All +// three timer variants are matched by substring so a later one is caught too. +func TestWorkflowRewrite_MatchesEveryTimerVariant(t *testing.T) { + for _, ty := range []string{ + "Workflows$InterruptingTimerBoundaryEvent", + "Workflows$NonInterruptingTimerBoundaryEvent", + "Workflows$TimerBoundaryEvent", + "Workflows$SomeFutureBoundaryEvent", + } { + ctx := storedWorkflowCtx(t, ty) + if err := checkNoDroppedWorkflowConstructs(ctx, "wf1", "M.W", parseWorkflowStmt(t, wfNoBoundary)); err == nil { + t.Errorf("%s was not detected", ty) + } + } +} + +// An unreadable stored unit is not this guard's business — the rewrite path +// reports its own errors, and failing here would block edits on a bad read. +func TestWorkflowRewrite_UnreadableUnitDoesNotBlock(t *testing.T) { + mb := &mock.MockBackend{IsConnectedFunc: func() bool { return true }} + ctx, _ := newMockCtx(t, withBackend(mb)) + if err := checkNoDroppedWorkflowConstructs(ctx, "wf1", "M.W", parseWorkflowStmt(t, wfNoBoundary)); err != nil { + t.Errorf("an unreadable unit must not block the rewrite: %v", err) + } +} + +// DESCRIBE emits MDL that must re-parse. The call-microflow describer emitted +// boundary events BEFORE outcomes, which the grammar rejects +// (workflowCallMicroflowStmt: … OUTCOMES? BOUNDARY EVENT?) — "mismatched input +// 'outcomes' expecting ';'". It was invisible while the default engine could not +// read boundary events back at all. +func TestWorkflowDescribe_BoundaryEventAfterOutcomesReparses(t *testing.T) { + src := `create workflow M.W parameter $C: M.Ctx +begin + call microflow M.ACT_Step comment 'Step' + outcomes + DEFAULT -> { } + boundary event interrupting timer 'addHours([%CurrentDateTime%], 2)' { + call microflow M.ACT_Escalate; + }; +end workflow;` + if _, errs := visitor.Build(src); len(errs) > 0 { + t.Fatalf("outcomes-then-boundary-event must parse: %v", errs) + } + + // The order DESCRIBE used to emit: the grammar rejects it, which is what + // made the round trip fail. + bad := `create workflow M.W parameter $C: M.Ctx +begin + call microflow M.ACT_Step comment 'Step' + boundary event interrupting timer 'x' { } + outcomes + DEFAULT -> { }; +end workflow;` + if _, errs := visitor.Build(bad); len(errs) == 0 { + t.Error("boundary-event-then-outcomes should NOT parse; if the grammar now " + + "accepts both orders this test's premise is stale, but the describer " + + "should still emit the documented order") + } +} diff --git a/mdl/executor/widget_engine.go b/mdl/executor/widget_engine.go index fbd4f0278..c30ab33d0 100644 --- a/mdl/executor/widget_engine.go +++ b/mdl/executor/widget_engine.go @@ -261,7 +261,13 @@ func NewPluggableWidgetEngine(b backend.WidgetBuilderBackend, pb *pageBuilder) * func (e *PluggableWidgetEngine) Build(def *WidgetDefinition, w *ast.WidgetV3) (*pages.CustomWidget, error) { // Save and restore entity context (DataSource mappings may change it) oldEntityContext := e.pageBuilder.entityContext - defer func() { e.pageBuilder.entityContext = oldEntityContext }() + oldContextVar := e.pageBuilder.contextVarName + oldContextKnown := e.pageBuilder.contextKnown + defer func() { + e.pageBuilder.entityContext = oldEntityContext + e.pageBuilder.contextVarName = oldContextVar + e.pageBuilder.contextKnown = oldContextKnown + }() // Remember the containing context for properties that name members of it // rather than of this widget's own data. Saved/restored for nested widgets. @@ -319,6 +325,8 @@ func (e *PluggableWidgetEngine) Build(def *WidgetDefinition, w *ast.WidgetV3) (* builder.SetDataSource(propKey, dataSource) if entityName != "" { e.pageBuilder.entityContext = entityName + e.pageBuilder.contextVarName = contextVarFor(ds) + e.pageBuilder.contextKnown = true } break } @@ -718,6 +726,8 @@ func (e *PluggableWidgetEngine) resolveMapping(mapping PropertyMapping, w *ast.W ctx.EntityName = entityName if entityName != "" { e.pageBuilder.entityContext = entityName + e.pageBuilder.contextVarName = contextVarFor(ds) + e.pageBuilder.contextKnown = true if w.Name != "" { e.pageBuilder.paramEntityNames[w.Name] = entityName } diff --git a/mdl/exprcheck/ast.go b/mdl/exprcheck/ast.go index fa1ca19ba..04581b399 100644 --- a/mdl/exprcheck/ast.go +++ b/mdl/exprcheck/ast.go @@ -57,6 +57,14 @@ type CallExpr struct { baseNode Name string Args []RobustExpr + // Qualified marks a `Module.Name(...)` call. Mendix expressions have no + // user-callable functions, so such a call is never valid — but it IS valid + // MDL in a decision condition, where mxcli stores it as a rule split. The + // parser records it rather than rejecting it so the trailing `(` is consumed + // (an unconsumed one was reported as "Unexpected token after expression"), + // and the unknown-function check skips it: MDL066 owns that diagnostic and + // knows which positions are legal. + Qualified bool } type BinExpr struct { diff --git a/mdl/exprcheck/parser.go b/mdl/exprcheck/parser.go index aa2ffa859..6938db26e 100644 --- a/mdl/exprcheck/parser.go +++ b/mdl/exprcheck/parser.go @@ -384,11 +384,42 @@ func parseIdentLed(s *Stream, ctx Context) (RobustExpr, []Hint) { return &QNameExpr{baseNode: baseNode{P: t.Pos}, Module: name, Name: n2, Sub: n3}, nil } } + if s.Peek().Kind == TokLParen { + return parseQualifiedCall(s, ctx, t.Pos, name+"."+n2) + } return &QNameExpr{baseNode: baseNode{P: t.Pos}, Module: name, Name: n2}, nil } return &VariableExpr{baseNode: baseNode{P: t.Pos}, Name: name}, nil } +// parseQualifiedCall consumes the argument list of a `Module.Name(...)` call. +// Nothing in a Mendix expression can call one, so the node exists to keep the +// parse whole: without it the `(` was left on the stream and Parse reported +// "Unexpected token after expression … glued keywords such as 'emptyor'" with an +// empty location — on the valid decision form as much as the invalid ones, so it +// carried no signal and pointed at a typo that was not there (#939). +func parseQualifiedCall(s *Stream, ctx Context, pos Position, name string) (RobustExpr, []Hint) { + s.Consume() // '(' + var args []RobustExpr + var hs []Hint + if s.Peek().Kind != TokRParen { + for { + a, h := parseOr(s, ctx) + args = append(args, a) + hs = append(hs, h...) + if s.Peek().Kind == TokComma { + s.Consume() + continue + } + break + } + } + if s.Peek().Kind == TokRParen { + s.Consume() + } + return &CallExpr{baseNode: baseNode{P: pos}, Name: name, Args: args, Qualified: true}, hs +} + func parseDollar(s *Stream, ctx Context) (RobustExpr, []Hint) { t := s.Consume() name := strings.TrimPrefix(t.Text, "$") diff --git a/mdl/exprcheck/qualified_call_test.go b/mdl/exprcheck/qualified_call_test.go new file mode 100644 index 000000000..551e5dcb2 --- /dev/null +++ b/mdl/exprcheck/qualified_call_test.go @@ -0,0 +1,60 @@ +// SPDX-License-Identifier: Apache-2.0 + +package exprcheck + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/exprcheck/hints" +) + +// A qualified call left its `(` on the stream, and Parse reported the leftover as +// "Unexpected token after expression … glued keywords such as 'emptyor'" with an +// empty location. It fired on `mxcli check -p` for the VALID decision form as +// much as the invalid ones, so it carried no signal and pointed the author at a +// typo that was not there (upstream #939). +func TestQualifiedCallParsesWithoutTrailingTokenHint(t *testing.T) { + expr, hs := (&parserImpl{}).Parse("Sample.Rule_IsActive(IsActive = $IsActive)", Context{}) + for _, h := range hs { + if h.Problem != "" && h.Severity == hints.SeverityError { + t.Errorf("unexpected error hint: %s", h.Problem) + } + } + call, ok := expr.(*CallExpr) + if !ok { + t.Fatalf("parsed %T, want *CallExpr", expr) + } + if call.Name != "Sample.Rule_IsActive" { + t.Errorf("Name = %q, want Sample.Rule_IsActive", call.Name) + } + if !call.Qualified { + t.Error("Qualified = false; the unknown-function check relies on it to skip this call") + } + if len(call.Args) != 1 { + t.Errorf("Args = %d, want 1", len(call.Args)) + } +} + +// A bare qualified name (no parentheses) is an enumeration or entity reference +// and must still parse as one — the call handling must not swallow it. +func TestQualifiedNameWithoutCallIsStillAQName(t *testing.T) { + expr, _ := (&parserImpl{}).Parse("Sample.Status.Open", Context{}) + if _, ok := expr.(*QNameExpr); !ok { + t.Fatalf("parsed %T, want *QNameExpr", expr) + } +} + +// UnknownFunctionCalls feeds MDL044 ("not a Mendix expression function", with a +// did-you-mean). A qualified call is never a built-in, so a suggestion would be +// nonsense and the diagnostic would also fire on the legal decision form — +// MDL066 reports it instead, with the working spelling. +func TestUnknownFunctionCallsSkipsQualifiedCalls(t *testing.T) { + if refs := UnknownFunctionCalls("Sample.Rule_IsActive(IsActive = $IsActive)"); len(refs) != 0 { + t.Errorf("qualified call reported as an unknown function: %+v", refs) + } + // Control: an unqualified unknown is still reported. + refs := UnknownFunctionCalls("randomInt(1)") + if len(refs) != 1 || refs[0].Name != "randomInt" { + t.Errorf("unqualified unknown function not reported: %+v", refs) + } +} diff --git a/mdl/exprcheck/unknown_funcs.go b/mdl/exprcheck/unknown_funcs.go index faed5e82e..bf3097962 100644 --- a/mdl/exprcheck/unknown_funcs.go +++ b/mdl/exprcheck/unknown_funcs.go @@ -29,7 +29,9 @@ func UnknownFunctionCalls(src string) []FuncRef { } var out []FuncRef walkCalls(root, func(c *CallExpr) { - if c.Name == "" { + if c.Name == "" || c.Qualified { + // A qualified call is never a built-in, so "did you mean toString()?" + // would be nonsense; MDL066 reports it with the right fix. return } if _, ok := funcTable[c.Name]; ok { diff --git a/mdl/grammar/MDLParser.g4 b/mdl/grammar/MDLParser.g4 index 41a57e7fd..91bd6ccd3 100644 --- a/mdl/grammar/MDLParser.g4 +++ b/mdl/grammar/MDLParser.g4 @@ -128,6 +128,7 @@ createStatement | createKnowledgeBaseStatement | createAgentStatement | createNanoflowStatement + | createRuleStatement | createMenuStatement ) ; @@ -330,6 +331,7 @@ dropStatement | DROP CONSTANT qualifiedName | DROP MICROFLOW qualifiedName | DROP NANOFLOW qualifiedName + | DROP RULE qualifiedName | DROP PAGE qualifiedName | DROP SNIPPET qualifiedName | DROP MENU_KW qualifiedName @@ -429,6 +431,7 @@ moveDocumentType : PAGE | MICROFLOW | NANOFLOW + | RULE | SNIPPET | BUILDING BLOCK | LAYOUT diff --git a/mdl/grammar/domains/MDLCatalog.g4 b/mdl/grammar/domains/MDLCatalog.g4 index b24462e82..d471e96eb 100644 --- a/mdl/grammar/domains/MDLCatalog.g4 +++ b/mdl/grammar/domains/MDLCatalog.g4 @@ -21,6 +21,7 @@ showStatement | showOrList ASSOCIATIONS (IN (qualifiedName | IDENTIFIER))? | showOrList MICROFLOWS (IN (qualifiedName | IDENTIFIER))? | showOrList NANOFLOWS (IN (qualifiedName | IDENTIFIER))? + | showOrList RULES (IN (qualifiedName | IDENTIFIER))? | showOrList WORKFLOWS (IN (qualifiedName | IDENTIFIER))? | showOrList PAGES (IN (qualifiedName | IDENTIFIER))? | showOrList SNIPPETS (IN (qualifiedName | IDENTIFIER))? @@ -150,6 +151,7 @@ describeStatement | DESCRIBE ASSOCIATION qualifiedName | DESCRIBE MICROFLOW qualifiedName | DESCRIBE NANOFLOW qualifiedName + | DESCRIBE RULE qualifiedName | DESCRIBE WORKFLOW qualifiedName | DESCRIBE PAGE qualifiedName | DESCRIBE SNIPPET qualifiedName @@ -218,6 +220,7 @@ catalogTableName | ASSOCIATIONS // keyword token — must be listed explicitly | MICROFLOWS | NANOFLOWS + | RULES | PAGES | SNIPPETS | LAYOUTS diff --git a/mdl/grammar/domains/MDLMicroflow.g4 b/mdl/grammar/domains/MDLMicroflow.g4 index fc83efb68..1cc8e3818 100644 --- a/mdl/grammar/domains/MDLMicroflow.g4 +++ b/mdl/grammar/domains/MDLMicroflow.g4 @@ -32,6 +32,22 @@ createNanoflowStatement BEGIN microflowBody END SEMICOLON? SLASH? ; +/** + * Rule creation — Mendix calls a rule "a special kind of microflow" that returns + * a Boolean or an enumeration and may only be used from a decision. It shares the + * microflow body, so this mirrors createNanoflowStatement; what a rule may not do + * is enforced by the validator, not by the grammar, because restricting the body + * rule here would duplicate it and turn every violation into a parse error with + * no explanation. + */ +createRuleStatement + : RULE qualifiedName + LPAREN microflowParameterList? RPAREN + microflowReturnType? + microflowOptions? + BEGIN microflowBody END SEMICOLON? SLASH? + ; + /** * Java Action creation with inline Java source code. */ diff --git a/mdl/grammar/domains/MDLSecurity.g4 b/mdl/grammar/domains/MDLSecurity.g4 index a49965547..7b7ab5eaa 100644 --- a/mdl/grammar/domains/MDLSecurity.g4 +++ b/mdl/grammar/domains/MDLSecurity.g4 @@ -98,6 +98,12 @@ revokePublishedRestServiceAccessStatement alterProjectSecurityStatement : ALTER PROJECT SECURITY LEVEL (PRODUCTION | PROTOTYPE | OFF) | ALTER PROJECT SECURITY DEMO USERS (ON | OFF) + // ROLE is optional here but effectively required by Mendix: mxbuild raises + // CE0133 when guest access is on with no role. It is optional so that + // re-enabling a project that already stores one does not force a retype; + // the executor refuses ON when neither source supplies a role. + | ALTER PROJECT SECURITY GUEST ACCESS ON (ROLE identifierOrKeyword)? + | ALTER PROJECT SECURITY GUEST ACCESS OFF ; createDemoUserStatement diff --git a/mdl/visitor/visitor_delete_behavior_test.go b/mdl/visitor/visitor_delete_behavior_test.go new file mode 100644 index 000000000..517a0d11f --- /dev/null +++ b/mdl/visitor/visitor_delete_behavior_test.go @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// upstream #901. `DELETE_BEHAVIOR PREVENT` parsed, reported "Modified +// association", and stored DeleteMeButKeepReferences — overwriting whatever the +// association had before, which for the reporter was DELETE_CASCADE. +// +// The grammar and the generated parser were never at fault: DeleteBehaviorContext +// exposes an accessor for all five tokens (mdl_parser.go, DELETE_AND_REFERENCES / +// DELETE_BUT_KEEP_REFERENCES / DELETE_IF_NO_REFERENCES / CASCADE / PREVENT). +// buildDeleteBehavior called CASCADE() and nothing else, so the other four fell +// through to the zero value, ast.DeleteKeepReferences — a legal behaviour, which +// is why nothing downstream could tell it had been substituted. +// +// The table is exhaustive on purpose. A test for PREVENT alone passes against a +// fix that maps PREVENT and leaves DELETE_AND_REFERENCES — the canonical spelling +// of cascade, whose alias CASCADE works — still silently downgraded. The intended +// mapping is not invented here; it is the one `mxcli syntax +// domain-model.association.delete-behavior` has always printed. +func TestBuildDeleteBehavior_EverySpelling(t *testing.T) { + cases := []struct { + token string + want ast.DeleteBehavior + }{ + {"DELETE_BUT_KEEP_REFERENCES", ast.DeleteKeepReferences}, + {"DELETE_AND_REFERENCES", ast.DeleteCascade}, + {"CASCADE", ast.DeleteCascade}, + {"DELETE_IF_NO_REFERENCES", ast.DeleteIfNoReferences}, + {"PREVENT", ast.DeleteIfNoReferences}, + } + + for _, tc := range cases { + t.Run("create/"+tc.token, func(t *testing.T) { + stmt := parseCreateAssoc(t, `CREATE ASSOCIATION M.C_P FROM M.C TO M.P TYPE Reference DELETE_BEHAVIOR `+tc.token+`;`) + if stmt.DeleteBehavior != tc.want { + t.Errorf("DELETE_BEHAVIOR %s = %v, want %v", tc.token, stmt.DeleteBehavior, tc.want) + } + }) + + t.Run("alter/"+tc.token, func(t *testing.T) { + stmt := parseAlterAssoc(t, `ALTER ASSOCIATION M.C_P SET DELETE_BEHAVIOR `+tc.token+`;`) + if stmt.Operation != ast.AlterAssociationSetDeleteBehavior { + t.Fatalf("operation = %v, want AlterAssociationSetDeleteBehavior", stmt.Operation) + } + if stmt.DeleteBehavior != tc.want { + t.Errorf("SET DELETE_BEHAVIOR %s = %v, want %v", tc.token, stmt.DeleteBehavior, tc.want) + } + }) + } +} + +// The control for the table above: an association that says nothing about delete +// behaviour must still land on Mendix's default. Without this, "map every token" +// could be satisfied by a change that broke the unspecified case. +func TestBuildDeleteBehavior_OmittedIsKeepReferences(t *testing.T) { + stmt := parseCreateAssoc(t, `CREATE ASSOCIATION M.C_P FROM M.C TO M.P TYPE Reference;`) + if stmt.DeleteBehavior != ast.DeleteKeepReferences { + t.Errorf("omitted DELETE_BEHAVIOR = %v, want DeleteKeepReferences", stmt.DeleteBehavior) + } +} + +func parseCreateAssoc(t *testing.T, src string) *ast.CreateAssociationStmt { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse %q: %v", src, errs) + } + stmt, ok := prog.Statements[0].(*ast.CreateAssociationStmt) + if !ok { + t.Fatalf("got %T, want *ast.CreateAssociationStmt", prog.Statements[0]) + } + return stmt +} + +func parseAlterAssoc(t *testing.T, src string) *ast.AlterAssociationStmt { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse %q: %v", src, errs) + } + stmt, ok := prog.Statements[0].(*ast.AlterAssociationStmt) + if !ok { + t.Fatalf("got %T, want *ast.AlterAssociationStmt", prog.Statements[0]) + } + return stmt +} diff --git a/mdl/visitor/visitor_entity.go b/mdl/visitor/visitor_entity.go index 645a0fd38..1a619b031 100644 --- a/mdl/visitor/visitor_entity.go +++ b/mdl/visitor/visitor_entity.go @@ -805,6 +805,10 @@ func (b *Builder) ExitDropStatement(ctx *parser.DropStatementContext) { b.statements = append(b.statements, &ast.DropNanoflowStmt{ Name: buildQualifiedName(names[0]), }) + } else if ctx.RULE() != nil { + b.statements = append(b.statements, &ast.DropRuleStmt{ + Name: buildQualifiedName(names[0]), + }) } else if ctx.PAGE() != nil { b.statements = append(b.statements, &ast.DropPageStmt{ Name: buildQualifiedName(names[0]), diff --git a/mdl/visitor/visitor_helpers.go b/mdl/visitor/visitor_helpers.go index 3b5ff0976..fe7adbf48 100644 --- a/mdl/visitor/visitor_helpers.go +++ b/mdl/visitor/visitor_helpers.go @@ -419,13 +419,22 @@ func buildDeleteBehavior(ctx parser.IDeleteBehaviorContext) ast.DeleteBehavior { } db := ctx.(*parser.DeleteBehaviorContext) - if db.CASCADE() != nil { + // Every token the deleteBehavior rule admits has to be read here. Only + // CASCADE was, so PREVENT, DELETE_IF_NO_REFERENCES and DELETE_AND_REFERENCES + // all fell through to the zero value — a legal behaviour, which is why the + // substitution was silent all the way to disk and overwrote whatever the + // association had (upstream #901). Adding a token to the grammar means + // adding it here; there is no default that can be trusted to be right. + switch { + case db.CASCADE() != nil, db.DELETE_AND_REFERENCES() != nil: return ast.DeleteCascade + case db.PREVENT() != nil, db.DELETE_IF_NO_REFERENCES() != nil: + return ast.DeleteIfNoReferences + default: + // DELETE_BUT_KEEP_REFERENCES, and Mendix's default for an association + // that names no behaviour at all. + return ast.DeleteKeepReferences } - // The new grammar may use different tokens for delete behaviors - // Add more cases as needed based on the actual grammar - - return ast.DeleteKeepReferences } // QuoteString escapes s for safe embedding inside an MDL single-quoted string diff --git a/mdl/visitor/visitor_microflow.go b/mdl/visitor/visitor_microflow.go index 36fbb0406..fcd0b7f90 100644 --- a/mdl/visitor/visitor_microflow.go +++ b/mdl/visitor/visitor_microflow.go @@ -114,6 +114,51 @@ func (b *Builder) ExitCreateNanoflowStatement(ctx *parser.CreateNanoflowStatemen b.statements = append(b.statements, stmt) } +// ExitCreateRuleStatement mirrors ExitCreateNanoflowStatement — a rule shares the +// microflow header and body grammar, so the only difference is the AST node. +func (b *Builder) ExitCreateRuleStatement(ctx *parser.CreateRuleStatementContext) { + stmt := &ast.CreateRuleStmt{ + Name: buildQualifiedName(ctx.QualifiedName()), + } + + if paramList := ctx.MicroflowParameterList(); paramList != nil { + stmt.Parameters = buildMicroflowParameters(paramList) + } + if retType := ctx.MicroflowReturnType(); retType != nil { + stmt.ReturnType = buildMicroflowReturnType(retType) + } + if opts := ctx.MicroflowOptions(); opts != nil { + optsCtx := opts.(*parser.MicroflowOptionsContext) + for _, opt := range optsCtx.AllMicroflowOption() { + optCtx := opt.(*parser.MicroflowOptionContext) + if optCtx.COMMENT() != nil && optCtx.STRING_LITERAL() != nil { + stmt.Comment = unquoteString(optCtx.STRING_LITERAL().GetText()) + } + if optCtx.FOLDER() != nil && optCtx.STRING_LITERAL() != nil { + stmt.Folder = unquoteString(optCtx.STRING_LITERAL().GetText()) + } + } + } + if body := ctx.MicroflowBody(); body != nil { + stmt.Body = buildMicroflowBody(body) + } + + if createStmt := findParentCreateStatement(ctx); createStmt != nil { + if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { + stmt.CreateOrModify = true + } + for _, ann := range createStmt.AllAnnotation() { + annCtx := ann.(*parser.AnnotationContext) + if strings.EqualFold(annCtx.AnnotationName().GetText(), "excluded") { + stmt.Excluded = true + } + } + } + stmt.Documentation = findDocCommentText(ctx) + + b.statements = append(b.statements, stmt) +} + // buildMicroflowDataType converts a data type context to ast.DataType for microflow context. // In microflow parameters/return types, bare qualified names are entity references (not enumerations). func buildMicroflowDataType(ctx parser.IDataTypeContext) ast.DataType { diff --git a/mdl/visitor/visitor_query.go b/mdl/visitor/visitor_query.go index 5b384ce42..07b36f5e2 100644 --- a/mdl/visitor/visitor_query.go +++ b/mdl/visitor/visitor_query.go @@ -169,6 +169,16 @@ func (b *Builder) ExitShowStatement(ctx *parser.ShowStatementContext) { } } b.statements = append(b.statements, stmt) + } else if ctx.RULES() != nil { + stmt := &ast.ShowStmt{ObjectType: ast.ShowRules} + if ctx.IN() != nil { + if qn := ctx.QualifiedName(); qn != nil { + stmt.InModule = getQualifiedNameText(qn) + } else if id := ctx.IDENTIFIER(); id != nil { + stmt.InModule = id.GetText() + } + } + b.statements = append(b.statements, stmt) } else if ctx.WORKFLOWS() != nil { stmt := &ast.ShowStmt{ObjectType: ast.ShowWorkflows} if ctx.IN() != nil { @@ -1084,6 +1094,11 @@ func (b *Builder) ExitDescribeStatement(ctx *parser.DescribeStatementContext) { ObjectType: ast.DescribeNanoflow, Name: name, }) + } else if ctx.RULE() != nil { + b.statements = append(b.statements, &ast.DescribeStmt{ + ObjectType: ast.DescribeRule, + Name: name, + }) } else if ctx.WORKFLOW() != nil { b.statements = append(b.statements, &ast.DescribeStmt{ ObjectType: ast.DescribeWorkflow, diff --git a/mdl/visitor/visitor_security.go b/mdl/visitor/visitor_security.go index b09cb7213..cc0e0af77 100644 --- a/mdl/visitor/visitor_security.go +++ b/mdl/visitor/visitor_security.go @@ -416,6 +416,12 @@ func (b *Builder) ExitAlterProjectSecurityStatement(ctx *parser.AlterProjectSecu } else if ctx.DEMO() != nil { enabled := ctx.ON() != nil stmt.DemoUsersEnabled = &enabled + } else if ctx.GUEST() != nil { + enabled := ctx.ON() != nil + stmt.GuestAccessEnabled = &enabled + if roleCtx := ctx.IdentifierOrKeyword(); roleCtx != nil { + stmt.GuestUserRole = unquoteIdentifier(roleCtx.GetText()) + } } b.statements = append(b.statements, stmt) diff --git a/mdl/visitor/visitor_security_test.go b/mdl/visitor/visitor_security_test.go index 7e388557e..2b98caac8 100644 --- a/mdl/visitor/visitor_security_test.go +++ b/mdl/visitor/visitor_security_test.go @@ -650,3 +650,47 @@ func TestUpdateSecurity_InModule(t *testing.T) { t.Errorf("Expected MyModule, got %q", stmt.Module) } } + +func TestAlterProjectSecurity_GuestAccess(t *testing.T) { + cases := []struct { + input string + wantOn bool + wantRole string + }{ + {`ALTER PROJECT SECURITY GUEST ACCESS ON ROLE Anonymous;`, true, "Anonymous"}, + {`ALTER PROJECT SECURITY GUEST ACCESS ON ROLE "Guest User";`, true, "Guest User"}, + // Bare ON is legal to parse — the executor decides whether a stored role + // makes it valid, since re-enabling should not force a retype. + {`ALTER PROJECT SECURITY GUEST ACCESS ON;`, true, ""}, + {`ALTER PROJECT SECURITY GUEST ACCESS OFF;`, false, ""}, + } + for _, tc := range cases { + t.Run(tc.input, func(t *testing.T) { + prog, errs := Build(tc.input) + if len(errs) > 0 { + t.Fatalf("Parse error: %v", errs[0]) + } + stmt, ok := prog.Statements[0].(*ast.AlterProjectSecurityStmt) + if !ok { + t.Fatalf("Expected AlterProjectSecurityStmt, got %T", prog.Statements[0]) + } + if stmt.GuestAccessEnabled == nil { + t.Fatal("GuestAccessEnabled is nil — the guest branch did not fire") + } + if *stmt.GuestAccessEnabled != tc.wantOn { + t.Errorf("GuestAccessEnabled = %v, want %v", *stmt.GuestAccessEnabled, tc.wantOn) + } + if stmt.GuestUserRole != tc.wantRole { + t.Errorf("GuestUserRole = %q, want %q", stmt.GuestUserRole, tc.wantRole) + } + // The other two ALTER PROJECT SECURITY forms share this node; a + // guest statement must not look like a level or demo-user one. + if stmt.SecurityLevel != "" { + t.Errorf("SecurityLevel = %q, want empty", stmt.SecurityLevel) + } + if stmt.DemoUsersEnabled != nil { + t.Errorf("DemoUsersEnabled = %v, want nil", *stmt.DemoUsersEnabled) + } + }) + } +} diff --git a/modelsdk/canon/identity.go b/modelsdk/canon/identity.go index 910e597e3..3de28c502 100644 --- a/modelsdk/canon/identity.go +++ b/modelsdk/canon/identity.go @@ -47,6 +47,12 @@ func Reconcile(contents, stored []byte) (out []byte, unchanged bool) { // reference with them. contents = TransplantIDs(contents, stored) + // And the nested identity property the transplant does not cover: every + // Workflows$* element carries a PersistentId that both engines re-mint on + // every write, so without this a workflow document never equals itself and + // no-op elision could never fire for one (issue #949). + contents = CarryPersistentIDs(contents, stored) + if alwaysWrite() { return contents, false } diff --git a/modelsdk/canon/persistentid.go b/modelsdk/canon/persistentid.go new file mode 100644 index 000000000..99f84eeaa --- /dev/null +++ b/modelsdk/canon/persistentid.go @@ -0,0 +1,185 @@ +// SPDX-License-Identifier: Apache-2.0 + +package canon + +import ( + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/x/bsonx/bsoncore" +) + +// persistentIDKey is the sole nested identity property carried here. +// +// Every Workflows$* element — activities, outcomes, boundary events, and the +// workflow document itself — stores one, and both engines mint a fresh GUID for +// it on every write (modelsdk's addFreshPersistentID, legacy's +// idToBsonBinary(generateUUID())). Neither reads the stored value back, so a +// workflow document never equals itself: no-op elision could not fire, and an +// ALTER that changed nothing still produced a version-control diff (issue #949). +const persistentIDKey = "PersistentId" + +// CarryPersistentIDs returns contents with each element's PersistentId replaced +// by the value the corresponding stored element already holds. +// +// This is the nested-element counterpart to CarryIdentity, which only reaches +// top-level properties of the document root — PersistentId lives on elements +// arbitrarily deep in the flow tree, so the correspondence has to be established +// structurally. It reuses the pairing TransplantIDs is built on: two elements +// correspond when they agree on $Type and align within their list, and only +// paired elements exchange anything. +// +// The same three properties that make the $ID transplant safe hold here: +// +// - The patch is in place over fixed-width 16-byte binaries, so no length +// prefix can be disturbed and nothing is re-marshalled. +// - The mapping is injective — a stored PersistentId is handed out at most +// once — so two elements cannot end up sharing one. +// - It moves the written document *towards* what is already on disk, which is +// what eliding the write would have done implicitly. +// +// Unlike an $ID, a PersistentId is not a pointer target: nothing in the document +// references it, so substituting one by value touches exactly its own element. +// That is what lets this run as a plain value substitution rather than needing +// the reference-rewriting argument TransplantIDs makes. +// +// A document that cannot be read on either side passes through untouched. +func CarryPersistentIDs(contents, stored []byte) []byte { + m := persistentIDMapping(contents, stored) + if len(m) == 0 { + return contents + } + out := append([]byte(nil), contents...) + if !patchDocument(bsoncore.Document(out), m) { + return contents + } + // A duplicate is the one real failure: two elements claiming one identity is + // worse than two fresh ones. Verify rather than trust the mapping. + if hasDuplicatePersistentID(out) { + return contents + } + return out +} + +// persistentIDMapping pairs the two documents structurally and returns +// new PersistentId → stored PersistentId for every element that corresponds. +func persistentIDMapping(contents, stored []byte) map[string][]byte { + var newDoc, oldDoc bson.D + if err := bson.Unmarshal(contents, &newDoc); err != nil { + return nil + } + if err := bson.Unmarshal(stored, &oldDoc); err != nil { + return nil + } + p := &persistPairer{pairs: map[string][]byte{}, claimed: map[string]bool{}} + p.pairValue(newDoc, oldDoc) + return p.pairs +} + +// persistPairer mirrors pairer, but records the PersistentId property instead of +// the element $ID. The traversal is deliberately identical: a divergence between +// the two would mean the $ID and the PersistentId of one element could be taken +// from two *different* stored elements. +type persistPairer struct { + pairs map[string][]byte + claimed map[string]bool +} + +func (p *persistPairer) record(newID, oldID string, oldData []byte) { + if _, seen := p.pairs[newID]; seen { + return + } + if p.claimed[oldID] { + return + } + p.claimed[oldID] = true + p.pairs[newID] = oldData +} + +func (p *persistPairer) pairValue(newV, oldV any) { + if nd, ok := asDoc(newV); ok { + od, ok := asDoc(oldV) + if !ok { + return + } + p.pairDoc(nd, od) + return + } + if news, ok := asSlice(newV); ok { + olds, ok := asSlice(oldV) + if !ok { + return + } + for _, pair := range alignSlices(news, olds) { + p.pairValue(news[pair[0]], olds[pair[1]]) + } + } +} + +// pairDoc corresponds two elements. As in pairer, a differing $Type means these +// are not the same element, so nothing is carried from it or from anything under +// it — inheriting an identity there would claim a replacement was an edit. +func (p *persistPairer) pairDoc(nd, od map[string]any) { + if typeName(nd) != typeName(od) { + return + } + if nb, ok := binary16(nd, persistentIDKey); ok { + if ob, ok := binary16(od, persistentIDKey); ok { + p.record(blobToUUID(nb), blobToUUID(ob), ob) + } + } + for _, k := range sortedKeys(nd) { + if ov, ok := od[k]; ok { + p.pairValue(nd[k], ov) + } + } +} + +// binary16 reads a fixed-width 16-byte binary property. Any other width or type +// is left alone: the patch is in place, so only an equal-length value is safe. +func binary16(d map[string]any, key string) ([]byte, bool) { + b, ok := d[key].(bson.Binary) + if !ok || len(b.Data) != 16 { + return nil, false + } + return b.Data, true +} + +// hasDuplicatePersistentID reports whether any two elements share a PersistentId. +func hasDuplicatePersistentID(raw []byte) bool { + seen := map[string]bool{} + dup := false + var walk func(doc bsoncore.Document) + walk = func(doc bsoncore.Document) { + elems, err := doc.Elements() + if err != nil { + return + } + for _, e := range elems { + v := e.Value() + switch v.Type { + case bsoncore.TypeBinary: + if e.Key() != persistentIDKey { + continue + } + _, data, ok := v.BinaryOK() + if !ok || len(data) != 16 { + continue + } + id := blobToUUID(data) + if seen[id] { + dup = true + } + seen[id] = true + case bsoncore.TypeEmbeddedDocument: + if sub, ok := v.DocumentOK(); ok { + walk(sub) + } + case bsoncore.TypeArray: + if arr, ok := v.ArrayOK(); ok { + walk(bsoncore.Document(arr)) + } + } + } + } + walk(bsoncore.Document(raw)) + return dup +} diff --git a/modelsdk/canon/persistentid_test.go b/modelsdk/canon/persistentid_test.go new file mode 100644 index 000000000..66ebecae5 --- /dev/null +++ b/modelsdk/canon/persistentid_test.go @@ -0,0 +1,190 @@ +// SPDX-License-Identifier: Apache-2.0 + +package canon + +import ( + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// wfDoc models a workflow flow with two activities, each carrying the +// PersistentId every Workflows$* element has. pidA/pidB are the identities; the +// captions are the content, so "same document, one value changed" is expressible. +func wfDoc(t *testing.T, idA, pidA, idB, pidB byte, capA, capB string) []byte { + t.Helper() + return marshal(t, bson.D{ + {Key: "$Type", Value: "Workflows$Workflow"}, + {Key: "$ID", Value: bin(1)}, + {Key: "PersistentId", Value: bin(2)}, + {Key: "Flow", Value: bson.D{ + {Key: "$Type", Value: "Workflows$Flow"}, + {Key: "$ID", Value: bin(3)}, + {Key: "Activities", Value: bson.A{ + int32(3), // typed-array marker + bson.D{ + {Key: "$Type", Value: "Workflows$SingleUserTaskActivity"}, + {Key: "$ID", Value: bin(idA)}, + {Key: "PersistentId", Value: bin(pidA)}, + {Key: "Name", Value: "TaskA"}, + {Key: "Caption", Value: capA}, + }, + bson.D{ + {Key: "$Type", Value: "Workflows$SingleUserTaskActivity"}, + {Key: "$ID", Value: bin(idB)}, + {Key: "PersistentId", Value: bin(pidB)}, + {Key: "Name", Value: "TaskB"}, + {Key: "Caption", Value: capB}, + }, + }}, + }}, + }) +} + +func persistentIDs(t *testing.T, raw []byte) []string { + t.Helper() + var doc bson.D + if err := bson.Unmarshal(raw, &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + var out []string + var walk func(v any) + walk = func(v any) { + if d, ok := asDoc(v); ok { + if b, ok := binary16(d, "PersistentId"); ok { + out = append(out, blobToUUID(b)) + } + for _, k := range sortedKeys(d) { + walk(d[k]) + } + return + } + if s, ok := asSlice(v); ok { + for _, e := range s { + walk(e) + } + } + } + walk(doc) + return out +} + +// Issue #949. Both engines mint a fresh PersistentId on every activity write, so +// a rebuilt workflow never equalled the stored one and no-op elision could not +// fire. The stored values must be carried onto the elements that correspond. +func TestCarryPersistentIDs_CarriesStoredValues(t *testing.T) { + stored := wfDoc(t, 10, 20, 11, 21, "A", "B") + rebuilt := wfDoc(t, 90, 91, 92, 93, "A", "B") // every identity re-minted + + got := persistentIDs(t, CarryPersistentIDs(rebuilt, stored)) + want := persistentIDs(t, stored) + if len(got) != len(want) { + t.Fatalf("got %d PersistentIds, want %d", len(got), len(want)) + } + for i := range want { + if got[i] != want[i] { + t.Errorf("PersistentId[%d] = %s, want %s", i, got[i], want[i]) + } + } +} + +// The point of carrying them: a rebuild of an unchanged workflow must compare +// equal, so the write is elided. +func TestReconcile_NoOpWorkflowRebuildIsElided(t *testing.T) { + stored := wfDoc(t, 10, 20, 11, 21, "A", "B") + rebuilt := wfDoc(t, 90, 91, 92, 93, "A", "B") + + _, unchanged := Reconcile(rebuilt, stored) + if !unchanged { + t.Error("a rebuild that changed nothing was not recognised as unchanged") + } +} + +// Control: a real edit must still be seen as a change, or the carry has turned +// into a way of losing writes. +func TestReconcile_ChangedWorkflowStillWrites(t *testing.T) { + stored := wfDoc(t, 10, 20, 11, 21, "A", "B") + rebuilt := wfDoc(t, 90, 91, 92, 93, "A", "CHANGED") + + if _, unchanged := Reconcile(rebuilt, stored); unchanged { + t.Error("a caption change was elided") + } +} + +// A differing $Type means the element was replaced, not edited — inheriting its +// identity would claim otherwise. +func TestCarryPersistentIDs_DoesNotCrossTypes(t *testing.T) { + stored := wfDoc(t, 10, 20, 11, 21, "A", "B") + rebuilt := marshal(t, bson.D{ + {Key: "$Type", Value: "Workflows$Workflow"}, + {Key: "$ID", Value: bin(1)}, + {Key: "PersistentId", Value: bin(2)}, + {Key: "Flow", Value: bson.D{ + {Key: "$Type", Value: "Workflows$Flow"}, + {Key: "$ID", Value: bin(3)}, + {Key: "Activities", Value: bson.A{ + int32(3), + bson.D{ // a different activity kind at the same position + {Key: "$Type", Value: "Workflows$CallMicroflowActivity"}, + {Key: "$ID", Value: bin(90)}, + {Key: "PersistentId", Value: bin(91)}, + {Key: "Name", Value: "TaskA"}, + }, + }}, + }}, + }) + got := persistentIDs(t, CarryPersistentIDs(rebuilt, stored)) + for _, id := range got { + if id == blobToUUID(bin(20).Data) { + t.Error("a CallMicroflowActivity inherited a SingleUserTaskActivity's PersistentId") + } + } +} + +// Injectivity: two elements must never end up sharing one identity, whatever the +// alignment does. +func TestCarryPersistentIDs_NeverDuplicates(t *testing.T) { + stored := wfDoc(t, 10, 20, 11, 20, "A", "B") // stored already has a duplicate + rebuilt := wfDoc(t, 90, 91, 92, 93, "A", "B") + + got := persistentIDs(t, CarryPersistentIDs(rebuilt, stored)) + seen := map[string]bool{} + for _, id := range got { + if seen[id] { + t.Fatalf("two elements share PersistentId %s", id) + } + seen[id] = true + } +} + +// A document that cannot be read on either side passes through untouched. +func TestCarryPersistentIDs_MalformedPassesThrough(t *testing.T) { + stored := wfDoc(t, 10, 20, 11, 21, "A", "B") + junk := []byte{1, 2, 3} + if got := CarryPersistentIDs(junk, stored); string(got) != string(junk) { + t.Error("malformed contents were modified") + } + good := wfDoc(t, 90, 91, 92, 93, "A", "B") + if got := CarryPersistentIDs(good, junk); string(got) != string(good) { + t.Error("contents were modified against malformed stored bytes") + } +} + +// MXCLI_ALWAYS_WRITE turns off eliding the write, not preserving what the +// document is — the same rule StableId follows. +func TestReconcile_AlwaysWriteStillCarriesPersistentIDs(t *testing.T) { + t.Setenv("MXCLI_ALWAYS_WRITE", "1") + stored := wfDoc(t, 10, 20, 11, 21, "A", "B") + rebuilt := wfDoc(t, 90, 91, 92, 93, "A", "B") + + out, unchanged := Reconcile(rebuilt, stored) + if unchanged { + t.Fatal("MXCLI_ALWAYS_WRITE must not elide") + } + got, want := persistentIDs(t, out), persistentIDs(t, stored) + for i := range want { + if got[i] != want[i] { + t.Errorf("forced write re-minted PersistentId[%d]: %s, want %s", i, got[i], want[i]) + } + } +} diff --git a/modelsdk/gen/keyaudit_test.go b/modelsdk/gen/keyaudit_test.go index 1d25fdea0..109066c28 100644 --- a/modelsdk/gen/keyaudit_test.go +++ b/modelsdk/gen/keyaudit_test.go @@ -117,7 +117,9 @@ var knownWrongKeys = []keyMismatch{ {"Microflows$ListRange", "ListVariableName", "ListName"}, {"Microflows$MappingRequestHandling", "Mapping", "MappingId"}, {"Microflows$MappingRequestHandling", "MappingArgumentVariableName", "MappingVariableName"}, - {"Microflows$RuleCall", "Rule", "Microflow"}, + // {"Microflows$RuleCall", "Rule", "Microflow"} — FIXED (#939): a rule split + // wrote the reference under "Rule", which Mendix never reads, so the decision + // lost its condition (CE0080). Overridden in initRuleCall + InitFromRaw. {"Microflows$Sort", "ListVariableName", "ListName"}, {"Microflows$Sort", "SortItemList", "Sortings"}, {"Microflows$Subtract", "ListVariableName", "ListName"}, @@ -134,8 +136,6 @@ var knownWrongKeys = []keyMismatch{ {"Reports$ReportZoomMapping", "TargetParameterName", "Parameter"}, {"Rest$ODataRemoteEntitySource", "EntityTypeName", "RemoteName"}, {"Rest$ODataRemoteEntitySource", "EntitySetName", "EntitySet"}, - {"Security$ProjectSecurity", "AdminUserRoleName", "AdminUserRole"}, - {"Security$ProjectSecurity", "GuestUserRoleName", "GuestUserRole"}, {"Settings$Configuration", "RuntimePortNumber", "HttpPortNumber"}, {"Settings$Configuration", "AdminPortNumber", "ServerPortNumber"}, {"Settings$Configuration", "RuntimePortOnlyLocal", "OpenHttpPort"}, diff --git a/modelsdk/gen/microflows/types.go b/modelsdk/gen/microflows/types.go index e2f1081d9..7a41915bf 100644 --- a/modelsdk/gen/microflows/types.go +++ b/modelsdk/gen/microflows/types.go @@ -10191,7 +10191,9 @@ func (o *RuleCall) RemoveParameterMappings(index int) { // InitFromRaw populates lazy-decoded property holders from raw BSON. func (o *RuleCall) InitFromRaw(raw bson.Raw) { - if val, err := raw.LookupErr("Rule"); err == nil { + // STORAGE-NAME OVERRIDE: see initRuleCall — the rule reference is stored + // under "Microflow". + if val, err := raw.LookupErr("Microflow"); err == nil { if s, ok := val.StringValueOK(); ok { o.rule.SetFromDecode(s) } @@ -16065,7 +16067,14 @@ func NewRule() *Rule { func initRuleCall() *RuleCall { o := &RuleCall{} o.SetTypeName("Microflows$RuleCall") - o.rule = property.NewByNameRef[element.Element]("Rule", "Microflows$Rule") + // STORAGE-NAME OVERRIDE: real BSON key is Microflow, not Rule. Rules share the + // microflow namespace and Mendix stores the reference under the microflow key + // (generated/metamodel: `json:"microflow"`; sdk/mpr/writer_microflow.go writes + // it; modelsdk/gen/keyaudit_test.go records the mismatch). Written as "Rule" + // the reference is invisible to Mendix — mx check CE0080 "The 'Condition' + // property is required" (upstream #939). Patched on BOTH sides: InitFromRaw + // below reads the same key. + o.rule = property.NewByNameRef[element.Element]("Microflow", "Microflows$Rule") o.rule.Bind(&o.Base, 0) o.parameterMappings = property.NewPartList[element.Element]("ParameterMappings") o.parameterMappings.Bind(&o.Base, 1) diff --git a/modelsdk/gen/security/storagename_projectsecurity_test.go b/modelsdk/gen/security/storagename_projectsecurity_test.go new file mode 100644 index 000000000..f8218cbde --- /dev/null +++ b/modelsdk/gen/security/storagename_projectsecurity_test.go @@ -0,0 +1,100 @@ +// SPDX-License-Identifier: Apache-2.0 + +package security + +import ( + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// TestProjectSecurityUserRoleKeysUseStorageNames pins the BSON keys of the two +// user-role references on Security$ProjectSecurity. +// +// The generator that produced this package bound both by their SDK names, +// "AdminUserRoleName" and "GuestUserRoleName". Mendix stores them without the +// suffix. The in-repo generator and real documents agree: +// +// generated/metamodel/types.go +// AdminUserRoleName string `json:"adminUserRole,omitempty"` +// GuestUserRoleName string `json:"guestUserRole,omitempty"` +// ^ SDK name ^ storage name +// +// A Studio Pro authored project's Security$ProjectSecurity unit: +// AdminUserRole "Administrator" +// GuestUserRole "" +// +// The guest one is what makes anonymous access authorable: mxbuild raises CE0133 +// ("No user role for anonymous users selected even though the feature anonymous +// users is enabled") when EnableGuestAccess is on and this key is empty, so a +// value written under the wrong name is not a cosmetic slip — the app does not +// build. Reading the wrong key is quieter and was live: SHOW PROJECT SECURITY +// printed no guest role on the modelsdk engine while the legacy engine printed +// it, and the documented Starlark fields anonymous_user_role and +// role.is_anonymous resolved to "" / never for every project. +// +// A Primitive decodes and encodes under the same bound name (property.Primitive +// passes p.name to both), so one literal covers both directions — but assert +// both, because a future re-vendor of gen restores the SDK name in one place. +func TestProjectSecurityUserRoleKeysUseStorageNames(t *testing.T) { + cases := []struct { + storageKey string + sdkName string + set func(*ProjectSecurity, string) + get func(*ProjectSecurity) string + }{ + { + storageKey: "AdminUserRole", + sdkName: "AdminUserRoleName", + set: (*ProjectSecurity).SetAdminUserRoleName, + get: (*ProjectSecurity).AdminUserRoleName, + }, + { + storageKey: "GuestUserRole", + sdkName: "GuestUserRoleName", + set: (*ProjectSecurity).SetGuestUserRoleName, + get: (*ProjectSecurity).GuestUserRoleName, + }, + } + + for _, tc := range cases { + t.Run(tc.storageKey+"/encode", func(t *testing.T) { + o := NewProjectSecurity() + tc.set(o, "Administrator") + + var found bool + for _, p := range o.Properties() { + switch p.Name() { + case tc.storageKey: + found = true + case tc.sdkName: + t.Errorf("property is bound as %q — that is the SDK name, not the key on disk. "+ + "Writing it adds a property the type does not have, which Studio Pro refuses "+ + "to open (System.InvalidOperationException at MprProperty.cs) even though "+ + "mxbuild tolerates it", p.Name()) + } + } + if !found { + t.Errorf("no %q property; this key is written to the .mxunit", tc.storageKey) + } + }) + + t.Run(tc.storageKey+"/decode", func(t *testing.T) { + raw, err := bson.Marshal(bson.D{ + {Key: "$Type", Value: "Security$ProjectSecurity"}, + {Key: tc.storageKey, Value: "Administrator"}, + }) + if err != nil { + t.Fatalf("marshal: %v", err) + } + + o := initProjectSecurity() + o.InitFromRaw(bson.Raw(raw)) + + if got := tc.get(o); got != "Administrator" { + t.Errorf("decoded %q, want Administrator — InitFromRaw is reading the wrong key, "+ + "so every Studio Pro authored project reads back with no %s", got, tc.storageKey) + } + }) + } +} diff --git a/modelsdk/gen/security/types.go b/modelsdk/gen/security/types.go index d4a05770a..da5057150 100644 --- a/modelsdk/gen/security/types.go +++ b/modelsdk/gen/security/types.go @@ -865,7 +865,12 @@ func initProjectSecurity() *ProjectSecurity { o.adminUserName.Bind(&o.Base, 4) o.adminPassword = property.NewPrimitive[string]("AdminPassword", property.DecodeString) o.adminPassword.Bind(&o.Base, 5) - o.adminUserRoleName = property.NewPrimitive[string]("AdminUserRoleName", property.DecodeString) + // STORAGE-NAME OVERRIDE: BSON key is "AdminUserRole", not the SDK name + // "AdminUserRoleName" (generated/metamodel tags it `json:"adminUserRole"`, + // and Studio Pro authored projects store "AdminUserRole": "Administrator"). + // A Primitive uses its bound name for both decode and encode, so this one + // literal covers InitFromRaw too. + o.adminUserRoleName = property.NewPrimitive[string]("AdminUserRole", property.DecodeString) o.adminUserRoleName.Bind(&o.Base, 6) o.enableDemoUsers = property.NewPrimitive[bool]("EnableDemoUsers", property.DecodeBool) o.enableDemoUsers.Bind(&o.Base, 7) @@ -873,7 +878,12 @@ func initProjectSecurity() *ProjectSecurity { o.demoUsers.Bind(&o.Base, 8) o.enableGuestAccess = property.NewPrimitive[bool]("EnableGuestAccess", property.DecodeBool) o.enableGuestAccess.Bind(&o.Base, 9) - o.guestUserRoleName = property.NewPrimitive[string]("GuestUserRoleName", property.DecodeString) + // STORAGE-NAME OVERRIDE: BSON key is "GuestUserRole", not the SDK name + // "GuestUserRoleName" — same split as AdminUserRole above. This one is + // load-bearing rather than cosmetic: mxbuild raises CE0133 when + // EnableGuestAccess is on and no role is stored under this key, so writing + // the SDK name means anonymous access never builds. + o.guestUserRoleName = property.NewPrimitive[string]("GuestUserRole", property.DecodeString) o.guestUserRoleName.Bind(&o.Base, 10) o.signInMicroflow = property.NewByNameRef[element.Element]("SignInMicroflow", "Microflows$Microflow") o.signInMicroflow.Bind(&o.Base, 11) diff --git a/sdk/microflows/microflows.go b/sdk/microflows/microflows.go index 6cef1ddf8..296fcb06e 100644 --- a/sdk/microflows/microflows.go +++ b/sdk/microflows/microflows.go @@ -75,14 +75,28 @@ func (n *Nanoflow) GetContainerID() model.ID { return n.ContainerID } -// Rule represents a rule in the Mendix model. +// Rule represents a rule (Microflows$Rule) in the Mendix model — Mendix's own +// reference calls it "a special kind of microflow" that returns a Boolean or an +// enumeration and may only be used from a decision. +// +// The fields are the ten properties a rule document stores, measured against two +// Studio Pro-authored rules (ako/TestApp, Mendix 11.13.0). A rule is a microflow +// minus nine properties, and the nine are the ones a rule has no concept of: +// AllowedModuleRoles (a rule is not independently callable, so there is nothing +// to grant), the concurrency group, Url/UrlSearchParameters, StableId, and the +// two action-info slots. type Rule struct { model.BaseElement - ContainerID model.ID `json:"containerId"` - Name string `json:"name"` - Documentation string `json:"documentation,omitempty"` - - // Return type (always boolean) + ContainerID model.ID `json:"containerId"` + Name string `json:"name"` + Documentation string `json:"documentation,omitempty"` + Excluded bool `json:"excluded"` + MarkAsUsed bool `json:"markAsUsed"` + ApplyEntityAccess bool `json:"applyEntityAccess"` + ReturnVariableName string `json:"returnVariableName,omitempty"` + + // Return type — Boolean or an enumeration. Not "always boolean": Rules.Rule2 + // in the reference app returns Rules.RuleResult. ReturnType DataType `json:"returnType,omitempty"` // Parameters diff --git a/sdk/mpr/parser_rule.go b/sdk/mpr/parser_rule.go new file mode 100644 index 000000000..50650ee10 --- /dev/null +++ b/sdk/mpr/parser_rule.go @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" + + "go.mongodb.org/mongo-driver/bson" +) + +// parseRule parses a Microflows$Rule document. A rule shares a microflow's +// object collection, flows, parameters and return type, so this mirrors +// parseNanoflow with two differences measured against Studio Pro-authored rules +// (ako/TestApp, Mendix 11.13.0): +// +// - a rule stores no AllowedModuleRoles — it is not independently callable, so +// it has no module-role security; +// - gen declares a ReturnType string beside MicroflowReturnType, but Studio Pro +// does not write it and generated/metamodel does not list it, so it is not +// read and must not be written. +func (r *Reader) parseRule(unitID, containerID string, contents []byte) (*microflows.Rule, error) { + contents, err := r.resolveContents(unitID, contents) + if err != nil { + return nil, err + } + + var raw map[string]any + if err := bson.Unmarshal(contents, &raw); err != nil { + return nil, fmt.Errorf("failed to unmarshal BSON: %w", err) + } + + rule := µflows.Rule{} + rule.ID = model.ID(unitID) + rule.TypeName = "Microflows$Rule" + rule.ContainerID = model.ID(containerID) + + if name, ok := raw["Name"].(string); ok { + rule.Name = name + } + if doc, ok := raw["Documentation"].(string); ok { + rule.Documentation = doc + } + if excluded, ok := raw["Excluded"].(bool); ok { + rule.Excluded = excluded + } + if markAsUsed, ok := raw["MarkAsUsed"].(bool); ok { + rule.MarkAsUsed = markAsUsed + } + if applyEntityAccess, ok := raw["ApplyEntityAccess"].(bool); ok { + rule.ApplyEntityAccess = applyEntityAccess + } + if returnVariableName, ok := raw["ReturnVariableName"].(string); ok { + rule.ReturnVariableName = returnVariableName + } + + // Return type — Boolean or an enumeration, under the microflow's BSON key. + if rt, ok := raw["MicroflowReturnType"].(map[string]any); ok { + rule.ReturnType = parseMicroflowDataType(rt) + } + + if oc := extractBsonMap(raw["ObjectCollection"]); oc != nil { + rule.ObjectCollection = parseMicroflowObjectCollection(oc) + for _, obj := range extractBsonSlice(oc["Objects"]) { + if objMap := extractBsonMap(obj); objMap != nil { + if typeName, _ := objMap["$Type"].(string); typeName == "Microflows$MicroflowParameter" { + rule.Parameters = append(rule.Parameters, parseMicroflowParameter(objMap)) + } + } + } + } + + if flowsRaw := raw["Flows"]; flowsRaw != nil { + if rule.ObjectCollection == nil { + rule.ObjectCollection = µflows.MicroflowObjectCollection{} + } + for _, f := range extractBsonSlice(flowsRaw) { + flowMap := extractBsonMap(f) + if flowMap == nil { + continue + } + typeName, _ := flowMap["$Type"].(string) + switch typeName { + case "Microflows$AnnotationFlow": + if af := parseAnnotationFlow(flowMap); af != nil { + rule.ObjectCollection.AnnotationFlows = append(rule.ObjectCollection.AnnotationFlows, af) + } + default: + if flow := parseSequenceFlow(flowMap); flow != nil { + rule.ObjectCollection.Flows = append(rule.ObjectCollection.Flows, flow) + } + } + } + } + + return rule, nil +} diff --git a/sdk/mpr/parser_rule_test.go b/sdk/mpr/parser_rule_test.go new file mode 100644 index 000000000..d05343975 --- /dev/null +++ b/sdk/mpr/parser_rule_test.go @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "testing" + + "github.com/mendixlabs/mxcli/sdk/microflows" + "go.mongodb.org/mongo-driver/bson" +) + +// studioProRuleBSON reproduces Rules.Rule2 from the reference app +// (ako/TestApp, Mendix 11.13.0) — an enumeration-returning rule with an entity +// parameter, in the key order Studio Pro stores. +// +// The two deliberate omissions are the point of the fixture: a real rule +// document carries no AllowedModuleRoles and no ReturnType, so a parser that +// reaches for either is reading a key Mendix never wrote. +func studioProRuleBSON(t *testing.T) []byte { + t.Helper() + doc := bson.D{ + {Key: "$ID", Value: "rule-1"}, + {Key: "$Type", Value: "Microflows$Rule"}, + {Key: "ApplyEntityAccess", Value: false}, + {Key: "Documentation", Value: "decides the outcome"}, + {Key: "Excluded", Value: false}, + {Key: "ExportLevel", Value: "Hidden"}, + {Key: "Flows", Value: bson.A{int32(3)}}, + {Key: "MarkAsUsed", Value: false}, + {Key: "MicroflowReturnType", Value: bson.D{ + {Key: "$ID", Value: "rt-1"}, + {Key: "$Type", Value: "DataTypes$EnumerationType"}, + {Key: "Enumeration", Value: "Rules.RuleResult"}, + }}, + {Key: "Name", Value: "Rule2"}, + {Key: "ObjectCollection", Value: bson.D{ + {Key: "$ID", Value: "oc-1"}, + {Key: "$Type", Value: "Microflows$MicroflowObjectCollection"}, + {Key: "Objects", Value: bson.A{ + int32(3), + bson.D{ + {Key: "$ID", Value: "p-1"}, + {Key: "$Type", Value: "Microflows$MicroflowParameter"}, + {Key: "Name", Value: "pName"}, + {Key: "VariableType", Value: bson.D{ + {Key: "$ID", Value: "vt-1"}, + {Key: "$Type", Value: "DataTypes$ObjectType"}, + {Key: "Entity", Value: "Pages.Bus"}, + }}, + }, + bson.D{ + {Key: "$ID", Value: "end-1"}, + {Key: "$Type", Value: "Microflows$EndEvent"}, + {Key: "ReturnValue", Value: "Rules.RuleResult.Approved"}, + }, + }}, + }}, + {Key: "ReturnVariableName", Value: "Variable"}, + } + b, err := bson.Marshal(doc) + if err != nil { + t.Fatalf("marshal rule fixture: %v", err) + } + return b +} + +// The legacy engine reads a rule with the same fidelity as the codec engine: +// name, documentation, the enumeration return type, the entity parameter, and +// ReturnVariableName (which Studio Pro writes and a rewrite must not drop). +func TestParseRule_StudioProDocument(t *testing.T) { + rule, err := testReader().parseRule("rule-1", "container-1", studioProRuleBSON(t)) + if err != nil { + t.Fatalf("parseRule: %v", err) + } + + if rule.Name != "Rule2" { + t.Errorf("Name = %q, want Rule2", rule.Name) + } + if rule.Documentation != "decides the outcome" { + t.Errorf("Documentation = %q", rule.Documentation) + } + if rule.ReturnVariableName != "Variable" { + t.Errorf("ReturnVariableName = %q, want %q", rule.ReturnVariableName, "Variable") + } + if rule.TypeName != "Microflows$Rule" { + t.Errorf("TypeName = %q, want Microflows$Rule", rule.TypeName) + } + + enum, ok := rule.ReturnType.(*microflows.EnumerationType) + if !ok { + t.Fatalf("ReturnType = %T, want *microflows.EnumerationType — a rule may return an enumeration, not only Boolean", rule.ReturnType) + } + if enum.EnumerationQualifiedName != "Rules.RuleResult" { + t.Errorf("enumeration = %q, want Rules.RuleResult", enum.EnumerationQualifiedName) + } + + if len(rule.Parameters) != 1 { + t.Fatalf("Parameters = %d, want 1", len(rule.Parameters)) + } + if rule.Parameters[0].Name != "pName" { + t.Errorf("parameter = %q, want pName", rule.Parameters[0].Name) + } + if rule.ObjectCollection == nil || len(rule.ObjectCollection.Objects) == 0 { + t.Error("ObjectCollection did not come back") + } +} diff --git a/sdk/mpr/reader_documents.go b/sdk/mpr/reader_documents.go index 45f1e7f3c..4fabd2e49 100644 --- a/sdk/mpr/reader_documents.go +++ b/sdk/mpr/reader_documents.go @@ -214,6 +214,40 @@ func (r *Reader) GetMicroflow(id model.ID) (*microflows.Microflow, error) { return r.parseMicroflow(unit.ID, unit.ContainerID, unit.Contents) } +// ListRules returns every rule document (Microflows$Rule). Rules are a distinct +// doctype and deliberately absent from ListMicroflows. +func (r *Reader) ListRules() ([]*microflows.Rule, error) { + units, err := r.listUnitsByType("Microflows$Rule") + if err != nil { + return nil, err + } + + var result []*microflows.Rule + for _, u := range units { + rule, err := r.parseRule(u.ID, u.ContainerID, u.Contents) + if err != nil { + return nil, fmt.Errorf("failed to parse rule %s: %w", u.ID, err) + } + result = append(result, rule) + } + + return result, nil +} + +// GetRule retrieves a rule by ID. +func (r *Reader) GetRule(id model.ID) (*microflows.Rule, error) { + rules, err := r.ListRules() + if err != nil { + return nil, err + } + for _, rule := range rules { + if rule.ID == id { + return rule, nil + } + } + return nil, nil +} + // IsRule reports whether the given qualified name refers to a rule // (Microflows$Rule). Rules share the microflow namespace but are stored // under a distinct BSON type — the flow-builder needs this distinction so diff --git a/sdk/mpr/writer_security.go b/sdk/mpr/writer_security.go index 916e1ff80..4f999d9c0 100644 --- a/sdk/mpr/writer_security.go +++ b/sdk/mpr/writer_security.go @@ -297,6 +297,19 @@ func (w *Writer) SetProjectDemoUsersEnabled(unitID model.ID, enabled bool) error }) } +// SetProjectGuestAccess patches EnableGuestAccess — and, when guestUserRole is +// non-empty, GuestUserRole — on Security$ProjectSecurity. An empty role leaves +// the stored one alone so that toggling access off and on does not lose it. +func (w *Writer) SetProjectGuestAccess(unitID model.ID, enabled bool, guestUserRole string) error { + return w.readPatchWrite(unitID, func(doc bson.D) (bson.D, error) { + doc = setBsonField(doc, "EnableGuestAccess", enabled) + if guestUserRole != "" { + doc = setBsonField(doc, "GuestUserRole", guestUserRole) + } + return doc, nil + }) +} + // AddUserRole adds a new user role to Security$ProjectSecurity. func (w *Writer) AddUserRole(unitID model.ID, name string, moduleRoles []string, manageAllRoles bool) error { return w.readPatchWrite(unitID, func(doc bson.D) (bson.D, error) {