diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 8a7db17d9..51ae363e2 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -576,3 +576,20 @@ extracting `OffsetExpression`/`LimitExpression`. | Binary upload is reported as impossible in MDL — "a consumed REST operation has no binary body, so use a Java action". A Studio Pro-authored binary POST also **describes with no body at all**, and re-executing that DESCRIBE produces a request that sends nothing | Right conclusion about the **wrong document**. Mendix models a binary request body on the microflow **REST CALL activity** as `Microflows$BinaryRequestHandling` + an action-level `RequestHandlingType: "Binary"` — a `Microflows$` type, which is why a metamodel grep for `Rest$*Body` finds only the three non-binary ones and "proves" it impossible. mxcli could **parse** it (`sdk/mpr/parser_microflow_actions.go`) and could neither write, read (modelsdk) nor describe it, so it survived a legacy read and vanished everywhere else | Grammar `mdl/grammar/domains/MDLMicroflow.g4` (`restCallBodyClause`); `mdl/ast/ast_microflow.go` (`RestBodyBinary`); `mdl/visitor/visitor_microflow_actions.go`; `mdl/executor/cmd_microflows_builder_calls.go`; writers `mdl/backend/modelsdk/microflow_write.go` + `sdk/mpr/writer_microflow_actions.go`; reader `mdl/backend/modelsdk/microflow_read_actions.go` (`restRequestHandlingFromRaw`); formatter `mdl/executor/cmd_microflows_format_action.go`; tests `mdl/executor/cmd_microflows_binary_body_test.go`; fixture `mdl-examples/bug-tests/rest-binary-post.mdl` | **"Not in the metamodel" needs the right namespace before it is a conclusion** — the search was for `Rest$…Body` and the answer lives under `Microflows$…RequestHandling`. Ask for a Studio Pro example instead of reasoning from an absence: one 4-KB unit settled in minutes what a metamodel grep had "disproved". **The discriminator and the sub-element must agree** — `RequestHandlingType` was hardcoded `"Custom"` in BOTH engines regardless of the handler; only the Binary case is derived here, because the others have no measured reference and work today (a latent mismatch worth a separate look). **The expression is the `Contents` MEMBER** (`$Doc/Contents`), not the document, and is stored as source text — quoting it sends the path as a string literal. **Verify a round trip by re-executing DESCRIBE and diffing the BSON against Studio Pro's**: mxcli's output reproduced the Studio Pro action exactly (same type, same discriminator, same expression) with mxbuild 0 errors, which is stronger than any assertion about what "should" be written. Beware the sibling trap: read support in one engine and not the other looks like a describe bug | | A REST call's `body mapping Mod.EMM from $var` is written with the variable under a key Mendix does not read, an empty `ContentType`, and an action-level `RequestHandlingType` of `Custom` that contradicts its own `Microflows$MappingRequestHandling` sub-element. A form-data body is dropped entirely by DESCRIBE, so describe → edit → exec silently produces a call that posts nothing | `generated/metamodel` is decisive: `MicroflowsMappingRequestHandling` owns exactly three properties — `contentType` (enum Json\|Xml), `mappingId`, `mappingVariableName`. mxcli wrote `ParameterVariable`, which the type does not own, and omitted `MappingVariableName`; its own READER had known the right key since #843 and the writer was never corrected. `RequestHandlingType` was hardcoded `"Custom"` in both engines regardless of handler. `FormDataRequestHandling` / `AdvancedRequestHandling` can be parsed and not written, so a rewrite dropped them | `mdl/backend/modelsdk/microflow_write.go` (`requestHandlingTypeOf`, the Mapping case) and `sdk/mpr/writer_microflow_actions.go` (`restRequestHandlingTypeOf`, same case); guard `mdl/executor/validate_rest_request_handling.go` wired from `cmd_microflows_create.go`; tests `mdl/backend/modelsdk/microflow_restbody_test.go`, `mdl/executor/validate_rest_request_handling_test.go` | **A reader that compensates for a writer hides the writer's bug** — the `MappingVariableName` fallback made DESCRIBE round-trip correctly while every document mxcli wrote carried the wrong key. When a reader has a "the real key is X" comment, check that the writer agrees. **An unknown property is worse than a wrong value**: mxbuild tolerates it and Studio Pro refuses to open the document, so a green build proves nothing (same rule as the overlay section in CLAUDE.md). **Ask for one Studio Pro example per variant** — four microflows covering Custom/Mapping/FormData/Binary settled the discriminator question that a single example had left as "unverified, so leave it alone"; guessing the other five enum values from one measured case would have been a coin flip. **Parse-but-cannot-write is a data-loss path, not a gap**: DESCRIBE omits what the writer cannot express, so the omission looks like faithful output — refuse the rewrite (ADR-0005), and make the guard's allow-list the writable set so it stops refusing the moment a type becomes expressible. Not fixed: the legacy engine encodes `MappingId` as binary where Studio Pro stores a qualified-name string | | `DESCRIBE MICROFLOW` emits MDL that is **not equivalent** to the microflow: a branch's activities are missing from the output, and re-executing the description builds a different flow. Reported as "roundtrip for @merge does not work" plus a separate complaint about the diagram coming back tangled | The graph is not **properly nested**. MDL's `IF/THEN/ELSE` is a single-entry/single-exit block; a Mendix microflow is an arbitrary graph, and when a branch re-enters a sibling branch's path there is no nesting that means the same thing. The describer walks it as a tree anyway. Measured on the reporter's graph: `log` ran on `¬c1 ∨ c2` and was described as `c1 ∧ c2` — with their actual expressions (`not(true)` on both decisions) the original **always** logs and the description **never** does, i.e. the exact inverse, not a near-miss. The tangled diagram is the *same* cause: `findSplitMergePointsForGraph` and `commonMergeAfter` are two independent merge-finders that agree on every nested graph and disagree here (merge2 vs merge1), so the emitted `@merge(x,y)` places the merge before an activity that structurally follows it | `mdl/microflowgraph/structure.go` (`Analyze` — post-dominators + branch-body overlap); `mdl/linter/rules/flow_irreducible_graph.go` (MDL-FLOW01); `irreducibleGraphWarnings` in `mdl/executor/cmd_microflows_show.go` | **Reconstruct the reporter's graph from the coordinates in their own output** before touching code — `@position`/`@merge` values pin the shape exactly, and `commonMergeAfter` reproducing their `@merge(-200,173)`/`@merge(100,173)` byte-for-byte is what confirmed the reconstruction was right rather than merely plausible. **Do not reuse the describer's join search in the detector**: there are already two implementations and they disagree on exactly the graphs being detected, so a detector built on either inherits whichever is wrong. Post-dominance is self-contained (`pdom(n) = {n} ∪ ⋂ pdom(succ)`, universe-initialised so it converges on the cyclic graphs retry loops create) and makes a describer/detector disagreement a *signal*. Classification: overlap with **one** entry point is a shared suffix (recombinable — the guards fold, `¬c1 ∨ c2`), **two or more** is genuine crossing (interleaved — needs activity duplication or a synthetic boolean per Böhm-Jacopini, so it is refused). **The false positive is worse than the bug** — flagging ordinary nesting would warn on nearly every microflow — so pin the shapes most likely to trip it: `if` with no `else`, an inner split whose join *is* the outer's, branches that both return, and error-handler flows (excluded, or every activity with one gets flagged). **A rule that never runs and a rule that finds nothing look identical**: prove the wiring with a forced-fire control (temporarily append a finding per microflow, lint a real project, confirm the count, revert) — `mxcli lint` reporting "No issues found" on a blank app is also just marketplace modules being excluded. Rendering it faithfully needs a label form; Mendix gives no field to hang one on (`Microflows$ExclusiveMerge` stores only `RelativeMiddlePoint` and `Size` — confirmed against `generated/metamodel`, `modelsdk/gen` and real 11.13 documents), but labels only need to be deterministic, not stored. Issue #923; design in `docs/11-proposals/PROPOSAL_structured_microflow_description.md` | +| A CLI flag is accepted but does nothing: `mxcli test tests/ -p app.mpr --require-assertions` exits 0 on a suite where every test asserts nothing. `--help` lists the flag, it parses, and the summary line even prints "1 test(s) asserted nothing" — so the run looks like it noticed and chose to pass | The flag was consulted in exactly one of the two result-assembly loops. `vacuousResult` had a single caller, in the endpoint runner's loop (`--local`, `--attach`); the after-startup runner (Docker default, and `--local --legacy-runner`) assembles results in `ParseLogResults(logReader, suite)`, whose signature had no parameter the flag could arrive through — it could not honour it even in principle | `cmd/mxcli/testrunner/results.go` (`ParseLogResults`, `preRunResult`), `cmd/mxcli/testrunner/runner_endpoint.go` (`runSuite`), `cmd/mxcli/testrunner/runner.go` (the `ParseLogResults` call in `runAfterStartup`) | Give the pre-run verdicts — the ones decided by the *parsed test case* rather than by the run — **one** function (`preRunResult`) and route both loops through it, then pin that it is their only caller (`TestPreRunVerdictsHaveOneCallSite`). Same lesson as the `newResult` constructor two rows up: a decision reachable from one path and not the other is the defect, and a single call site is what prevents the next one. **Do not reach for the refusal precedent without checking it applies**: `rejectVerifyOnLegacyRunner` refuses `@verify` on this runner because it genuinely cannot evaluate it (its tests run during boot, there is no seam to query the app at). `AssertionCount()` is a static property of the parsed test case, known before any runner starts — refusing there would have denied a capability the runner has. **Generalisable**: when auditing a flag for dead ends, grep for its *consumers* and count them against the number of code paths that should honour it; a unit test on the helper (`TestRequireAssertionsMakesVacuousTestsErrors` called `vacuousResult` directly) proves the helper works and says nothing about whether any runner calls it. Issue #926 | +| A pluggable widget authored by MDL passes `mxcli check` and then fails **CE0463** "the definition of this widget has changed" on `mx check`, so `mx create-module-package` refuses the whole module. mxcli's own MDL-WIDGET10 warns about several properties — all of them harmless — and says nothing about the one that breaks the build | mxbuild evaluates the widget's compiled `editorConfig.js#getProperties()`, and a property that function **hides** must hold its **default** value. mxcli lifted only the top-level `hidePropertyIn`/`hidePropertiesIn` calls: `hideNestedPropertiesIn` and the 5-argument nested form were skipped by design (#574 Phase 1), which is exactly where the Accordion hides its group's `initialCollapsedState`/`initiallyCollapsed`. Two further gaps in the same extractor: a comparison guard read from a `\|\|` connector or a ternary's ELSE branch was recorded with the WRONG polarity (`eq` where the code means `ne`), and the shipped compilation is `forEach((function(n,o){…}))` — a double paren the callback-parameter regex read as the parameter name, so every nested condition was scoped to the widget instead of the item | `mdl/executor/editorconfig_extract.go` (`hideTargetKeys`, `parseGuard`, `enclosingForEachParam`); `mdl/executor/validate_widget_hidden.go` (`validateWidgetItemVisibility`, `hiddenPropertySeverity`); `mdl/types/widget_visibility.go` (`ListPropertyKey`, `Scope`); bump `WidgetDefGeneratorVersion` in `mdl/executor/widget_engine.go`; tests `mdl/executor/editorconfig_nested_test.go`, `mdl/executor/validate_widget_hidden_test.go`; fixture `mdl-examples/bug-tests/931-accordion-hidden-property.mdl` | **Split the severity on the VALUE, not the property** — the same hidden property is harmless at its default (Studio Pro ignores it) and a build failure otherwise, which is why four warnings fired on defaults while the two real failures were silent. **Measure the rule instead of reading the JS**: a matrix of eight one-property variants through `mx check` (`collapsible` on/off × default/non-default) pinned it in minutes and became the test oracle — mxcli's verdict now matches mxbuild on all nine measured cases. Note `"web" !== platform` evaluates TRUE under mxbuild, so guards behind it never fire; the conjunct rule below is what makes the Accordion's compound guard liftable anyway. **`X && Y \|\| hide(…)` soundly implies "hide when Y falsy"** whatever X is, so a compound guard with one unresolvable operand is still liftable — but `X && Y && hide(…)` is not (hiding needs both), and lifting it would over-fire. **Widget defs are cached per project** in `.mxcli/widgets/*.def.json`; a rules change is invisible until `WidgetDefGeneratorVersion` is bumped, and the refresh happens during `exec`, so the run that regenerates them still validates against the old set. Issue mendixlabs/mxcli#931 | +| `ALTER STYLING … SET '' = ''` prints `Updated styling on widget "x"` and writes **nothing**. The same statement works on a Studio Pro-authored widget, and the inline `DesignProperties: [...]` form works everywhere | Two defects composing into a silent no-op. `bsonnav.DSet` takes a `bson.D` **by value**, so it can only overwrite an existing key — it returns `false` and does nothing when the key is absent, and `DSetArray` ignored that result. And the key WAS absent: the codec omits an empty, never-appended `PartList`, so `Forms$Appearance` on an mxcli-authored widget carried no `DesignProperties` at all (Studio Pro writes the empty marker `[3]` on all 16 pages of a blank app) | `mdl/backend/bsonnav/bsonnav.go` (`DSetArray` now returns bool + refuses a markerless write, new `DSetArrayIn`); `mdl/backend/pagemutator/mutator.go` (`writeDesignProperties`); `mdl/backend/modelsdk/widget_write.go` (`Forms$Appearance` MandatoryLists, `CustomWidgets$CustomWidget` LabelTemplate) | **Fixing the append without fixing the marker turns a silent no-op into an unopenable project.** A Mendix array's first entry is its typed-array marker; measured on 11.13, a `DesignProperties` array written bare fails to LOAD (`Type OptionDesignPropertyValue does not contain a constructor with a parameter of type Appearance`) and `create-module-package` dies with "Unknown export error", while a doubled marker throws `StorageLoadException`. So `DSetArray` refuses rather than guessing, and the marker is passed explicitly at the one call site that creates the property. **Diff against `mx update-widgets`' own output** to find missing keys: handing an mxcli-authored page to Mendix's tool and diffing the unit it writes back (pointer identity preserved, key order intact) surfaced `DesignProperties`, `LabelTemplate` and a set of int32/int64 width mismatches in one pass. Not changed: `Forms$Page.AllowedModuleRoles` is marker 3 where Studio Pro writes 1 — `NewByNameRefListV3` documents CE0557 as the reason, and nothing currently misbehaves. Issue mendixlabs/mxcli#931 | +| `mx create-module-package` aborts on a module mxcli created with **`Exception occurred: Unable to cast object of type 'Newtonsoft.Json.Linq.JValue' to type 'Newtonsoft.Json.Linq.JObject'`** at `MprProperty.Init` → `MprUnit.get_Contents` → `MprDocumentHasher.Write`, while `mx check` reports **0 errors**. Removing the widget the reporter blamed changes nothing | `Forms$Page.AllowedModuleRoles` was written with typed-array marker **3** by the modelsdk (default) engine. The marker tells Mendix's reader what the entries ARE, and the entries are qualified-name **strings** — so a NON-EMPTY list under marker 3 is a JValue where the reader wants a JObject. The empty list never crashed (nothing to mis-cast), which is why only a module whose page carries a role hit it: `create module` auto-creates a `User` role and grants it. The legacy engine (`sdk/mpr/writer_security.go`) has always written 1 | `modelsdk/gen/pages/types.go` (`initPage`, the `allowedRoles` line — LIST-MARKER OVERRIDE next to the existing STORAGE-NAME OVERRIDE); `modelsdk/property/reference.go` (`NewByNameRefListV3`, whose comment asserted the opposite) | **Minimise before believing the reporter's attribution.** #931 blamed an Accordion; `create module MMin; create page MMin.P { DYNAMICTEXT }` reproduces the identical exception with no pluggable widget in the project — and the same crash appears on a binary built from `origin/main`, which is the control that proves it pre-existing rather than introduced by the fix in the same branch. **`mx check` is not a safety net for storage shape**: it read this project fine at all three security levels while the exporter could not. **Check the comment's premise before honouring it** — this marker was deliberately 3 because a comment said 1 caused CE0557 "even when roles are set"; measured at Off, Prototype and Production, marker 1 is 0 errors and exports, marker 3 is 0 errors and crashes. Note the repair is not retroactive: idempotent writes leave an existing page alone, so a project keeps the bad marker until each page is rewritten. Issue mendixlabs/mxcli#931 | +| An `else` branch on a `split type` "never fires" — the author expects it to catch object types with no `case`, and it only runs when the object is **null**. Often surfaces as CE0090 "The '' value should be configured for an outgoing flow" on a split that visibly has an `else` | Not a bug: `else` on an inheritance split IS the `(empty)` flow (a null object), not a default. Mendix requires a separate outgoing flow for **every** concrete subtype and the base entity; an `else` contributes nothing to that coverage. The keyword is misleading, which is the defect | `mdl/executor/cmd_microflows_builder_actions.go` (the `addBranch("", s.ElseBody)` call and its CE0089/CE0090 comment) | Add a `case` for every subtype **and** the base entity, keeping the `else`. Do NOT drop the `else` to "fix" CE0090 — that flow is load-bearing and its absence is CE0089. Measured on mxbuild 11.13.0: `case Dog` + `else` → CE0090 for `Cat` and `Animal`; every type cased, `else` kept → 0 errors. The branch is now spelled `when (empty) then`, which says what it does; `else` still parses and warns **MDL065**. Background in [the #913 report](../../docs/12-bug-reports/2026-08-20-issue-913-split-statement-syntax.md) | +| `DESCRIBE microflow` output for a split cannot be read: a branch body sits at the SAME column as its `when`/`case` keyword, so a nested `if`'s `else` lands exactly where a case branch would — in output where `else` on a `case` is an MDL008 error. The enum split and the type split are wrong in OPPOSITE directions (enum indents the branch but not the body; type indents the body but not the branch), and `if/else` is right | Both emitters wrote the branch keyword at `indentStr+" "` and then traversed the body at `indent+1` — the same column. No test caught it because `assertLineContains` matches substrings and never sees leading whitespace | `mdl/executor/cmd_microflows_show_helpers.go` (`emitEnumSplitStatement`, `emitInheritanceSplitStatement`) **and** `mdl/executor/cmd_diff_mdl.go` (`microflowStatementToMDL`) | Traverse branch bodies at `indent+2`. **There are two MDL emitters, not one** — the diff renderer had the identical bug and is only found by grepping for the pattern rather than fixing the file the report names. Prove it with the control: revert one `indent+2` and the test must fail with `branch body indent = 4, want 6`; a test written only against fixed code detects nothing. Test `TestSplitBranchBodiesIndentFromTheirBranchKeyword` (includes an `if is the reference` sub-test, so the splits follow `if` rather than a hardcoded number) | +| A type split spelled `split type $x case Mod.E … else …` warns **MDL065**, or a script using `when Mod.E then` is rejected by an older mxcli | The type split moved onto the enum split's branch syntax (#913): `case` used to introduce a BRANCH here while introducing the SUBJECT in `case $x when V then` and in expressions, so the word meant two things. Both spellings build the identical flow and both still parse | `mdl/grammar/domains/MDLMicroflow.g4` (`inheritanceSplitCase`, `inheritanceSplitElse`), `mdl/visitor/visitor_microflow_statements.go` (`buildInheritanceSplitStatement`), `mdl/executor/validate_microflow.go` (`checkInheritanceSplitSpelling`) | Write `when Module.Entity then` and `when (empty) then`. **Never promote MDL065 to an error** — `TestMDL065_DoesNotBlockExecution` pins that, because `exec`'s pre-flight gate halts on errors and would refuse every script using the old form. When adding a spelling alias, pin the equivalence (`TestInheritanceSplit_BothSpellingsProduceTheSameAST`) rather than assuming it: the deprecation message promises "both build the identical flow", and that promise needs a test. Fixtures `mdl-examples/bug-tests/913-split-type-{unified,legacy}-syntax.mdl` | +| `SHOW ACCESS ON ENTITY Mod.E` is a parse error — `extraneous input '.' expecting the start of a statement` — while `SHOW ACCESS ON MICROFLOW\|PAGE Mod.Name` parse. The generated project CLAUDE.md documents all three. Separately, and found by the same sweep: `SHOW ACCESS ON PAGE` **parses and answers the wrong question**, printing the page's whole widget tree instead of its allowed module roles | Two ordering defects, one at each layer. (a) There was no `ACCESS ON ENTITY` grammar alternative, and `ENTITY` is in the `keyword` rule, so `showOrList ACCESS ON qualifiedName` matched the word ENTITY as the *whole* name; the statement ended there and the real name was extraneous. The command itself always existed — the bare `SHOW ACCESS ON Mod.E` is the same AST node and the same handler. (b) `ExitShowStatement` is a long if/else chain: the singular `SHOW PAGE`→DESCRIBE alias branch (added when four show alternatives had no visitor branch) sits ~200 lines BEFORE the `ACCESS` branch and matches on `ctx.PAGE() != nil` alone, so it swallowed `SHOW ACCESS ON PAGE`. The `ENTITY` branch would have swallowed the new alternative the same way | `mdl/grammar/domains/MDLCatalog.g4` (`showStatement`), `mdl/visitor/visitor_query.go` (`ExitShowStatement` — the `ENTITY` and `PAGE` branches) | Add the alternative, then guard **every earlier branch that matches a bare token the new alternative also contains** with `ctx.ACCESS() == nil`. **Generalisable — the shape to look for**: a token accessor (`ctx.PAGE()`) is true for several alternatives, so in an if/else visitor the first branch wins and the later one is dead. `TestEveryShowAlternativeProducesAStatement` cannot catch it: a statement IS produced, just the wrong one — so a new alternative needs a test asserting the ObjectType, not merely that something came out. Grep `ctx\.[A-Z]*()` in the chain for a token that appears in more than one alternative before adding one. **The `-c` parse-error column is offset** by the `CONNECT LOCAL ''; ` prefix `-p` prepends, which is why the report shows 1:60 for a 40-character command — do not chase the position, reproduce with `mxcli check `. Three controls, each reverted separately: no grammar alternative ⇒ the original parse error; no ENTITY guard ⇒ `ObjectType = ENTITY`; no PAGE guard ⇒ `*ast.DescribeStmt`. Verified end-to-end on a real 10.24 project (access rules / allowed roles reported, `show entity` and `show page` still describing). Tests `mdl/visitor/visitor_show_access_test.go`, repro `mdl-examples/bug-tests/925-show-access-on-entity.mdl`. Issue #925 | +| `CREATE OR MODIFY … FOLDER 'x'` on a document that already exists reports success ("Modified json structure: …") and the document does not move. The folder itself *does* appear, empty, which makes it look like a partial write rather than a no-op. Reported for JSON structures and mappings | Not doctype-specific — **every** doctype with a FOLDER clause did this. A document's folder is its unit's `ContainerID`, a column in the `Unit` row, and every `Update*` on both engines rewrites only the unit's *contents*. So the handlers resolved the folder, stored it on the model object, and had it dropped one layer down. `resolveFolder` creates missing folders as a side effect, which is where the empty folder comes from. The typed `Move*` methods that do reparent were only ever called from `cmd_move.go` / `cmd_folders.go`; `MoveImportMapping` and `MoveExportMapping` existed on the interface, both engines and the mock, and **nothing called them** | `mdl/executor/document_placement.go` (new: `resolveRequestedFolder` + `applyDocumentFolder`), wired into every CREATE OR MODIFY handler that resolves a folder (`cmd_jsonstructures.go`, `cmd_microflows_create.go`, `cmd_nanoflows_create.go`, `cmd_enumerations.go`, `cmd_constants.go`, `cmd_pages_create_v3.go`, `cmd_odata.go` ×2, `cmd_published_rest.go`, `cmd_businessevents.go`, `cmd_rest_clients.go` ×2); primitive in `modelsdk/mpr/writer_core.go` (`MoveUnit`) + `sdk/mpr/writer_domainmodel.go` (`moveUnitByID`); interface `mdl/backend/placement.go` | Fix the **class**, with one type-agnostic `MoveDocument(unitID, containerID)` on the backend, not a `Move` each — placement is the same row update whatever the document is, and a per-doctype method means the next document type inherits the bug by not having one written for it. **The counting is half the fix**: a move changes no byte of the document, so it is invisible to content-based no-op elision (ADR-0008) — without `writesOffered`/`writesLanded` on the row update, `ReportMutation` calls a real move "Unchanged", and without the equality check re-running an in-sync script dirties the `.mpr`. Both directions have controls (strip either and `TestMoveDocumentPersistsAndIsIdempotent` fails). **The control that decides fix-vs-new-bug**: an omitted FOLDER must leave placement alone. `resolveFolder("")` answers with the *module root*, so a handler that passes it through unfiles every foldered document on the next `CREATE OR MODIFY` — far worse than the no-op. `resolveRequestedFolder` returns `""` for "not asked" and `applyDocumentFolder` ignores `""`, so the distinction cannot be forgotten per-site. Doctypes rewritten as **delete+create** (snippets, rest clients) had the *inverse* bug already live — they really do re-apply the container, so a silent statement filed them into the module root; they need `existingContainerID` carried forward, not a move. **Diagnose against storage, not mxcli's own reads** (cf. #722/Bug 12b, where the same symptom was a read illusion): diff `Documents in Modules` / `Documents in Folders` counts from the raw `Unit` table, with an explicit `MOVE` of a supported doctype as the positive control. Repro `mdl-examples/bug-tests/932-folder-on-modify.mdl`; tests `mdl/executor/document_placement_test.go`, `mdl/backend/modelsdk/placement_test.go`. Issue #932 | +| `MOVE IMPORT MAPPING …` / `MOVE JSON STRUCTURE …` is a parse error, and neither mapping type takes a `FOLDER` clause on create — so those documents can never leave the module root from MDL. The documented doctype list ("PAGE \| MICROFLOW \| NANOFLOW \| SNIPPET \| ENUMERATION \| CONSTANT") is itself stale, omitting three doctypes MOVE already supported | `moveStatement` listed nine doctypes and nothing else; the other twenty-two were never unimplemented, just unlisted. `MoveImportMapping` and `MoveExportMapping` were fully wired — interface, both engines, mock — and **called from nowhere**. Same shape as mxcli-formula1 #32, one layer wider | grammar `mdl/grammar/MDLParser.g4` (new `moveDocumentType` rule), registry `mdl/ast/ast.go` (`MoveDocumentTypeByKeyword`, `IsMoveDocumentType`), visitor `mdl/visitor/visitor_entity.go`, executor `mdl/executor/cmd_move_document.go` (generic path) + `cmd_move.go` (default case), backend `FindDocumentUnit` / `ListDocumentUnits` in `mdl/backend/placement.go` | **Make the doctype list a grammar RULE, not an inline alternation.** `MOVE FOLDER` is told from a document move by the *absence* of a doctype, and that discriminator was a hand-written negation of every doctype keyword — with twenty-two more keywords it would never have stayed in step. As a sub-rule it is one `ctx.MoveDocumentType() == nil` check that cannot go stale. **Resolve the document through the unit table, not a per-kind list**: one `FindDocumentUnit(module, name)` covers every doctype including ones added later, where twenty-two `List()` scans would each be a place to forget a kind. **Do not build a doctype → `$Type` table to validate against** — two such tables already exist in this repo and disagree with each other and with a real project (`mdl/types/unit_types.go` says `JavaActions$JavaScriptAction`; the writer inserts and Studio Pro stores `JavaScriptActions$JavaScriptAction`). Derive the kind from the stored `$Type` instead (`types.DocumentKind`) and refuse only when the derived kind is *itself* a doctype MOVE can spell, so a derivation you cannot vouch for defers to the name lookup rather than blocking a legitimate move. **Report the kind you found, not the kind the statement named** — the lookup is by name, so the two can differ, and echoing the user's word would be a confident-sounding lie. Same reasoning fixed `LIST FOLDERS`, whose hand-maintained `documentsByContainer` still hid layouts, menus, JavaScript actions and building blocks after #892 added five kinds to it: it now fills gaps from `ListDocumentUnits()` (measured 78 → 259 documents listed on a 7-module project). Tests `mdl/visitor/visitor_move_alldoctypes_test.go` (drives the registry, so a doctype added to it without grammar support fails), `mdl/executor/cmd_move_document_test.go`; example in `mdl-examples/doctype-tests/18-folder-examples.mdl`. Issue #932 | +| Import and export mappings (and a dozen other doctypes — queues, scheduled events, regular expressions, workflows, menus, image collections, Java/JavaScript actions, database connections, data transformers, the AI-agent documents) have **no `FOLDER` clause on `create` at all**, so they can only ever be created at the module root. `DESCRIBE` on one that Studio Pro filed away emits no folder either, so a describe → re-exec round trip recreates it unfiled | The clause was simply never added to those grammar rules; `CreateImportMappingStmt` had no `Folder` field and the executor hardcoded `containerID := module.ID` | grammar: 16 create rules across `MDLParser.g4`, `MDLDomainModel.g4`, `MDLMicroflow.g4`, `MDLService.g4`, `MDLWorkflow.g4`, `MDLAgent.g4`; AST `Folder` field on each `Create*Stmt`; visitors; executors (`containerForDocument` + `applyDocumentFolder`); DESCRIBE emitters via `describeFolderClause` | Put the clause in **one place per statement — straight after the qualified name** — rather than matching each rule's local style. Pages/snippets (`Folder: 'path'` property) and microflows (`folder 'path'` before `begin`) already disagree, and a third convention per doctype would make the syntax unlearnable. **Watch for rules that already contain a direct `STRING_LITERAL`**: `createWorkflowStatement` read DISPLAY / DESCRIPTION / DUE DATE by *counting* `AllSTRING_LITERAL()` in order, so a folder path — now the rule's first string — silently became the display name and shifted every later clause by one. Fix by **labelling the tokens in the grammar** (`display=STRING_LITERAL`) and reading `ctx.GetDisplay()`, not by placing the new clause last: positional counting is wrong for *any* combination of optional clauses, not just this one. Same for `createDataTransformerStatement`, whose source path was `ctx.STRING_LITERAL()`. Control: revert the workflow visitor to positional and `TestWorkflowHeaderClausesAreReadByLabel` reports `DisplayName = "Private/Filed"`. The executor side is uniform — `containerForDocument(ctx, moduleID, s.Folder, existingContainer)` encodes the precedence (named folder → current placement → module root) once, so no handler decides it again. Verify the round trip by replaying a DESCRIBE **into a module where the document does not exist** — replaying over the original says "Unchanged" whether or not the clause survived. Tests `mdl/visitor/visitor_folder_clause_test.go` (all 16 doctypes); repro `mdl-examples/bug-tests/932-folder-on-modify.mdl`. Issue #932 | +| `SHOW REFERENCES TO ` reports **0 pages, 0 widgets** for an entity that is demonstrably used on a page, and `SEARCH` cannot find content that is in the model. The entity is used inside a **List View specialization template** (one template per specialization of the list view's generalization — Studio Pro only; MDL has no surface for it) | `Templates` was not a recognised widget-container key. `extractWidgetsRecursive` knew `Widgets`, `Rows`/`Columns`, `FooterWidgets`, `TabPages`, pluggable `Object.Properties[].Value.Widgets` and NavigationList `Items[].Widgets` — not `Templates` — so nothing inside a template reached the widgets table, and the refs projection is built from that table. `Templates` was also missing from `widgetChildKeys`, so the **same omission had a second, opposite effect**: the list view's own ref scan descended into its templates and, returning the lexicographically smallest candidate, recorded a template's specialization as the list view's datasource entity | `mdl/catalog/builder_pages.go` (`extractWidgetsRecursive`, `widgetChildKeys`) | Walk each template as a container in its own right — `extractWidgetsRecursive(tplMap)` — so its row carries the specialization it renders and its children are indexed like any other nested widgets, **and** add the key to `widgetChildKeys` so the parent stops absorbing them. **Both halves or neither**: recursion alone leaves the wrong-entity displacement in place, and the key alone hides the contents without indexing them. `widgetChildKeys` and the recursion are two sides of one contract — a key in one and not the other is a bug in a specific direction, so audit them as a pair. (`Items` is correctly asymmetric and must stay that way: a nav item's own `Form`/`Entity` belongs to the NavigationList because items are not emitted as rows, and the walk already stops at the item's `Widgets`.) **Generalisable**: two omissions of the same key in complementary walkers partly mask each other — a stray ref still leaks through — which is why this reads as intermittent rather than absent. mendixlabs/mxcli#940 | +| `DESCRIBE PAGE` renders a List View as though it had one plain body, and re-executing that output **destroys** the page's specialization templates — measured on ako/TestApp: 4 templates before a describe → exec round trip, 0 after, with `mx check` reporting **0 errors both times** | `Forms$ListViewTemplate` was write-only and entity-less across the whole stack. `parseListViewContent` read only `w["Widgets"]`, so DESCRIBE never saw the sibling `Templates` array; the typed parser never populated `ListView.Templates` either, so a rebuild wrote zero; `sdk/pages.ListViewTemplate` had no entity field at all; and `modelsdk/gen` bound the entity under the SDK name `Specialization` where every Studio Pro document stores `Entity` | `mdl/executor/cmd_pages_describe_parse.go` (`parseListViewTemplates`), `cmd_pages_describe_output.go`, `mdl/executor/cmd_pages_builder_v3_widgets.go` (`buildListViewTemplateV3`), `sdk/pages/pages_widgets_data.go`, `sdk/mpr/writer_widgets_display.go`, `modelsdk/gen/pages/types.go` | Add the MDL surface (`template for Module.Entity { … }`) and read the array on the way back out. Four traps. (1) **`modelsdk/gen` is wrong about the key** — `Entity`, not `Specialization`; patch `init` AND `InitFromRaw`, per the STORAGE-NAME OVERRIDE rule. (2) **Order is authored, not derived** — TestApp's is Bus, Truck, Car, SUV, so a DESCRIBE that sorted would not round-trip. (3) **Inside a template the context object is the specialization**, not the list view's entity, or an attribute only the specialization has fails to resolve. (4) **`FOR` is in the parser's `keyword` rule**, so the new alternative must come FIRST in `widgetV3` or it collides with Gallery's pre-existing `template ` content slot — which is a different construct and must keep working. **Generalisable**: `mx check` is no signal for a dropped container — the document stays valid, it just holds less. Control on a **round trip through the real project** with the previous binary (`4 → 0`), not on the validator. mendixlabs/mxcli#940 | +| `describe page` emits MDL that will not re-execute for any non-database datasource. Three shapes: a chart series bound to a microflow comes back as `DataSource: database from Module.TheMicroflow` (→ `entity not found: Module.TheMicroflow`); a pluggable list whose source carries no entity comes back as `DataSource: database from ` with an empty slot (→ `entity not found: from`, the parser reading `from` as the entity); and a **gallery** bound over an association loses its datasource **entirely** while a **listview** bound the same way keeps it | One cause, three faces: the datasource switch was copied per widget family — five copies on the read side (`extractDataViewDataSource`, `extractListViewDataSource`, `extractDataGrid2DataSource`, `extractGalleryDataSource`, `parseCustomWidgetDataSource`) and six on the write side — and they drifted apart. The pluggable read copy, which every Gallery/DataGrid2/chart goes through, knew `Forms$MicroflowSource` and `Forms$NanoflowSource` but **not** `Forms$AssociationSource`, `Forms$ListenTargetSource` or the context forms, so those were dropped; its `CustomWidgets$CustomWidgetXPathSource` case returned `{database, ""}` unconditionally where its own near-twin guarded on a non-empty reference; and the object-list item emitter had **no type switch at all** | new `mdl/executor/cmd_pages_describe_datasource.go` (`parseDataSource`, `dataSourceExpr`, `appendDataSourceProp`); the five readers in `cmd_pages_describe_parse.go` / `cmd_pages_describe_pluggable.go` now delegate; all six emit sites in `cmd_pages_describe_output.go` go through one renderer | **One reader and one renderer, because a datasource means the same thing wherever it sits.** The per-widget copies were the defect, not the individual missing cases — patching the pluggable switch alone would have left five copies to drift again. Take the `$Type` set from `generated/metamodel`'s `DataSource` interface rather than from the copies. **Read sorting from whichever shape is present** (`SortBar.SortItems` for grid-like sources, `Sort.Paths` for list views) so the reader stays container-agnostic. **Distinguish "unknown type" from "known type, empty payload"**: the first is mxcli's gap and is reported as a `-- DataSource (…) has no MDL spelling` comment (never dropped, never guessed into a `database from` that cannot re-execute); the second is the model's own incompleteness and yields nil, which an existing test deliberately pinned (`TestExtractDataViewDataSource_ListenTargetSourceEmptyTarget`) — collapsing the two broke it. **Verify with a whole-project describe diff, not a unit test alone**: running `describe page` over every page before and after showed the four intended changes and nothing else — and it surfaced two WHERE/SORT clauses the combobox emitter had been silently dropping, which no unit test would have found. Controls: stub `dataSourceExpr`'s switch to `"database"` and the tests report the issue's own string, `got "database from Mod.DS_GetItems"`; drop the association/selection cases and they report `datasource was dropped`. **Note what the fix does NOT fix**: 10 of 21 pages in the probe project still fail `describe`→`check`, identically before and after — unquoted XPath containing `[`, duplicate generated widget names, context errors. Those are separate round-trip gaps; counting the failures on both sides is what tells them apart from a regression. **Generalisable measurement trap, and it invalidated this fix's first control**: copying a project for an `mx check` baseline with `cp rp.mpr mprcontents/` is NOT a copy of the project. The theme lives in `theme/`, so a copy without it reports **924 CE6083** "Design property … is not supported by your theme" plus CE0535 for the missing `widgets/` — a baseline of ~950 errors where the real project has **0**. Before/after still compares equal, so the check looks like it passed, but it has no power: a genuine new error is one line in a thousand and the theme-dependent checks never ran at all. Copy the whole directory (`cp -r project/. dest/`) and expect the baseline to be 0 — a non-zero baseline on a healthy project means the copy is incomplete, not that the project is broken. Repro `mdl-examples/bug-tests/941-describe-page-datasources.mdl`; tests `mdl/executor/cmd_pages_describe_datasource_test.go`. Issue #941 | +| `DESCRIBE PAGE` output that **used to parse stops parsing** after a fix that restores a WHERE clause — reported against stock `Administration.Account_New`, which every project from the Mendix template carries | Two older gaps the restored clause collided with. The constraint was emitted **raw** (`expr += " where " + xpath`), so a stored XPath carrying an inner predicate — `System.grantableRoles[reversed()]/…` — ended the MDL `expression` parse at the `[`. The grammar takes either a bare expression or a bracketed `xpathConstraint`, and only the second is safe for arbitrary XPath. The same six lines also stripped the outer brackets by testing the first and last byte, turning `[a][b]` into `a][b`, and guarded on `strings.Trim(s, "")` — an empty cutset trims nothing, so a whitespace-only constraint emitted a bare `where` | `mdl/executor/cmd_pages_describe_datasource.go` (`xpathConstraintClause`) | **Emit the bracketed production always, and stop deciding.** The bare form reads better but parses only for simple comparisons, and "is this XPath also a valid MDL expression" is not a question an emitter can answer — a bracketed group is accepted for every constraint. Reuse `visitor.SplitXPathPredicateGroups` (the quote- and nesting-aware splitter written for #772) rather than keeping a second, simpler copy: the naive strip is exactly the bug that helper exists to prevent. **Measure the round trip as a SET, not a count.** The claim this regression contradicted came from counting failing pages before and after, seeing 10 both times, and concluding nothing had changed — but an equal count hides a page flipping each way, and here a template page flipped from parse to fail while the total held. `join` the pass-sets: `join <(grep PASS before) <(grep FAIL after)` names the regressions directly. Control: revert the bracketing and `Account_New` stops parsing again. Reported in mxcli-formula1 §57.1/§57.3 | +| A widget datasource bound to a microflow **loses its arguments** through `describe`: `microflow Mod.DS(Race: $Race)` describes as `microflow Mod.DS`, and replaying that description fails the build with **CE1571** "No argument has been selected for parameter 'Race' and no default is available" — the very error #835 fixed on the write side. `check --references` calls the argument-less form valid, so nothing catches it before mxbuild | The write side stores the mappings correctly; **no describe path ever read them**. Not confined to pluggable widgets — a plain `dataview` loses them too. Older than #941, which merely printed the whole line wrong so the loss hid behind a larger bug | `mdl/executor/cmd_pages_describe_datasource.go` (`flowSourceArgs`, and the `case "microflow", "nanoflow"` renderer) | Storage puts the bindings in `Settings.ParameterMappings`, each `{Parameter, Expression}` behind typed-array marker 3, with **Parameter qualified by the flow** (`Mod.DS_Filtered.Term`) while MDL names the parameter alone — the flow is already left of the parentheses. **Strip that prefix by matching the flow's own qualified name, not by cutting at the last dot**: a document storing the name bare would otherwise lose its first segment. The value is in `Expression` for both a variable ref and a literal; `Variable` is the older spelling and is honoured when present. A parameterless flow must render **without** parentheses — the grammar makes the list optional and empty parens would churn every existing description. **Get the BSON shape from a real unit, not from the writers**: the two serializers here disagree in spelling (gen's `SetParameterQualifiedName` vs legacy's `Parameter`), and only a dump settles which key is on disk. Verify by replaying a description and counting CE1571 (4 → 0 on a two-widget page), not by eyeballing the text. Reported in mxcli-formula1 §57.2; same family as §39, where `SHOW_PAGE` drops `(Race: $currentObject)` | +| `use building block X (datasource: …)` starts failing with "datasource override supplied, but the block has no datasource widget to rebind" — on both engines, for a stock Atlas block, immediately after a fix that made DESCRIBE stop emitting malformed datasources | The rebind expands the block by **re-rendering it to MDL through the describe emitter and re-parsing that text**, then looked for a widget already carrying a `DataSource` property. A reusable block is a *template*: its datasource is unbound. That used to render as the malformed `DataSource: database from ,` — the exact output #941 fixed — and the re-parse turned it into a `DataSource` property, so the rebind found its target. Emitting nothing for an empty datasource, which is correct, left it with nothing to match | `mdl/executor/cmd_pages_builder_v3.go` (`isDataSourceWidget`, and the `DataSourceOverride` branch) | Match the target by **widget type — what can carry a datasource, not what does**. The sibling `ActionOverride` branch already did exactly this, and its comment says why: Atlas blocks "ship placeholder buttons with no action". Same fact about the same blocks; the datasource branch just hadn't learned it. **The generalisable shape**: a consumer that re-parses another component's *rendered output* silently depends on that output's bugs, so fixing a renderer can break something that never reads the model. Grep for callers of the emitter (`outputWidgetMDLV3` here) before changing what it emits — unit tests on the emitter cannot see this, and the failure surfaced only in the integration suite. Control: restore the property-based predicate and the stock-block rebind fails again. Tests `mdl/executor/cmd_pages_builder_bb_rebind_test.go`; CI repro `mdl-examples/doctype-tests/15c-fragment-bindings-examples.mdl` | diff --git a/.claude/skills/mendix/alter-page.md b/.claude/skills/mendix/alter-page.md index 87bc8d972..2ab9a2615 100644 --- a/.claude/skills/mendix/alter-page.md +++ b/.claude/skills/mendix/alter-page.md @@ -55,6 +55,46 @@ For changes that should be applied across **many pages** (e.g., "add `Class='car ## Operations +### List View Specialization Templates + +A List View template has no name, so it cannot be reached by a widget ref like +every other target. Adding one reuses `INSERT INTO` with the same +`template for` block `create page` uses — a template has one spelling +everywhere. Removing one has its own form: + +```sql +alter page Pages.Vehicle_Overview { + insert into vehicleListView { + template for Pages.Motorcycle { + dynamictext mcLabel (content: 'Motorcycle {1}', contentparams: [{1} = Brand]) + } + }; + drop template for Pages.SUV in vehicleListView +}; +``` + +Naming the list view in the `drop` is required, not optional: one page can hold +two list views with a template for the same entity. + +Most template edits need none of this. The widgets **inside** a template are +ordinary named widgets, so `set content = '…' on busLabel` and +`insert after busLabel { … }` already work and land in the right template. To +replace a whole template, `drop` it and `insert` the new one in the same block — +operations apply in order. + +Refused, each naming the problem: + +- `insert before` / `insert after` a template — templates are not siblings of the + widgets in the list view's body, so only `insert into` makes sense. +- mixing `template for …` blocks with ordinary widgets in one `insert` — they go + to different places (the Templates array and the default body). Use two inserts. +- a template for an entity that is not the list view's entity or a specialization + of it — it could never match an object the list view shows. +- a second template for an entity that already has one. +- `drop template for` an entity with no template — the error names the ones that + are there, because dropping nothing and reporting success is how a typo becomes + a silent no-op. + ### SET - Modify Widget Properties ```sql diff --git a/.claude/skills/mendix/create-page.md b/.claude/skills/mendix/create-page.md index 720422c50..967f8f8e2 100644 --- a/.claude/skills/mendix/create-page.md +++ b/.claude/skills/mendix/create-page.md @@ -348,6 +348,40 @@ actionbutton btnNew (caption: 'New', action: create_object Module.Product then s **Using `$currentObject`:** Use `$currentObject` inside DATAGRID, LISTVIEW, or GALLERY columns to reference the current row's object. This is typically used in columns with `ShowContentAs: customContent` for action buttons. +### LISTVIEW Specialization Templates + +A List View over a **generalization** can render a different body per +specialization. The template is identified by the entity it renders — it has no +name, which is why the keyword takes `for` and a qualified entity: + +```sql +listview vehicleListView (datasource: database from Pages.Vehicle) { + -- the default body: used for an object no template matches + dynamictext defaultVehicle (content: '{1} {2}', contentparams: [{1} = Brand, {2} = Model]) + + template for Pages.Bus { + dynamictext busLabel (content: 'Bus, capacity {1}', contentparams: [{1} = PassengerCapacity]) + } + template for Pages.Truck { + dynamictext truckLabel (content: 'Truck, max load {1} kg', contentparams: [{1} = MaxLoadKg]) + } +} +``` + +Rules: + +- The entity must be the list view's entity **or a specialization of it**. A + template for an unrelated entity can never match, so it is refused. +- **At most one template per entity.** +- Templates keep their **source order** — Mendix stores and matches in that + order, so it is authored, not derived, and DESCRIBE emits them as stored. +- Inside a template the context object **is the specialization**, so an + attribute only that specialization has resolves there (`PassengerCapacity` on + `Pages.Bus` above, which `Pages.Vehicle` does not have). + +Do not confuse this with a **Gallery's** `template `, which is a named +content slot, not a per-specialization body. + ### LINKBUTTON Widget Similar to ActionButton but rendered as link: diff --git a/.claude/skills/mendix/custom-widgets.md b/.claude/skills/mendix/custom-widgets.md index e45d6e0f2..454aceac6 100644 --- a/.claude/skills/mendix/custom-widgets.md +++ b/.claude/skills/mendix/custom-widgets.md @@ -89,6 +89,11 @@ pluggablewidget 'com.mendix.widget.web.barchart.BarChart' chart1 { } ``` +A series datasource takes any of the usual kinds — `database from …`, +`microflow …`, `nanoflow …`, `$Param`, `selection …` — not just `database`. +(Before #941 `describe page` rendered every series datasource as `database +from`, so a microflow-backed series described back as a missing entity.) + **Pie / HeatMap bind at the WIDGET level** (no series block). Both need `DataSource:` + `ValueAttribute:`; Pie also needs a required `SeriesName:`; HeatMap adds `scalecolor` items: diff --git a/.claude/skills/mendix/json-structures-and-mappings.md b/.claude/skills/mendix/json-structures-and-mappings.md index e0cd91b10..3e9a379c9 100644 --- a/.claude/skills/mendix/json-structures-and-mappings.md +++ b/.claude/skills/mendix/json-structures-and-mappings.md @@ -598,6 +598,29 @@ end; --- +## Placing Documents in Folders + +Every one of these documents takes a `folder` clause on `create`, straight after +the qualified name. Missing folders in the path are created: + +```mdl +create json structure Sales.JSON_Order folder 'Private/JSON structures' + snippet '{"id": 1, "total": 9.99}'; + +create import mapping Sales.IMM_Order folder 'Private/Import mappings' + with json structure Sales.JSON_Order +{ + create Sales.Order { OrderId = id, Total = total } +}; +``` + +On `create or modify` the clause **moves** an existing document. Omitting it +leaves placement alone — it never returns a document to the module root — so +adding a folder to an existing script is safe and removing one is a no-op. +`describe` emits the clause, so a description replays into the same folder. + +See `organize-project.md` for `move` and the full folder story. + ## Common Mistakes | Mistake | Fix | diff --git a/.claude/skills/mendix/manage-security.md b/.claude/skills/mendix/manage-security.md index 06576b773..1662e37fe 100644 --- a/.claude/skills/mendix/manage-security.md +++ b/.claude/skills/mendix/manage-security.md @@ -38,7 +38,8 @@ show demo users; -- Access on specific elements show access on microflow MyModule.ProcessOrder; show access on page MyModule.CustomerOverview; -show access on MyModule.Customer; +show access on entity MyModule.Customer; +show access on MyModule.Customer; -- a bare name means the entity -- Full security matrix show security matrix; diff --git a/.claude/skills/mendix/organize-project.md b/.claude/skills/mendix/organize-project.md index 64929037f..f1d989863 100644 --- a/.claude/skills/mendix/organize-project.md +++ b/.claude/skills/mendix/organize-project.md @@ -202,25 +202,58 @@ move page OldModule.CustomerPage to NewModule; ## Supported Document Types -| Document Type | FOLDER on Create | MOVE Command | -|---------------|-----------------|--------------| -| Page | `folder: 'path'` (property) | `move page ...` | -| Microflow | `folder 'path'` (keyword) | `move microflow ...` | -| Nanoflow | `folder 'path'` (keyword) | `move nanoflow ...` | -| Snippet | `folder: 'path'` (property) | `move snippet ...` | -| Enumeration | N/A | `move enumeration ...` | -| Constant | N/A | `move constant ...` | -| Database connection | N/A | `move database connection ...` | -| Java action | N/A | `move java action ...` | -| OData service (published) | N/A | `move odata service ...` | -| Entity | N/A | `move entity ...` (module only, no folders) | - -**Java actions and published OData services have no folder clause on `create`**, so -`move` is the only way to place them — before this they were stuck at the module -root forever. Both are plain document units, so the move is model-level only: it -changes containment and nothing else. - -**Note:** Pages and snippets use property syntax (`folder: 'path'` inside parentheses). Microflows and nanoflows use keyword syntax (`folder 'path'` before `begin`). Entities are embedded in domain models and can only be moved to a different module (no folder support). +`move` accepts **every top-level document type**, spelled the way `describe` +spells it: + +| Group | Types | +|-------|-------| +| Pages | `page`, `snippet`, `building block`, `layout`, `menu` | +| Logic | `microflow`, `nanoflow`, `workflow`, `queue`, `scheduled event` | +| Domain | `enumeration`, `constant`, `regular expression` | +| Mappings | `json structure`, `import mapping`, `export mapping` | +| Code | `java action`, `javascript action`, `database connection`, `data transformer` | +| Resources | `image collection`, `icon collection` | +| Integration | `rest client`, `published rest service`, `odata client`, `odata service`, `business event service` | +| AI | `model`, `agent`, `knowledge base`, `consumed mcp service` | + +`move entity` is the exception: an entity lives inside a domain model, so it +moves between **modules** only, never into a folder. + +If the named document turns out to be a different type, the statement is refused +and the error names what it really is — `move queue Mod.JSON_Order` reports that +`Mod.JSON_Order` is a json structure. + +### FOLDER on Create + +Every document type takes a folder clause on `create`, so a document can be +placed in the statement that creates it rather than in a separate `move`. Where +the clause goes depends on the statement's shape: + +| Document Type | FOLDER on Create | +|---------------|-----------------| +| Page, Snippet | `folder: 'path'` — a property, inside the parentheses | +| Microflow, Nanoflow | `folder 'path'` — a keyword, before `begin` | +| Enumeration, Constant | `folder 'path'` — a keyword, after the definition | +| Everything else | `folder 'path'` — a keyword, straight after the qualified name | + +```mdl +create import mapping CRM.IMM_Order folder 'Private/Import mappings' + with json structure CRM.JSON_Order { create CRM.Order { Id = id } }; + +create queue CRM.Q_Orders folder 'Private/Queues' ( Parallelism: 3 ); + +create java action CRM.JA_Sync folder 'Private/Java' () returns string + as $$return null;$$; +``` + +**A folder clause on `create or modify` moves an existing document.** It used to +be silently ignored: the statement reported success, the folder was created, and +the document stayed where it was (#932). Omitting the clause leaves placement +alone — it never returns a document to the module root — so adding a folder to a +script is safe and removing it is a no-op. + +`describe` emits the clause, so a description replays into the same folder +rather than into the module root. ## Example: Reorganize a Module diff --git a/.claude/skills/mendix/scheduled-events-and-queues.md b/.claude/skills/mendix/scheduled-events-and-queues.md index 40ba31c43..65f3deceb 100644 --- a/.claude/skills/mendix/scheduled-events-and-queues.md +++ b/.claude/skills/mendix/scheduled-events-and-queues.md @@ -223,6 +223,20 @@ Core.userActionCall("Ops.RefreshData") Use `Core.microflowCall(...)` when the unit of work really is a microflow. +## Placing Them in Folders + +Both take a `folder` clause on `create`, straight after the qualified name: + +```mdl +create scheduled event Ops.SE_Nightly folder 'Private/Scheduled events' + ( Microflow: Ops.ACT_Nightly, Repeat: Day, StartDateTime: '2026-01-01T02:00:00Z' ); + +create queue Ops.Q_Imports folder 'Private/Queues' ( Parallelism: 3 ); +``` + +On `create or modify` the clause moves an existing document; omitting it leaves +placement alone. See `organize-project.md`. + ## Validation Checklist Before presenting a script: diff --git a/.claude/skills/mendix/write-microflows.md b/.claude/skills/mendix/write-microflows.md index d5012fd07..e629f1fd1 100644 --- a/.claude/skills/mendix/write-microflows.md +++ b/.claude/skills/mendix/write-microflows.md @@ -571,18 +571,32 @@ end case; Use `split type` when a microflow branches on an object's runtime specialization. Use `cast` inside a type branch to create the specialized variable used by the branch body. +Branches are `when then`, the same as an enumeration split — one +statement, two subjects. + ```mdl declare $IsSpecialized boolean = false; split type $Input -case Sample.SpecializedInput - cast $SpecificInput; - set $IsSpecialized = true; -case Sample.BaseInput + when Sample.SpecializedInput then + cast $SpecificInput; + set $IsSpecialized = true; + when Sample.BaseInput then + when (empty) then end split; return $IsSpecialized; ``` -`case` values are qualified entity names. +Branch values are qualified entity names. + +> **`when (empty) then` is the null-object branch, not a default.** It is +> Mendix's `(empty)` outgoing flow, taken when the split variable is empty. It +> does **not** cover types you did not name — see the CE0090 note below. +> +> **The older spelling still works.** `case Sample.SpecializedInput` (no `then`) +> and `else` for the empty branch parse and build the identical flow, but warn +> **MDL065**. `case` introduced a *branch* here while introducing the *subject* +> in `case $x when V then` and in expressions, so the word meant two things +> (mxcli #913); `else` read as a default and never was one. > **Every type needs a branch — including the base entity.** An object-type > decision gets one outgoing flow per listed type, and a type with no flow fails @@ -590,10 +604,15 @@ return $IsSpecialized; > flow."* The base entity (the split variable's own type) counts: `case > Sample.BaseInput` above is what covers "it is not any of the specializations". > -> **`else` does not stand in for the base-type case.** It is accepted — it -> serializes as `Microflows$NoCase` — but it does not satisfy coverage, so -> `case Spec` + `else` still fails CE0090. Once every type has a branch, `else` -> is redundant. Verified on Mendix 11.6.6 and 11.13.0. +> **The `(empty)` branch does not stand in for the base-type case.** It is +> accepted — it serializes as `Microflows$NoCase` — but it does not satisfy +> coverage, so one named type plus `when (empty) then` still fails CE0090. +> Measured on 11.13.0: `when Zoo.Dog then` + an empty branch gives CE0090 for +> `Zoo.Cat` **and** `Zoo.Animal`; branch on every type, keep the empty branch, +> and it is 0 errors. Verified on Mendix 11.6.6 and 11.13.0. +> +> You cannot drop the empty branch either — that is **CE0089**. mxcli emits the +> flow unconditionally for that reason. > > **The split needs somewhere to go afterwards.** Branch bodies converge on a > merge that continues to the microflow's end event, so a non-void microflow diff --git a/cmd/mxcli/syntax/features_integration.go b/cmd/mxcli/syntax/features_integration.go index b4583a8ff..a67aa2fef 100644 --- a/cmd/mxcli/syntax/features_integration.go +++ b/cmd/mxcli/syntax/features_integration.go @@ -351,7 +351,7 @@ func init() { "jdbc", "byod", "database connector", "execute database query", "postgresql", "mysql", "oracle", "snowflake", "sql server", }, - Syntax: `CREATE [OR MODIFY] DATABASE CONNECTION Module.Name + Syntax: `CREATE [OR MODIFY] DATABASE CONNECTION Module.Name [FOLDER 'path'] TYPE '' CONNECTION STRING @Module.UrlConstant USERNAME @Module.UserConstant @@ -508,7 +508,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "java action", "java", "call java", "type parameter", "exposed as", "javaaction", }, - Syntax: "SHOW JAVA ACTIONS [IN Module];\nDESCRIBE JAVA ACTION Module.Name;\nCREATE [OR MODIFY] JAVA ACTION Module.Name(...) RETURNS Type AS $$ ... $$;\nDROP JAVA ACTION Module.Name;\n\nNOTE: AS $$ ... $$ is mandatory — omitting the body causes a parse error.", + Syntax: "SHOW JAVA ACTIONS [IN Module];\nDESCRIBE JAVA ACTION Module.Name;\nCREATE [OR MODIFY] JAVA ACTION Module.Name [FOLDER 'path'](...) RETURNS Type AS $$ ... $$;\nDROP JAVA ACTION Module.Name;\n\nNOTE: AS $$ ... $$ is mandatory — omitting the body causes a parse error.", Example: "SHOW JAVA ACTIONS;\nDESCRIBE JAVA ACTION Utils.FormatCurrency;", SeeAlso: []string{"java-action.create"}, }) @@ -520,7 +520,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "create java action", "or modify java action", "type parameter", "entity parameter", "exposed as", "returns", "generics", "drop java action", }, - Syntax: "CREATE [OR MODIFY] JAVA ACTION Module.Name(\n Param: Type [NOT NULL],\n EntityType: ENTITY NOT NULL,\n Obj: pEntity\n) RETURNS ReturnType\n[EXPOSED AS 'Label' IN 'Category']\nAS $$\n// Java code — AS $$ ... $$ is mandatory, cannot be omitted\n$$;\n\nOR MODIFY: updates signature/body in-place, preserves UUID.", + Syntax: "CREATE [OR MODIFY] JAVA ACTION Module.Name [FOLDER 'path'](\n Param: Type [NOT NULL],\n EntityType: ENTITY NOT NULL,\n Obj: pEntity\n) RETURNS ReturnType\n[EXPOSED AS 'Label' IN 'Category']\nAS $$\n// Java code — AS $$ ... $$ is mandatory, cannot be omitted\n$$;\n\nOR MODIFY: updates signature/body in-place, preserves UUID.", Example: "CREATE JAVA ACTION Utils.FormatCurrency(\n Amount: Decimal NOT NULL\n) RETURNS String\nEXPOSED AS 'Format Currency' IN 'Formatting'\nAS $$\nreturn String.format(\"%.2f\", Amount);\n$$;\n\n-- Generic entity validator with type parameter\nCREATE JAVA ACTION Utils.IsValid(\n EntityType: ENTITY NOT NULL,\n Obj: pEntity NOT NULL\n) RETURNS Boolean\nAS $$\nreturn Obj != null;\n$$;\n\n-- Idempotent update (preserves UUID)\nCREATE OR MODIFY JAVA ACTION Utils.FormatCurrency(\n Amount: Decimal NOT NULL,\n Decimals: Integer NOT NULL\n) RETURNS String\nAS $$\nreturn String.format(\"%.\" + Decimals + \"f\", Amount);\n$$;", SeeAlso: []string{"java-action", "javascript-action"}, }) @@ -532,7 +532,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "javascript action", "javascript", "js action", "call javascript", "platform", "exposed as", "javascriptaction", }, - Syntax: "SHOW JAVASCRIPT ACTIONS [IN Module];\nDESCRIBE JAVASCRIPT ACTION Module.Name;\nCREATE [OR MODIFY] JAVASCRIPT ACTION Module.Name(...) RETURNS Type [PLATFORM Web|Native|Hybrid|All] AS $$ ... $$;\nDROP JAVASCRIPT ACTION Module.Name;\n\nWrites the unit plus javascriptsource//actions/.js. AS $$ ... $$ is mandatory. PLATFORM defaults to Web.", + Syntax: "SHOW JAVASCRIPT ACTIONS [IN Module];\nDESCRIBE JAVASCRIPT ACTION Module.Name;\nCREATE [OR MODIFY] JAVASCRIPT ACTION Module.Name [FOLDER 'path'](...) RETURNS Type [PLATFORM Web|Native|Hybrid|All] AS $$ ... $$;\nDROP JAVASCRIPT ACTION Module.Name;\n\nWrites the unit plus javascriptsource//actions/.js. AS $$ ... $$ is mandatory. PLATFORM defaults to Web.", Example: "CREATE JAVASCRIPT ACTION Utils.IsStrictMode() RETURNS Boolean\nPLATFORM Web\nAS $$\nreturn Promise.resolve((function(){ return !this; })());\n$$;\n\n-- Exposed, native, with parameters\nCREATE JAVASCRIPT ACTION Utils.ShowToast(\n Message: String NOT NULL,\n Duration: Integer\n) RETURNS Boolean\nEXPOSED AS 'Show Toast' IN 'UI'\nPLATFORM Native\nAS $$\nreturn Promise.resolve(true);\n$$;", SeeAlso: []string{"java-action"}, }) @@ -546,7 +546,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "json structure", "create json structure", "drop json structure", "snippet", "schema", "json schema", }, - Syntax: "SHOW JSON STRUCTURES [IN Module];\nDESCRIBE JSON STRUCTURE Module.Name;\nCREATE JSON STRUCTURE Module.Name [COMMENT 'text'] SNIPPET '{ ... }';\nCREATE OR MODIFY JSON STRUCTURE Module.Name SNIPPET '{ ... }';\nDROP JSON STRUCTURE Module.Name;", + Syntax: "SHOW JSON STRUCTURES [IN Module];\nDESCRIBE JSON STRUCTURE Module.Name;\nCREATE JSON STRUCTURE Module.Name [FOLDER 'path'] [COMMENT 'text'] SNIPPET '{ ... }';\nCREATE OR MODIFY JSON STRUCTURE Module.Name SNIPPET '{ ... }';\nDROP JSON STRUCTURE Module.Name;", Example: "CREATE OR MODIFY JSON STRUCTURE MyModule.JSON_Pet\n SNIPPET '{\"id\": 1, \"name\": \"Fido\", \"status\": \"available\"}';\n\nDESCRIBE JSON STRUCTURE MyModule.JSON_Pet;", SeeAlso: []string{"import-mapping", "export-mapping"}, }) @@ -560,7 +560,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "image collection", "create image collection", "drop image collection", "export level", "image", "icon", "logo", }, - Syntax: "SHOW IMAGE COLLECTION [IN Module];\nDESCRIBE IMAGE COLLECTION Module.Name;\nCREATE IMAGE COLLECTION Module.Name\n [EXPORT LEVEL 'Hidden'|'Public']\n [COMMENT 'text']\n [(IMAGE name FROM FILE 'path', ...)];\nCREATE OR MODIFY IMAGE COLLECTION Module.Name [...];\nDROP IMAGE COLLECTION Module.Name;", + Syntax: "SHOW IMAGE COLLECTION [IN Module];\nDESCRIBE IMAGE COLLECTION Module.Name;\nCREATE IMAGE COLLECTION Module.Name [FOLDER 'path']\n [EXPORT LEVEL 'Hidden'|'Public']\n [COMMENT 'text']\n [(IMAGE name FROM FILE 'path', ...)];\nCREATE OR MODIFY IMAGE COLLECTION Module.Name [...];\nDROP IMAGE COLLECTION Module.Name;", Example: "CREATE OR MODIFY IMAGE COLLECTION MyModule.AppIcons\n EXPORT LEVEL 'Public'\n COMMENT 'Application icons' (\n IMAGE logo FROM FILE 'assets/logo.png',\n IMAGE \"favicon\" FROM FILE 'assets/favicon.ico'\n);\n\nDESCRIBE IMAGE COLLECTION MyModule.AppIcons;", SeeAlso: []string{"integration", "icon-collection"}, }) @@ -588,7 +588,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "with json structure", "find or create", "object handling", }, Syntax: "SHOW IMPORT MAPPINGS [IN Module];\nDESCRIBE IMPORT MAPPING Module.Name;\n" + - "CREATE [OR MODIFY] IMPORT MAPPING Module.Name\n WITH JSON STRUCTURE Module.JsonStruct\n{\n" + + "CREATE [OR MODIFY] IMPORT MAPPING Module.Name [FOLDER 'path']\n WITH JSON STRUCTURE Module.JsonStruct\n{\n" + " create|find|find or create Module.Entity {\n Attr = jsonField [KEY],\n" + " Attr = a/b/c,\n" + " Assoc/Module.Child = nestedKey { ... }\n }\n};\nDROP IMPORT MAPPING Module.Name;\n\n" + @@ -616,7 +616,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "show export mappings", "describe export mapping", "with json structure", "null values", "as jsonKey", }, - Syntax: "SHOW EXPORT MAPPINGS [IN Module];\nDESCRIBE EXPORT MAPPING Module.Name;\nCREATE [OR MODIFY] EXPORT MAPPING Module.Name\n WITH JSON STRUCTURE Module.JsonStruct\n [NULL VALUES LeaveOutElement|SendAsNil]\n{\n Module.Entity {\n jsonField = Attr,\n Assoc/Module.Child AS nestedKey { ... }\n }\n};\nDROP EXPORT MAPPING Module.Name;\n\nOR MODIFY: updates mapping in-place, preserves UUID.\n\n" + + Syntax: "SHOW EXPORT MAPPINGS [IN Module];\nDESCRIBE EXPORT MAPPING Module.Name;\nCREATE [OR MODIFY] EXPORT MAPPING Module.Name [FOLDER 'path']\n WITH JSON STRUCTURE Module.JsonStruct\n [NULL VALUES LeaveOutElement|SendAsNil]\n{\n Module.Entity {\n jsonField = Attr,\n Assoc/Module.Child AS nestedKey { ... }\n }\n};\nDROP EXPORT MAPPING Module.Name;\n\nOR MODIFY: updates mapping in-place, preserves UUID.\n\n" + "No nested-member form:\n" + " An import mapping can write `Attr = a/b/c` to reach a leaf without an\n" + " entity per level. An export mapping cannot: it has to PRODUCE the\n" + @@ -636,7 +636,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "data transformer", "create data transformer", "drop data transformer", "list data transformers", "jslt", "xslt", "transform", }, - Syntax: "LIST DATA TRANSFORMERS [IN Module];\nDESCRIBE DATA TRANSFORMER Module.Name;\nCREATE [OR MODIFY] DATA TRANSFORMER Module.Name\n SOURCE JSON '{ ... }'\n{\n JSLT 'single-line-expression';\n -- or multi-line:\n JSLT $$\n{ ... }\n $$;\n};\nDROP DATA TRANSFORMER Module.Name;\n\nOR MODIFY: updates transformer in-place, preserves UUID.", + Syntax: "LIST DATA TRANSFORMERS [IN Module];\nDESCRIBE DATA TRANSFORMER Module.Name;\nCREATE [OR MODIFY] DATA TRANSFORMER Module.Name [FOLDER 'path']\n SOURCE JSON '{ ... }'\n{\n JSLT 'single-line-expression';\n -- or multi-line:\n JSLT $$\n{ ... }\n $$;\n};\nDROP DATA TRANSFORMER Module.Name;\n\nOR MODIFY: updates transformer in-place, preserves UUID.", Example: "CREATE DATA TRANSFORMER ETL.FlattenOrder\n SOURCE JSON '{\"order\": {\"id\": 1, \"total\": 99.0}}'\n{\n JSLT '{\"id\": .order.id, \"total\": .order.total}';\n};\n\n-- Multi-line JSLT\nCREATE OR MODIFY DATA TRANSFORMER ETL.WeatherSummary\n SOURCE JSON '{\"current\": {\"temp\": 12.8, \"wind\": 18.3}}'\n{\n JSLT $$\n{\n \"temperature\": .current.temp,\n \"wind_speed\": .current.wind\n}\n $$;\n};", SeeAlso: []string{"integration"}, }) @@ -659,7 +659,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, Path: "agents.model", Summary: "CREATE/DROP MODEL documents for AI agents", Keywords: []string{"create model", "drop model", "describe model", "list models", "provider", "mxcloudgenai"}, - Syntax: "CREATE [OR MODIFY] MODEL Module.Name (\n Provider: MxCloudGenAI,\n Key: Module.ApiKeyConstant\n);\nDESCRIBE MODEL Module.Name;\nLIST MODELS [IN Module];\nDROP MODEL Module.Name;", + Syntax: "CREATE [OR MODIFY] MODEL Module.Name [FOLDER 'path'] (\n Provider: MxCloudGenAI,\n Key: Module.ApiKeyConstant\n);\nDESCRIBE MODEL Module.Name;\nLIST MODELS [IN Module];\nDROP MODEL Module.Name;", Example: "create model MyModule.GPT4 (\n Provider: MxCloudGenAI,\n Key: MyModule.ModelApiKey\n);", SeeAlso: []string{"agents"}, }) @@ -668,7 +668,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, Path: "agents.knowledge-base", Summary: "CREATE/DROP KNOWLEDGE BASE documents for AI agents", Keywords: []string{"create knowledge base", "drop knowledge base", "knowledge base", "kb", "rag"}, - Syntax: "CREATE [OR MODIFY] KNOWLEDGE BASE Module.Name (\n Provider: MxCloudGenAI,\n Key: Module.KBApiKeyConstant\n);\nDESCRIBE KNOWLEDGE BASE Module.Name;\nLIST KNOWLEDGE BASES [IN Module];\nDROP KNOWLEDGE BASE Module.Name;", + Syntax: "CREATE [OR MODIFY] KNOWLEDGE BASE Module.Name [FOLDER 'path'] (\n Provider: MxCloudGenAI,\n Key: Module.KBApiKeyConstant\n);\nDESCRIBE KNOWLEDGE BASE Module.Name;\nLIST KNOWLEDGE BASES [IN Module];\nDROP KNOWLEDGE BASE Module.Name;", Example: "create knowledge base MyModule.ProductDocs (\n Provider: MxCloudGenAI,\n Key: MyModule.KBApiKey\n);", SeeAlso: []string{"agents"}, }) @@ -677,7 +677,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, Path: "agents.mcp-service", Summary: "CREATE/DROP CONSUMED MCP SERVICE documents for AI agents", Keywords: []string{"consumed mcp service", "mcp", "mcp service", "protocol version"}, - Syntax: "CREATE [OR MODIFY] CONSUMED MCP SERVICE Module.Name (\n ProtocolVersion: v2025_03_26,\n Version: '1.0',\n ConnectionTimeoutSeconds: 30,\n Documentation: 'description'\n);\nDESCRIBE CONSUMED MCP SERVICE Module.Name;\nLIST CONSUMED MCP SERVICES [IN Module];\nDROP CONSUMED MCP SERVICE Module.Name;", + Syntax: "CREATE [OR MODIFY] CONSUMED MCP SERVICE Module.Name [FOLDER 'path'] (\n ProtocolVersion: v2025_03_26,\n Version: '1.0',\n ConnectionTimeoutSeconds: 30,\n Documentation: 'description'\n);\nDESCRIBE CONSUMED MCP SERVICE Module.Name;\nLIST CONSUMED MCP SERVICES [IN Module];\nDROP CONSUMED MCP SERVICE Module.Name;", Example: "create consumed mcp service MyModule.WebSearch (\n ProtocolVersion: v2025_03_26,\n Version: '1.0',\n ConnectionTimeoutSeconds: 30\n);", SeeAlso: []string{"agents"}, }) @@ -689,7 +689,7 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, "create agent", "drop agent", "usagetype", "systemprompt", "userprompt", "variables", "toolchoice", "temperature", "topp", "maxtokens", }, - Syntax: `CREATE [OR MODIFY] AGENT Module.Name ( + Syntax: `CREATE [OR MODIFY] AGENT Module.Name [FOLDER 'path'] ( UsageType: Task|Chat, Model: Module.MyModel, [Description: 'text',] diff --git a/cmd/mxcli/syntax/features_microflow.go b/cmd/mxcli/syntax/features_microflow.go index 408987557..4211c86e8 100644 --- a/cmd/mxcli/syntax/features_microflow.go +++ b/cmd/mxcli/syntax/features_microflow.go @@ -79,7 +79,37 @@ func init() { }, Syntax: "IF condition THEN\n ...\nELSIF condition THEN\n ...\nELSE\n ...\nEND IF;\n\nLOOP $Item IN $List BEGIN ... END LOOP;\nWHILE condition BEGIN ... END WHILE;\nRETURN $Value;\nRETURN empty;", Example: "IF $Customer = empty THEN\n LOG ERROR NODE 'Svc' 'Not found';\n RETURN empty;\nELSIF $Customer/Active = false THEN\n LOG WARNING 'Inactive customer';\nELSE\n CHANGE $Customer (LastAccess = [%CurrentDateTime%]);\nEND IF;\n\nLOOP $Item IN $OrderLines BEGIN\n COMMIT $Item;\nEND LOOP;", - SeeAlso: []string{"microflow.variables", "microflow.error-handling"}, + SeeAlso: []string{"microflow.variables", "microflow.error-handling", "microflow.splits"}, + }) + + Register(SyntaxFeature{ + Path: "microflow.splits", + Summary: "CASE (enum split) and SPLIT TYPE (object type split)", + Keywords: []string{ + "case", "when", "then", "end case", "enum split", "enumeration split", + "split type", "end split", "type split", "inheritance split", + "specialization", "cast", "empty", "switch", "decision", + }, + Syntax: "CASE $EnumVarOrAttr -- branches on an ENUMERATION\n" + + " WHEN Value1, Value2 THEN\n" + + " ...\n" + + " WHEN (empty) THEN -- required (MDL056 / CE0079)\n" + + " ...\n" + + "END CASE; -- no ELSE (MDL008)\n\n" + + "SPLIT TYPE $ObjectVar -- branches on the RUNTIME TYPE\n" + + " WHEN Module.Specialization THEN\n" + + " CAST $Specific;\n" + + " ...\n" + + " WHEN Module.BaseEntity THEN -- required too (CE0090)\n" + + " ...\n" + + " WHEN (empty) THEN -- the NULL-object branch, not a default\n" + + " ...\n" + + "END SPLIT;\n\n" + + "Both take `WHEN ... THEN` branches. `SPLIT TYPE` needs one per subtype AND\n" + + "the base entity; its `(empty)` branch is the null object and covers no type.\n" + + "Legacy `CASE Module.Entity` / `ELSE` inside SPLIT TYPE still parse (MDL065).", + Example: "CASE $Order/Status\n WHEN Draft, Submitted THEN\n LOG INFO 'Not shipped yet';\n WHEN Approved THEN\n LOG INFO 'Ready to ship';\n WHEN (empty) THEN\n LOG INFO 'No status';\nEND CASE;\n\nSPLIT TYPE $Animal\n WHEN Zoo.Dog THEN\n LOG INFO 'woof';\n WHEN Zoo.Cat THEN\n LOG INFO 'meow';\n WHEN Zoo.Animal THEN\n LOG INFO 'some other animal';\n WHEN (empty) THEN\n LOG INFO 'no animal at all';\nEND SPLIT;", + SeeAlso: []string{"microflow.control-flow", "microflow.variables"}, }) Register(SyntaxFeature{ diff --git a/cmd/mxcli/syntax/features_misc.go b/cmd/mxcli/syntax/features_misc.go index 1744c9b38..bf2c1a625 100644 --- a/cmd/mxcli/syntax/features_misc.go +++ b/cmd/mxcli/syntax/features_misc.go @@ -26,7 +26,52 @@ func init() { "-- Both reuse the existing element's ID, so references from other\n" + "-- documents survive.", Example: "CREATE OR REPLACE MICROFLOW MyModule.ACT_Recalculate ()\nBEGIN\n RETURN;\nEND;\n\nCREATE OR MODIFY PERSISTENT ENTITY MyModule.Customer (\n Name: String(200)\n);", - SeeAlso: []string{"microflow", "domain-model.entity", "page"}, + SeeAlso: []string{"microflow", "domain-model.entity", "page", "document-folder"}, + }) + + // The folder clause is the other cross-cutting CREATE modifier, and gets one + // topic for the same reason OR MODIFY does: it applies to every document + // type, so documenting it in all 27 places would guarantee 27 chances to + // drift. + Register(SyntaxFeature{ + Path: "document-folder", + Summary: "FOLDER — placing a document in a module folder as you create it", + Keywords: []string{ + "folder", "folder clause", "place document", "module folder", + "create in folder", "organise", "organize", "subfolder", + "folder on create", "folder ignored", "document did not move", + }, + Syntax: "-- Every document type takes a folder clause on CREATE. Where it goes\n" + + "-- depends on the statement's shape:\n" + + "-- Pages, snippets Folder: 'path' a property, inside the parentheses\n" + + "-- Microflows, nanoflows FOLDER 'path' a keyword, before BEGIN\n" + + "-- Everything else FOLDER 'path' a keyword, after the qualified name\n" + + "--\n" + + "-- Missing folders in the path are created. Nested paths use '/'.\n" + + "--\n" + + "-- On CREATE OR MODIFY the clause MOVES an existing document. Omitting it\n" + + "-- leaves placement alone — it never returns a document to the module\n" + + "-- root — so adding a folder to an existing script is safe, and removing\n" + + "-- one is a no-op. DESCRIBE emits the clause, so a description replays\n" + + "-- into the same folder.\n" + + "--\n" + + "-- To move a document without rewriting it, use MOVE.", + Example: "CREATE QUEUE MyModule.Q_Orders FOLDER 'Private/Queues' ( Parallelism: 3 );\n\n" + + "CREATE IMPORT MAPPING MyModule.IMM_Order FOLDER 'Private/Import mappings'\n" + + " WITH JSON STRUCTURE MyModule.JSON_Order {\n" + + " CREATE MyModule.Order { Id = id }\n" + + " };\n\n" + + "CREATE PAGE MyModule.OrderList\n" + + " (\n" + + " Title: 'Orders',\n" + + " Folder: 'Orders',\n" + + " Layout: Atlas_Core.Atlas_Default\n" + + " )\n" + + " {\n" + + " DYNAMICTEXT txtHeading (Content: 'Orders')\n" + + " };\n\n" + + "CREATE OR MODIFY MICROFLOW MyModule.ACT_Sync ()\nFOLDER 'Private/Jobs'\nBEGIN\n RETURN;\nEND;", + SeeAlso: []string{"move", "folders", "create-modifiers"}, }) // ── Connection ────────────────────────────────────────────────────── @@ -237,7 +282,7 @@ CREATE CONFIGURATION 'Production' "describe queue", "show queues", "parallelism", "cluster wide", "background", "async microflow", }, - Syntax: `CREATE [OR MODIFY] QUEUE Module.Name [( : , ... )]; + Syntax: `CREATE [OR MODIFY] QUEUE Module.Name [FOLDER 'path'] [( : , ... )]; SHOW QUEUES [IN ]; LIST QUEUES [IN ]; DESCRIBE QUEUE Module.Name; @@ -294,7 +339,7 @@ DROP QUEUE Ops.Mail;`, "create regular expression", "drop regular expression", "describe regular expression", "show regular expressions", "email regex", "match", }, - Syntax: `CREATE [OR MODIFY] REGULAR EXPRESSION Module.Name ( + Syntax: `CREATE [OR MODIFY] REGULAR EXPRESSION Module.Name [FOLDER 'path'] ( Expression: '', [Documentation: '',] [ExportLevel: Hidden|Public,] @@ -406,7 +451,7 @@ CREATE VALIDATION RULE FOR Shop.Product.Price "create scheduled event", "drop scheduled event", "describe scheduled event", "repeat", "daily", "hourly", "weekly", "monthly", "yearly", "timer", "batch job", }, - Syntax: `CREATE [OR MODIFY] SCHEDULED EVENT Module.Name ( : , ... ); + Syntax: `CREATE [OR MODIFY] SCHEDULED EVENT Module.Name [FOLDER 'path'] ( : , ... ); SHOW SCHEDULED EVENTS [IN ]; LIST SCHEDULED EVENTS [IN ]; DESCRIBE SCHEDULED EVENT Module.Name; @@ -514,10 +559,21 @@ SHOW STRUCTURE DEPTH 1 ALL;`, "move", "relocate", "folder", "cross-module move", "move page", "move microflow", "move entity", "move folder", "drop folder", + "move import mapping", "move export mapping", "move json structure", + "move queue", "move workflow", "move menu", "move layout", }, Syntax: `MOVE Module.Name TO FOLDER 'Path'; --- doctype: PAGE | MICROFLOW | NANOFLOW | SNIPPET | ENUMERATION | CONSTANT --- | DATABASE CONNECTION | JAVA ACTION | ODATA SERVICE | ENTITY | FOLDER +-- doctype: every top-level document, spelled as DESCRIBE spells it — +-- PAGE | SNIPPET | BUILDING BLOCK | LAYOUT | MENU +-- MICROFLOW | NANOFLOW | WORKFLOW | QUEUE | SCHEDULED EVENT +-- ENUMERATION | CONSTANT | REGULAR EXPRESSION +-- JSON STRUCTURE | IMPORT MAPPING | EXPORT MAPPING +-- JAVA ACTION | JAVASCRIPT ACTION | DATABASE CONNECTION | DATA TRANSFORMER +-- IMAGE COLLECTION | ICON COLLECTION +-- REST CLIENT | PUBLISHED REST SERVICE | ODATA CLIENT | ODATA SERVICE +-- BUSINESS EVENT SERVICE +-- MODEL | AGENT | KNOWLEDGE BASE | CONSUMED MCP SERVICE +-- plus ENTITY (moves between domain models) and FOLDER (moves a folder). MOVE Module.Name TO TargetModule; MOVE OldModule.Name TO FOLDER 'Path' IN NewModule; MOVE FOLDER Module.FolderName TO FOLDER 'Path'; @@ -531,10 +587,22 @@ MOVE MICROFLOW MyModule.ACT_ProcessOrder TO FOLDER 'Orders/Processing'; -- Move entity to different module MOVE ENTITY OldModule.Customer TO NewModule; --- Java actions and published OData services have no folder clause on CREATE, --- so MOVE is the only way to place them +-- Many doctypes have no folder clause on CREATE, so MOVE is the only way to +-- place them MOVE JAVA ACTION MyModule.ODataQuery TO FOLDER 'Support'; MOVE ODATA SERVICE MyModule.PublicApi TO FOLDER 'Api/Published'; +MOVE IMPORT MAPPING MyModule.IMM_Order TO FOLDER 'Private/Import mappings'; +MOVE JSON STRUCTURE MyModule.JSON_Order TO FOLDER 'Private/JSON structures'; + +-- A FOLDER clause on CREATE OR MODIFY moves an existing document too, so a +-- script can place a document without a separate MOVE. Every doctype accepts +-- one; on most it goes straight after the qualified name +CREATE OR MODIFY JSON STRUCTURE MyModule.JSON_Order + FOLDER 'Private/JSON structures' + SNIPPET '{"id": 1}'; +CREATE QUEUE MyModule.Q_Orders FOLDER 'Private/Queues' ( Parallelism: 3 ); +CREATE IMPORT MAPPING MyModule.IMM_Order FOLDER 'Private/Import mappings' + WITH JSON STRUCTURE MyModule.JSON_Order { CREATE MyModule.Order { Id = id } }; -- Check impact before cross-module move SHOW IMPACT OF OldModule.CustomerPage; diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index 0bcc019d9..988560c17 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -38,12 +38,57 @@ func init() { "combobox", "checkbox", "radiobuttons", "actionbutton", "dynamictext", "snippetcall", "navigationlist", "column", "row", "footer", "header", "controlbar", + "template", "specialization", "list view template", }, - Syntax: "-- Containers\nLAYOUTGRID name { ROW r { COLUMN c (DesktopWidth: 6) { ... } } }\nCONTAINER name (Class: 'cls') { ... }\nCONTAINER name (OnClick: MICROFLOW Module.MF) { ... } -- clickable container\n\n-- Data widgets\nDATAVIEW name (DataSource: $Param) { ... FOOTER f { ... } }\nDATAGRID name (DataSource: DATABASE Module.Entity) { COLUMN c (Attribute: A) }\nGALLERY name (DataSource: DATABASE Module.Entity, DesktopColumns: 3) { ... }\nLISTVIEW name (DataSource: DATABASE Module.Entity) { ... }\n\n-- Inputs\nTEXTBOX name (Label: 'L', Attribute: Attr)\nTEXTAREA | DATEPICKER | COMBOBOX | CHECKBOX | RADIOBUTTONS\n\n-- Actions\nACTIONBUTTON name (Caption: 'C', Action: SAVE_CHANGES, ButtonStyle: Primary)\n\n-- Display\nDYNAMICTEXT name (Content: 'Hello, {1}!', ContentParams: [{1} = Name])", + Syntax: "-- Containers\nLAYOUTGRID name { ROW r { COLUMN c (DesktopWidth: 6) { ... } } }\nCONTAINER name (Class: 'cls') { ... }\nCONTAINER name (OnClick: MICROFLOW Module.MF) { ... } -- clickable container\n\n-- Data widgets\nDATAVIEW name (DataSource: $Param) { ... FOOTER f { ... } }\nDATAGRID name (DataSource: DATABASE Module.Entity) { COLUMN c (Attribute: A) }\nGALLERY name (DataSource: DATABASE Module.Entity, DesktopColumns: 3) { ... }\nLISTVIEW name (DataSource: DATABASE Module.Entity) { ... }\nLISTVIEW name (...) { ... TEMPLATE FOR Module.Specialization { ... } }\n\n-- Inputs\nTEXTBOX name (Label: 'L', Attribute: Attr)\nTEXTAREA | DATEPICKER | COMBOBOX | CHECKBOX | RADIOBUTTONS\n\n-- Actions\nACTIONBUTTON name (Caption: 'C', Action: SAVE_CHANGES, ButtonStyle: Primary)\n\n-- Display\nDYNAMICTEXT name (Content: 'Hello, {1}!', ContentParams: [{1} = Name])", Example: "DATAVIEW dvCustomer (DataSource: $Customer) {\n TEXTBOX txtName (Label: 'Name', Attribute: Name)\n COMBOBOX cbStatus (Label: 'Status', Attribute: Status)\n FOOTER footer1 {\n ACTIONBUTTON btnSave (Caption: 'Save', Action: SAVE_CHANGES, ButtonStyle: Primary)\n ACTIONBUTTON btnCancel (Caption: 'Cancel', Action: CANCEL_CHANGES)\n }\n}", SeeAlso: []string{"page.create", "page.datasource"}, }) + Register(SyntaxFeature{ + Path: "page.listview-template", + Summary: "List View specialization templates: one body per specialization", + Keywords: []string{ + "template", "listview template", "list view template", "specialization", + "generalization", "inheritance", "template for", "per type", + }, + Syntax: "LISTVIEW name (DataSource: DATABASE Module.Base) {\n" + + " ...widgets... -- the default body, used when no template matches\n" + + " TEMPLATE FOR Module.Specialization { -- one body per specialization\n" + + " ...widgets...\n" + + " }\n" + + "}\n\n" + + "A template is identified by the entity it renders, not by a name — that is why it is\n" + + "TEMPLATE FOR Module.Entity and not TEMPLATE name. (A Gallery's TEMPLATE name is a\n" + + "different thing: a named content slot.)\n\n" + + "Rules:\n" + + " - the entity must be the list view's entity or a specialization of it\n" + + " - at most one template per entity\n" + + " - templates keep their source order, which is the order Mendix stores and matches in\n" + + " - inside a template the context object is the specialization, so its own attributes resolve\n\n" + + "ALTER PAGE — adding one reuses INSERT INTO with the same block, so a template has one\n" + + "spelling everywhere. Removing one needs its own form, because a template has no name:\n\n" + + "ALTER PAGE Module.Page {\n" + + " INSERT INTO listViewName { TEMPLATE FOR Module.Specialization { ...widgets... } };\n" + + " DROP TEMPLATE FOR Module.Specialization IN listViewName;\n" + + "};\n\n" + + "Naming the list view in DROP is required, not optional: one page can hold two list views\n" + + "with a template for the same entity. To change what a template renders, edit the widgets\n" + + "inside it by name (SET / INSERT AFTER) — they are ordinary widgets. To replace a whole\n" + + "template, DROP it and INSERT the new one in the same ALTER block; operations apply in\n" + + "order.", + Example: "LISTVIEW vehicleListView (DataSource: DATABASE Pages.Vehicle) {\n" + + " DYNAMICTEXT defaultVehicle (Content: '{1} {2}', ContentParams: [{1} = Brand, {2} = Model])\n" + + " TEMPLATE FOR Pages.Bus {\n" + + " DYNAMICTEXT busLabel (Content: 'Bus, capacity {1}', ContentParams: [{1} = PassengerCapacity])\n" + + " }\n" + + " TEMPLATE FOR Pages.Truck {\n" + + " DYNAMICTEXT truckLabel (Content: 'Truck, max load {1} kg', ContentParams: [{1} = MaxLoadKg])\n" + + " }\n" + + "}", + SeeAlso: []string{"page.widgets", "page.datasource"}, + }) + Register(SyntaxFeature{ Path: "page.datasource", Summary: "Datasource bindings: variable, database, microflow, selection, association", @@ -88,8 +133,9 @@ func init() { "alter page", "modify page", "update page", "set property", "insert widget", "drop widget", "replace widget", "popup width", "popup height", "popup resizable", + "drop template", "insert template", "list view template", }, - Syntax: "ALTER PAGE Module.Name {\n SET property = value ON widgetName;\n SET Action = MICROFLOW Module.MF ON btnSave; -- any CREATE PAGE action form\n SET DataSource = $Param ON dvOrder;\n SET (prop1 = val1, prop2 = val2) ON widgetName;\n SET Title = 'New Title'; -- page-level (case-sensitive)\n SET Class = 'css-class'; -- page-level CSS class / style\n SET Style = 'css: rule';\n SET PopupWidth = 800; -- page-level pop-up dimensions\n SET PopupHeight = 480;\n SET PopupResizable = true;\n INSERT AFTER widgetName { };\n INSERT BEFORE widgetName { };\n INSERT INTO containerName { };\n DROP WIDGET name1, name2;\n REPLACE widgetName WITH { };\n};", + Syntax: "ALTER PAGE Module.Name {\n SET property = value ON widgetName;\n SET Action = MICROFLOW Module.MF ON btnSave; -- any CREATE PAGE action form\n SET DataSource = $Param ON dvOrder;\n SET (prop1 = val1, prop2 = val2) ON widgetName;\n SET Title = 'New Title'; -- page-level (case-sensitive)\n SET Class = 'css-class'; -- page-level CSS class / style\n SET Style = 'css: rule';\n SET PopupWidth = 800; -- page-level pop-up dimensions\n SET PopupHeight = 480;\n SET PopupResizable = true;\n INSERT AFTER widgetName { };\n INSERT BEFORE widgetName { };\n INSERT INTO containerName { };\n DROP WIDGET name1, name2;\n DROP TEMPLATE FOR Module.Specialization IN listViewName;\n REPLACE widgetName WITH { };\n};", Example: "ALTER PAGE Module.EditPage {\n SET (Caption = 'Save & Close', ButtonStyle = Success) ON btnSave;\n INSERT AFTER txtName {\n TEXTBOX txtMiddleName (Label: 'Middle Name', Attribute: MiddleName)\n };\n DROP WIDGET txtUnused;\n};", SeeAlso: []string{"page.create", "page.show", "snippet.alter"}, }) @@ -189,7 +235,7 @@ func init() { "create menu", "describe menu", "drop menu", "menu", "menus", "menu document", "menu item", }, - Syntax: "CREATE [OR MODIFY] MENU Module.Name (\n" + + Syntax: "CREATE [OR MODIFY] MENU Module.Name [FOLDER 'path'] (\n" + " MENU ITEM '' [PAGE Module.Page | MICROFLOW Module.Flow] [ICON Module.Collection.name];\n" + " MENU '' [ICON Module.Collection.name] ( );\n" + ");\n" + diff --git a/cmd/mxcli/syntax/features_workflow.go b/cmd/mxcli/syntax/features_workflow.go index ddc003d31..ba8027c54 100644 --- a/cmd/mxcli/syntax/features_workflow.go +++ b/cmd/mxcli/syntax/features_workflow.go @@ -32,7 +32,7 @@ func init() { "create workflow", "new workflow", "define workflow", "parameter", "overview page", "due date", }, - Syntax: "CREATE [OR MODIFY] WORKFLOW Module.Name\n PARAMETER $Context: Module.Entity\n [OVERVIEW PAGE Module.OverviewPage]\n [DUE DATE '']\nBEGIN\n \nEND WORKFLOW;", + Syntax: "CREATE [OR MODIFY] WORKFLOW Module.Name\n [FOLDER 'path']\n PARAMETER $Context: Module.Entity\n [OVERVIEW PAGE Module.OverviewPage]\n [DUE DATE '']\nBEGIN\n \nEND WORKFLOW;", Example: "CREATE WORKFLOW Module.ApprovalFlow\n PARAMETER $Context: Module.Request\n OVERVIEW PAGE Module.WF_Overview\nBEGIN\n USER TASK ReviewTask 'Review the request'\n PAGE Module.ReviewPage\n OUTCOMES 'Approve' { } 'Reject' { };\nEND WORKFLOW;", SeeAlso: []string{"workflow.user-task", "workflow.decision", "workflow.drop"}, }) diff --git a/cmd/mxcli/syntax/registry.go b/cmd/mxcli/syntax/registry.go index 8926ad301..57796db08 100644 --- a/cmd/mxcli/syntax/registry.go +++ b/cmd/mxcli/syntax/registry.go @@ -53,6 +53,12 @@ var topicAliases = map[string]string{ "snippets": "snippet", "fragments": "fragment", "workflows": "workflow", + // Folder aliases — "folder" is the word a user reaches for whether they + // mean placing a document (document-folder) or listing the layout (folders), + // and the placement question is the far more common one. + "folder": "document-folder", + "document-folders": "document-folder", + "list-folders": "folders", // Variant aliases "nav": "navigation", "project-settings": "settings", diff --git a/cmd/mxcli/testrunner/assertions_test.go b/cmd/mxcli/testrunner/assertions_test.go index 4a4ad6e0b..7ac68eeae 100644 --- a/cmd/mxcli/testrunner/assertions_test.go +++ b/cmd/mxcli/testrunner/assertions_test.go @@ -3,6 +3,9 @@ package testrunner import ( + "fmt" + "os" + "path/filepath" "strings" "testing" ) @@ -95,3 +98,91 @@ func TestJUnitCarriesSourceFileAndAssertions(t *testing.T) { t.Errorf("assertion count missing from the JUnit report:\n%s", out) } } + +// TestRequireAssertionsIsHonouredOnTheLogRunner is the regression test for #926. +// +// --require-assertions was registered on `mxcli test` globally but consulted in +// exactly one place — the endpoint runner's loop. The Docker path (and +// `--local --legacy-runner`) assembles its results from the runtime log in +// ParseLogResults, which had no parameter through which the flag could reach it, +// so `mxcli test tests/ -p app.mpr --require-assertions` — the command in the +// issue, note the absent --local — accepted the flag and exited 0 on a suite +// that asserted nothing. +// +// Whether a test is vacuous is a property of the *parsed* test case, known +// before any runner starts. No runner has an excuse for not knowing it, which is +// why this is implemented on the log path rather than refused there the way +// @verify is (that one genuinely cannot be evaluated during boot). +func TestRequireAssertionsIsHonouredOnTheLogRunner(t *testing.T) { + // One test that asserts nothing, which the runtime log reports as a pass. + newSuite := func() *TestSuite { + return &TestSuite{Name: "s", Tests: []TestCase{ + {ID: "test_1", Name: "asserts nothing", SourceFile: "a.test.mdl"}, + }} + } + const log = "MXTEST: MXTEST:START:\n" + + "MXTEST: MXTEST:RUN:test_1:asserts nothing\n" + + "MXTEST: MXTEST:PASS:test_1\n" + + "MXTEST: MXTEST:END:\n" + + // Off by default: a smoke test is legitimate. + off := ParseLogResults(strings.NewReader(log), newSuite(), false) + if got := off.Tests[0].Status; got != StatusPass { + t.Errorf("without --require-assertions: status = %v, want PASS", got) + } + if !off.AllPassed() { + t.Error("without --require-assertions the suite should still pass") + } + + on := ParseLogResults(strings.NewReader(log), newSuite(), true) + if got := on.Tests[0].Status; got != StatusError { + t.Errorf("with --require-assertions: status = %v, want ERROR "+ + "(the flag is a silent no-op on this runner — issue #926)", got) + } + if on.ErrorCount() != 1 { + t.Errorf("ErrorCount() = %d, want 1", on.ErrorCount()) + } + if on.AllPassed() { + t.Error("with --require-assertions the suite must not report all-passed") + } + if msg := on.Tests[0].Message; !strings.Contains(msg, "no assertions") { + t.Errorf("message = %q, want it to say the test asserts nothing", msg) + } +} + +// TestPreRunVerdictsHaveOneCallSite is the structural half of the #926 fix. +// +// The bug was not that vacuousResult was wrong — its unit test passed all along, +// because it exercised the helper and not any runner. The bug was that one of +// the two result-assembly loops called it and the other did not. Pinning the +// helpers to a single call site is what stops a third verdict (or a third +// runner) from reintroducing the same asymmetry. +func TestPreRunVerdictsHaveOneCallSite(t *testing.T) { + files, err := filepath.Glob("*.go") + if err != nil { + t.Fatal(err) + } + for _, helper := range []string{"assertionErrorResult(", "vacuousResult("} { + var callers []string + for _, f := range files { + if strings.HasSuffix(f, "_test.go") { + continue + } + src, err := os.ReadFile(f) + if err != nil { + t.Fatal(err) + } + for i, line := range strings.Split(string(src), "\n") { + if !strings.Contains(line, helper) || strings.Contains(line, "func "+helper[:len(helper)-1]) { + continue + } + callers = append(callers, fmt.Sprintf("%s:%d", f, i+1)) + } + } + if len(callers) != 1 { + t.Errorf("%s has %d call sites %v, want exactly 1 (preRunResult) — "+ + "a pre-run verdict reachable from one runner and not the other is issue #926", + helper, len(callers), callers) + } + } +} diff --git a/cmd/mxcli/testrunner/results.go b/cmd/mxcli/testrunner/results.go index 7364ebe75..3e11c7b18 100644 --- a/cmd/mxcli/testrunner/results.go +++ b/cmd/mxcli/testrunner/results.go @@ -164,7 +164,17 @@ func (sr *SuiteResult) AllPassed() bool { // ParseLogResults parses structured MXTEST: log lines from runtime output // and matches them to the test suite's test cases. -func ParseLogResults(logReader io.Reader, suite *TestSuite) *SuiteResult { +// ParseLogResults assembles a suite result from the runtime log the +// after-startup runner leaves behind (the Docker path, and --local +// --legacy-runner). +// +// requireAssertions is threaded in rather than applied by the caller afterwards +// because the pre-run verdicts have to be decided in the same place as the +// log-derived ones: a test that is an ERROR before it runs must not also pick up +// a PASS from the log. It reached this function late — it was honoured only by +// the endpoint runner, so `mxcli test -p app.mpr --require-assertions` without +// --local accepted the flag and did nothing (issue #926). +func ParseLogResults(logReader io.Reader, suite *TestSuite, requireAssertions bool) *SuiteResult { result := &SuiteResult{ Name: suite.Name, Started: time.Now(), @@ -282,10 +292,11 @@ func ParseLogResults(logReader io.Reader, suite *TestSuite) *SuiteResult { // Collect results in test order for _, tc := range suite.Tests { - // A test whose @expect did not compile was never generated, so the log - // has nothing to say about it. Report the parse error rather than the - // generic "not executed". - if res, bad := assertionErrorResult(tc); bad { + // Verdicts that are settled before the test runs. A test whose @expect + // did not compile was never generated, so the log has nothing to say + // about it; report the parse error rather than the generic "not + // executed". Same for a vacuous test under --require-assertions. + if res, bad := preRunResult(tc, requireAssertions); bad { result.Tests = append(result.Tests, res) continue } @@ -421,3 +432,19 @@ func vacuousResult(tc TestCase, require bool) (TestResult, bool) { "(add an @expect, or drop --require-assertions)" return res, true } + +// preRunResult returns the verdict a test earns before it is executed at all — +// the ones that depend only on the parsed test case, never on the run. +// +// Both result-assembly loops go through here, and TestPreRunVerdictsHaveOneCallSite +// pins that they are its only callers. That is the actual fix for #926: the +// helpers below were correct and unit-tested throughout, but only the endpoint +// runner called vacuousResult, so --require-assertions was silently inert on the +// log runner. A verdict reachable from one runner and not the other is the +// defect; a single call site is what prevents the next one. +func preRunResult(tc TestCase, requireAssertions bool) (TestResult, bool) { + if res, bad := assertionErrorResult(tc); bad { + return res, true + } + return vacuousResult(tc, requireAssertions) +} diff --git a/cmd/mxcli/testrunner/results_test.go b/cmd/mxcli/testrunner/results_test.go index d845ab3e6..b1d4ce481 100644 --- a/cmd/mxcli/testrunner/results_test.go +++ b/cmd/mxcli/testrunner/results_test.go @@ -31,7 +31,7 @@ Core: Successfully ran after-startup-action. }, } - result := ParseLogResults(strings.NewReader(logs), suite) + result := ParseLogResults(strings.NewReader(logs), suite, false) if len(result.Tests) != 3 { t.Fatalf("Result count: got %d, want 3", len(result.Tests)) @@ -78,7 +78,7 @@ MXTEST: MXTEST:PASS:test_2 }, } - result := ParseLogResults(strings.NewReader(logs), suite) + result := ParseLogResults(strings.NewReader(logs), suite, false) if result.Tests[2].Status != StatusError { t.Errorf("Test 3 status: got %v, want ERROR (not executed)", result.Tests[2].Status) diff --git a/cmd/mxcli/testrunner/runner.go b/cmd/mxcli/testrunner/runner.go index f0f4555a5..12b9019ce 100644 --- a/cmd/mxcli/testrunner/runner.go +++ b/cmd/mxcli/testrunner/runner.go @@ -359,7 +359,7 @@ func runAfterStartup(opts RunOptions, suite *TestSuite, timeout time.Duration, w } fmt.Fprintln(w, "Parsing test results...") - result := ParseLogResults(strings.NewReader(logOutput), suite) + result := ParseLogResults(strings.NewReader(logOutput), suite, opts.RequireAssertions) fmt.Fprintln(w, "Cleaning up...") cleanupErr := cleanup(opts.ProjectPath, state, w) diff --git a/cmd/mxcli/testrunner/runner_endpoint.go b/cmd/mxcli/testrunner/runner_endpoint.go index c8119ed5e..d94e90d75 100644 --- a/cmd/mxcli/testrunner/runner_endpoint.go +++ b/cmd/mxcli/testrunner/runner_endpoint.go @@ -129,11 +129,7 @@ func runSuite(client *endpointClient, admin docker.M2EEOptions, suite *TestSuite leaked := 0 for _, tc := range suite.Tests { - if res, bad := assertionErrorResult(tc); bad { - result.Tests = append(result.Tests, res) - continue - } - if res, bad := vacuousResult(tc, opts.RequireAssertions); bad { + if res, bad := preRunResult(tc, opts.RequireAssertions); bad { result.Tests = append(result.Tests, res) continue } diff --git a/docs-site/src/appendixes/quick-reference.md b/docs-site/src/appendixes/quick-reference.md index 0fe1a2032..694b257fe 100644 --- a/docs-site/src/appendixes/quick-reference.md +++ b/docs-site/src/appendixes/quick-reference.md @@ -188,7 +188,10 @@ AUTHENTICATION Basic, Session |-----------|--------|-------| | Microflow folder | `FOLDER 'path'` (before BEGIN) | `CREATE MICROFLOW ... FOLDER 'ACT' BEGIN ... END;` | | Page folder | `Folder: 'path'` (in properties) | `CREATE PAGE ... (Folder: 'Pages/Detail') { ... }` | -| Move to folder | `MOVE PAGE\|MICROFLOW\|SNIPPET\|NANOFLOW\|ENUMERATION Module.Name TO FOLDER 'path';` | Folders created automatically | +| Move to folder | `MOVE Module.Name TO FOLDER 'path';` | Folders created automatically. Any top-level doctype, spelled as `DESCRIBE` spells it | +| Move a mapping / structure | `MOVE IMPORT MAPPING\|EXPORT MAPPING\|JSON STRUCTURE Module.Name TO FOLDER 'path';` | | +| Place while creating | `CREATE Module.Name FOLDER 'path' ...` | Every doctype. Pages/snippets use `Folder: 'path'` as a property; microflows/nanoflows a keyword before `BEGIN` | +| Place an existing document | `CREATE OR MODIFY ... FOLDER 'path' ...` | Moves it; omitting the clause leaves placement alone | | Move to module root | `MOVE PAGE Module.Name TO Module;` | Removes from folder | | Move across modules | `MOVE PAGE Old.Name TO NewModule;` | **Breaks by-name references** -- use `LIST IMPACT OF` first | | Move to folder in other module | `MOVE PAGE Old.Name TO FOLDER 'path' IN NewModule;` | | @@ -204,7 +207,7 @@ Nested folders use `/` separator: `'Parent/Child/Grandchild'`. Missing folders a | List module roles | `LIST MODULE ROLES [IN Module];` | All roles or filtered by module | | List user roles | `LIST USER ROLES;` | Project-level user roles | | List demo users | `LIST DEMO USERS;` | Configured demo users | -| List access on element | `LIST ACCESS ON MICROFLOW\|PAGE\|Entity Mod.Name;` | Which roles can access | +| List access on element | `LIST ACCESS ON [ENTITY\|MICROFLOW\|PAGE\|NANOFLOW] Mod.Name;` | Which roles can access; a bare name means the entity | | List security matrix | `LIST SECURITY MATRIX [IN Module];` | Full access overview | | Create module role | `CREATE MODULE ROLE Mod.Role [DESCRIPTION 'text'];` | | | Drop module role | `DROP MODULE ROLE Mod.Role;` | | diff --git a/docs-site/src/language/control-flow.md b/docs-site/src/language/control-flow.md index d4dd881d4..945202188 100644 --- a/docs-site/src/language/control-flow.md +++ b/docs-site/src/language/control-flow.md @@ -240,6 +240,42 @@ Rules, each of which `mxcli check` enforces: Several values may share a branch (`WHEN Draft, Submitted THEN …`). `CASE` works in nanoflows on the same terms. +## SPLIT TYPE (Object Type Split) + +`SPLIT TYPE` branches on an object's **runtime specialization** and compiles to a Mendix +object type decision. Branches use the same `WHEN … THEN` shape as [CASE](#case-enum-split), +so the two splits differ only in what they branch on. + +```sql +SPLIT TYPE $Animal + WHEN Zoo.Dog THEN + CAST $Dog; + LOG INFO 'woof'; + WHEN Zoo.Cat THEN + LOG INFO 'meow'; + WHEN Zoo.Animal THEN + LOG INFO 'some other animal'; + WHEN (empty) THEN + LOG INFO 'no animal at all'; +END SPLIT; +``` + +Use `CAST` inside a branch to bind the specialized variable its body needs. + +| Rule | Why | +|------|-----| +| A branch for **every** subtype **and** the base entity | mxbuild reports **CE0090** *"The 'X' value should be configured for an outgoing flow"* for each type with no branch. The base entity is what covers "none of the more specific ones" | +| `WHEN (empty) THEN` is the **null-object** branch, not a default | It is taken when the variable is empty. It does **not** cover unnamed types — measured on 11.13.0, one named type plus an empty branch still gives CE0090 for every other type. Branch on the base entity for that | +| The empty branch cannot be omitted | **CE0089**. mxcli emits the flow unconditionally, so MDL cannot express a split without one | +| A non-void microflow needs a `RETURN` after `END SPLIT;` | Branches converge on a merge that continues to the end event — otherwise **MDL003** and **CE0067** | + +The pre-#913 spelling — `CASE Zoo.Dog` for a branch and `ELSE` for the empty one — still +parses and builds the identical flow, but warns **MDL065**. It was replaced because `CASE` +introduced a *branch* here while introducing the *subject* in `CASE $x WHEN v THEN`, and +because `ELSE` reads as a default branch and never was one. + +`SPLIT TYPE` works in nanoflows on the same terms. + ## Unsupported Control Flow The following constructs are **not** supported in MDL and will cause parse errors: diff --git a/docs-site/src/language/entity-access.md b/docs-site/src/language/entity-access.md index df7aea2af..0249fc285 100644 --- a/docs-site/src/language/entity-access.md +++ b/docs-site/src/language/entity-access.md @@ -138,7 +138,8 @@ A full `REVOKE` (without rights list) removes the entire access rule. A partial ## Viewing Entity Access ```sql --- See which roles have access to an entity +-- See which roles have access to an entity (the two spellings are synonyms) +SHOW ACCESS ON ENTITY Shop.Customer; SHOW ACCESS ON Shop.Customer; -- Full matrix across a module diff --git a/docs-site/src/language/security.md b/docs-site/src/language/security.md index 82f2458ef..ebbd778dc 100644 --- a/docs-site/src/language/security.md +++ b/docs-site/src/language/security.md @@ -42,7 +42,8 @@ SHOW USER ROLES; -- Access rules SHOW ACCESS ON MICROFLOW Shop.ACT_ProcessOrder; SHOW ACCESS ON PAGE Shop.Order_Edit; -SHOW ACCESS ON Shop.Customer; +SHOW ACCESS ON ENTITY Shop.Customer; +SHOW ACCESS ON Shop.Customer; -- a bare name means the entity -- Full matrix SHOW SECURITY MATRIX; diff --git a/docs-site/src/language/widget-types.md b/docs-site/src/language/widget-types.md index 328d48bbf..72ed451bf 100644 --- a/docs-site/src/language/widget-types.md +++ b/docs-site/src/language/widget-types.md @@ -156,6 +156,41 @@ LISTVIEW lvProducts (DataSource: DATABASE MyModule.Product) { } ``` +#### Specialization templates + +When the list view's entity is a **generalization**, it can render a different +body per specialization. A template is identified by the entity it renders — it +has no name, hence `TEMPLATE FOR `: + +```sql +LISTVIEW vehicleListView (DataSource: DATABASE Pages.Vehicle) { + -- the default body, used for an object no template matches + DYNAMICTEXT defaultVehicle (Content: '{1} {2}', ContentParams: [{1} = Brand, {2} = Model]) + + TEMPLATE FOR Pages.Bus { + DYNAMICTEXT busLabel (Content: 'Bus, capacity {1}', ContentParams: [{1} = PassengerCapacity]) + } + TEMPLATE FOR Pages.Truck { + DYNAMICTEXT truckLabel (Content: 'Truck, max load {1} kg', ContentParams: [{1} = MaxLoadKg]) + } +} +``` + +Widgets written directly in the list view body are the default rendering. +Templates keep their **source order**: Mendix stores and matches in that order, +so it is authored rather than derived, and `DESCRIBE PAGE` emits them as stored. + +Inside a template the context object is the specialization, so an attribute only +that specialization has still resolves — `PassengerCapacity` above exists on +`Pages.Bus`, not on `Pages.Vehicle`. + +The entity must be the list view's entity or a specialization of it, and there +may be at most one template per entity; both are refused rather than written, +because a template that cannot match never renders. + +> A Gallery's `TEMPLATE ` is a different construct — a named content slot, +> not a per-specialization body. + ### GALLERY A pluggable widget that displays items in a card/grid layout: diff --git a/docs-site/src/reference/capabilities.md b/docs-site/src/reference/capabilities.md index 34367323d..4463722b8 100644 --- a/docs-site/src/reference/capabilities.md +++ b/docs-site/src/reference/capabilities.md @@ -21,7 +21,7 @@ Everything mxcli can do, organized by use case. | Full-text search | `SEARCH 'keyword'` | Across all strings and source | | Show languages | `LIST LANGUAGES` | All languages in the project | | Show project security | `LIST PROJECT SECURITY` | Security overview | -| Show access rules | `LIST ACCESS ON Module.Entity` | Entity/microflow/page access | +| Show access rules | `LIST ACCESS ON [ENTITY] Module.Name` | Entity/microflow/page/nanoflow access | | Show settings | `LIST SETTINGS` | Project-level settings | ## Write: Create and modify documents diff --git a/docs-site/src/reference/integration/create-data-transformer.md b/docs-site/src/reference/integration/create-data-transformer.md index fec2f6a3c..4f6b7dde1 100644 --- a/docs-site/src/reference/integration/create-data-transformer.md +++ b/docs-site/src/reference/integration/create-data-transformer.md @@ -4,6 +4,7 @@ ```sql CREATE [ OR MODIFY ] DATA TRANSFORMER module.Name + [ FOLDER 'folder_path' ] SOURCE { JSON | XML } 'sample' { { JSLT | XSLT } 'transformation'; @@ -37,6 +38,9 @@ The `SOURCE` clause provides a representative input document. Mendix uses it to `module.Name` : The qualified name of the data transformer. +`FOLDER 'folder_path'` +: Optional. Places the document in the named module folder, creating missing folders in the path. On `CREATE OR MODIFY` this **moves** an existing document; omitting the clause leaves placement alone rather than returning the document to the module root. See [MOVE](../organization/move.md). + `SOURCE JSON 'sample'` : A representative JSON input document. Multi-line samples can use `$$...$$` quoting. diff --git a/docs-site/src/reference/integration/create-export-mapping.md b/docs-site/src/reference/integration/create-export-mapping.md index 8820a03a9..0615b97f8 100644 --- a/docs-site/src/reference/integration/create-export-mapping.md +++ b/docs-site/src/reference/integration/create-export-mapping.md @@ -4,6 +4,7 @@ ```sql CREATE [ OR MODIFY ] EXPORT MAPPING module.Name + [ FOLDER 'folder_path' ] [ WITH JSON STRUCTURE module.JsonStructure ] [ WITH XML SCHEMA module.XmlSchema ] [ NULL VALUES { LeaveOutElement | SendAsNil } ] @@ -53,6 +54,9 @@ If `OR MODIFY` is specified and the mapping already exists, it is updated in pla `module.Name` : The qualified name of the export mapping. +`FOLDER 'folder_path'` +: Optional. Places the document in the named module folder, creating missing folders in the path. On `CREATE OR MODIFY` this **moves** an existing document; omitting the clause leaves placement alone rather than returning the document to the module root. See [MOVE](../organization/move.md). + `WITH JSON STRUCTURE module.JsonStructure` : Associates the mapping with the named JSON structure. diff --git a/docs-site/src/reference/integration/create-import-mapping.md b/docs-site/src/reference/integration/create-import-mapping.md index 490c1999c..9b2a37e4e 100644 --- a/docs-site/src/reference/integration/create-import-mapping.md +++ b/docs-site/src/reference/integration/create-import-mapping.md @@ -4,6 +4,7 @@ ```sql CREATE [ OR MODIFY ] IMPORT MAPPING module.Name + [ FOLDER 'folder_path' ] [ WITH JSON STRUCTURE module.JsonStructure ] [ WITH XML SCHEMA module.XmlSchema ] { @@ -55,6 +56,9 @@ If `OR MODIFY` is specified and the mapping already exists, it is updated in pla `module.Name` : The qualified name of the import mapping. +`FOLDER 'folder_path'` +: Optional. Places the document in the named module folder, creating missing folders in the path. On `CREATE OR MODIFY` this **moves** an existing document; omitting the clause leaves placement alone rather than returning the document to the module root. See [MOVE](../organization/move.md). + `WITH JSON STRUCTURE module.JsonStructure` : Associates the mapping with the named JSON structure. The structure defines the JSON shape. diff --git a/docs-site/src/reference/integration/create-json-structure.md b/docs-site/src/reference/integration/create-json-structure.md index 0b4058600..9928e4319 100644 --- a/docs-site/src/reference/integration/create-json-structure.md +++ b/docs-site/src/reference/integration/create-json-structure.md @@ -26,7 +26,7 @@ The optional `CUSTOM_NAME_MAP` clause overrides the attribute names generated fr : The qualified name of the JSON structure. `FOLDER 'folder/path'` -: Optional. Places the document in the specified Studio Pro folder (forward-slash separated). +: Optional. Places the document in the specified Studio Pro folder (forward-slash separated), creating missing folders in the path. On `CREATE OR MODIFY` this **moves** an existing document; omitting the clause leaves placement alone rather than returning the document to the module root. See [MOVE](../organization/move.md). `COMMENT 'description'` : Optional. A description for the JSON structure document. diff --git a/docs-site/src/reference/microflow/create-java-action.md b/docs-site/src/reference/microflow/create-java-action.md index b809d6fb0..4ce111b47 100644 --- a/docs-site/src/reference/microflow/create-java-action.md +++ b/docs-site/src/reference/microflow/create-java-action.md @@ -3,7 +3,7 @@ ## Synopsis ```sql -CREATE [ OR MODIFY ] JAVA ACTION module.Name ( parameters ) +CREATE [ OR MODIFY ] JAVA ACTION module.Name [ FOLDER 'folder_path' ] ( parameters ) RETURNS type [ EXPOSED AS 'caption' IN 'category' ] AS $$ java_code $$ @@ -34,6 +34,9 @@ The optional `EXPOSED AS` clause makes the action visible in the Studio Pro tool `module.Name` : The qualified name of the Java action. +`FOLDER 'folder_path'` +: Optional. Places the document in the named module folder, creating missing folders in the path. On `CREATE OR MODIFY` this **moves** an existing document; omitting the clause leaves placement alone rather than returning the document to the module root. See [MOVE](../organization/move.md). + `parameters` : Comma-separated parameter declarations. Each parameter has a name, colon, and type. Supported types: - Primitives: `String`, `Integer`, `Long`, `Decimal`, `Boolean`, `DateTime` diff --git a/docs-site/src/reference/microflow/create-javascript-action.md b/docs-site/src/reference/microflow/create-javascript-action.md index fd1e40f27..edeb9229c 100644 --- a/docs-site/src/reference/microflow/create-javascript-action.md +++ b/docs-site/src/reference/microflow/create-javascript-action.md @@ -3,7 +3,7 @@ ## Synopsis ```sql -CREATE [ OR MODIFY ] JAVASCRIPT ACTION module.Name ( parameters ) +CREATE [ OR MODIFY ] JAVASCRIPT ACTION module.Name [ FOLDER 'folder_path' ] ( parameters ) RETURNS type [ EXPOSED AS 'caption' IN 'category' ] [ PLATFORM Web | Native | Hybrid | All ] @@ -47,6 +47,9 @@ toolbox under the given category. `module.Name` : The qualified name of the JavaScript action. +`FOLDER 'folder_path'` +: Optional. Places the document in the named module folder, creating missing folders in the path. On `CREATE OR MODIFY` this **moves** an existing document; omitting the clause leaves placement alone rather than returning the document to the module root. See [MOVE](../organization/move.md). + `parameters` : Comma-separated parameter declarations (name, colon, type). Supported types: - Primitives: `String`, `Integer`, `Long`, `Decimal`, `Boolean`, `DateTime` diff --git a/docs-site/src/reference/navigation/menu.md b/docs-site/src/reference/navigation/menu.md index 8e3fecec8..01fd65075 100644 --- a/docs-site/src/reference/navigation/menu.md +++ b/docs-site/src/reference/navigation/menu.md @@ -3,7 +3,7 @@ ## Synopsis ```sql -CREATE [ OR MODIFY ] MENU module.name ( menu_item [ menu_item ... ] ) +CREATE [ OR MODIFY ] MENU module.name [ FOLDER 'folder_path' ] ( menu_item [ menu_item ... ] ) DESCRIBE MENU module.name DROP MENU module.name ``` @@ -48,6 +48,9 @@ a fixed point. `module.name` : Qualified name of the menu document. +`FOLDER 'folder_path'` +: Optional. Places the document in the named module folder, creating missing folders in the path. On `CREATE OR MODIFY` this **moves** an existing document; omitting the clause leaves placement alone rather than returning the document to the module root. See [MOVE](../organization/move.md). + `'caption'` : The item's label, in single quotes. diff --git a/docs-site/src/reference/organization/move.md b/docs-site/src/reference/organization/move.md index 17a5ff35a..10793154f 100644 --- a/docs-site/src/reference/organization/move.md +++ b/docs-site/src/reference/organization/move.md @@ -8,14 +8,34 @@ ## Description -Moves a document to a different folder or module. Supported document types are `PAGE`, `MICROFLOW`, `NANOFLOW`, `SNIPPET`, `ENUMERATION`, and `ENTITY`. When moving to a folder, missing intermediate folders are created automatically. Cross-module moves change the qualified name of the document, which may break by-name references elsewhere in the project. +Moves a document to a different folder or module. Every top-level document type is supported, spelled the way `DESCRIBE` spells it. When moving to a folder, missing intermediate folders are created automatically. Cross-module moves change the qualified name of the document, which may break by-name references elsewhere in the project. Entity moves only support moving to a module (not to a folder), because entities are embedded in domain model documents. +A `FOLDER` clause on `CREATE OR MODIFY` moves an existing document as well, so a +script that already creates a document does not need a separate `MOVE` to place +it. + ## Parameters **document_type** -: The type of document to move. One of: `PAGE`, `MICROFLOW`, `NANOFLOW`, `SNIPPET`, `ENUMERATION`, `ENTITY`. +: The type of document to move: + +| Group | Types | +|-------|-------| +| Pages | `PAGE`, `SNIPPET`, `BUILDING BLOCK`, `LAYOUT`, `MENU` | +| Logic | `MICROFLOW`, `NANOFLOW`, `WORKFLOW`, `QUEUE`, `SCHEDULED EVENT` | +| Domain | `ENUMERATION`, `CONSTANT`, `REGULAR EXPRESSION`, `ENTITY` | +| Mappings | `JSON STRUCTURE`, `IMPORT MAPPING`, `EXPORT MAPPING` | +| Code | `JAVA ACTION`, `JAVASCRIPT ACTION`, `DATABASE CONNECTION`, `DATA TRANSFORMER` | +| Resources | `IMAGE COLLECTION`, `ICON COLLECTION` | +| Integration | `REST CLIENT`, `PUBLISHED REST SERVICE`, `ODATA CLIENT`, `ODATA SERVICE`, `BUSINESS EVENT SERVICE` | +| AI | `MODEL`, `AGENT`, `KNOWLEDGE BASE`, `CONSUMED MCP SERVICE` | + +`FOLDER` moves a folder rather than a document — see the example below. + +If the named document turns out to be a different type, the statement is refused +and the error names what it actually is. **qualified_name** : The current `Module.Name` of the document to move. @@ -58,6 +78,44 @@ MOVE ENTITY OldModule.Customer TO NewModule; MOVE PAGE OldModule.CustomerPage TO FOLDER 'Screens' IN NewModule; ``` +### Move a mapping or JSON structure + +```sql +MOVE IMPORT MAPPING MyModule.IMM_Order TO FOLDER 'Private/Import mappings'; +MOVE EXPORT MAPPING MyModule.EXM_Order TO FOLDER 'Private/Export mappings'; +MOVE JSON STRUCTURE MyModule.JSON_Order TO FOLDER 'Private/JSON structures'; +``` + +### Place a document while creating it + +Every document type takes a `FOLDER` clause on `CREATE`, so a document can be +placed by the statement that creates it. On pages and snippets it is a property +(`Folder: 'path'`); on microflows and nanoflows a keyword before `BEGIN`; on +everything else a keyword straight after the qualified name: + +```sql +CREATE OR MODIFY JSON STRUCTURE MyModule.JSON_Order + FOLDER 'Private/JSON structures' + SNIPPET '{"id": 1}'; + +CREATE QUEUE MyModule.Q_Orders FOLDER 'Private/Queues' ( Parallelism: 3 ); + +CREATE JAVA ACTION MyModule.JA_Sync FOLDER 'Private/Java' () + RETURNS String AS $$return null;$$; +``` + +The clause applies to an existing document too, so re-running a script that +gained one files the document rather than leaving it where it was. Omitting the +clause leaves the document where it is; it never returns it to the module root. + +`DESCRIBE` emits the clause, so a description replays into the same folder. + +### Move a folder + +```sql +MOVE FOLDER MyModule.OldName TO FOLDER 'Archive'; +``` + ### Check impact before a cross-module move ```sql diff --git a/docs-site/src/reference/security/README.md b/docs-site/src/reference/security/README.md index b325e938b..73d8eabc1 100644 --- a/docs-site/src/reference/security/README.md +++ b/docs-site/src/reference/security/README.md @@ -22,7 +22,7 @@ Mendix security operates at two levels. **Module roles** define permissions with | Show module roles | `SHOW MODULE ROLES [IN module]` | | Show user roles | `SHOW USER ROLES` | | Show demo users | `SHOW DEMO USERS` | -| Show access on element | `SHOW ACCESS ON MICROFLOW\|PAGE module.Name` | +| Show access on element | `SHOW ACCESS ON [ENTITY\|MICROFLOW\|PAGE\|NANOFLOW] module.Name` | | Show security matrix | `SHOW SECURITY MATRIX [IN module]` | | Alter project security level | `ALTER PROJECT SECURITY LEVEL OFF\|PROTOTYPE\|PRODUCTION` | | Toggle demo users | `ALTER PROJECT SECURITY DEMO USERS ON\|OFF` | diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index bdf80ef9a..37b663659 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -144,7 +144,7 @@ create constant MyModule.EnableLogging type boolean default true; |-----------|--------|-------| | Show queues | `show queues [in module];` (`list queues` too) | Parallelism + cluster-wide flag | | Describe queue | `describe queue Module.Name;` | Re-executable MDL | -| Create queue | `create [or modify] queue Module.Name ( Parallelism: 3, ClusterWide: true );` | Body optional; defaults `1` / `false` | +| Create queue | `create [or modify] queue Module.Name [folder 'path'] ( Parallelism: 3, ClusterWide: true );` | Body optional; defaults `1` / `false` | | Drop queue | `drop queue Module.Name;` | | `Parallelism` is an **expression**, not a number — Mendix stores it as a string @@ -174,7 +174,7 @@ Named patterns, shared by attribute validation rules. |-----------|--------|-------| | Show regular expressions | `show regular expressions [in module];` (`list` too) | Pattern + documentation | | Describe regular expression | `describe regular expression Module.Name;` | Re-executable MDL | -| Create regular expression | `create [or modify] regular expression Module.Name ( Expression: '' );` | `Expression` required | +| Create regular expression | `create [or modify] regular expression Module.Name [folder 'path'] ( Expression: '' );` | `Expression` required | | Drop regular expression | `drop regular expression Module.Name;` | | A regex is a **document**, not a string on a rule: Mendix stores a validation @@ -249,7 +249,7 @@ Mendix's cron: run a microflow on a repeating schedule. |-----------|--------|-------| | Show scheduled events | `show scheduled events [in module];` (`list` too) | Repeat, microflow, enabled | | Describe scheduled event | `describe scheduled event Module.Name;` | Re-executable MDL | -| Create scheduled event | `create [or modify] scheduled event Module.Name ( Microflow: ..., Repeat: ..., ... );` | | +| Create scheduled event | `create [or modify] scheduled event Module.Name [folder 'path'] ( Microflow: ..., Repeat: ..., ... );` | | | Drop scheduled event | `drop scheduled event Module.Name;` | | `Microflow` and `Repeat` are always required. Each repeat takes **only** its own @@ -482,7 +482,7 @@ it is for pages. | Free annotation | `@annotation 'text'` before `@position(...)` | Free-floating visual note preserved by order | | IF | `if condition then ... [else ...] end if;` | | | Enum split | `case $Var when Value then ... end case;` | Enumeration decision branches. Bare enum values (never quoted or qualified), one branch per value **including `(empty)`** (MDL056), no `else` (MDL008), no `AS` alias | -| Type split | `split type $Var case Module.Entity ... end split;` | Runtime specialization branches | +| Type split | `split type $Var when Module.Entity then ... when (empty) then ... end split;` | Runtime specialization branches. Same `when ... then` shape as the enum split. Needs a branch per subtype **and** the base entity (CE0090); `when (empty) then` is the **null-object** flow, not a default, and cannot be omitted (CE0089). Legacy `case Module.Entity` / `else` still parse (MDL065 warns) | | Cast | `cast $SpecificVar;` | Downcast inside a type split branch | | LOOP | `loop $item in $list begin ... end loop;` | FOR EACH over list. No `return` inside — an End event cannot sit in a loop (CE0068 / MDL062); use `break` and return after the loop | | WHILE | `while condition begin ... end while;` | Condition-based loop | @@ -512,7 +512,10 @@ it is for pages. | Page folder | `folder: 'path'` (in properties) | `create page ... (folder: 'pages/Detail') { ... }` | | Drop folder | `drop folder 'path' in module;` | Folder must be empty | | Move folder | `move folder Module.FolderName to folder 'path';` | Target folders auto-created | -| Move to folder | `move page\|microflow\|snippet\|nanoflow\|enumeration Module.Name to folder 'path';` | Folders created automatically | +| Move to folder | `move Module.Name to folder 'path';` | Folders created automatically. Any top-level doctype, spelled as `describe` spells it | +| Move a mapping / structure | `move import mapping\|export mapping\|json structure Module.Name to folder 'path';` | | +| Place while creating | `create Module.Name folder 'path' ...` | Every doctype. Pages/snippets use `folder: 'path'` as a property; microflows/nanoflows a keyword before `begin` | +| Place an existing document | `create or modify ... folder 'path' ...` | Moves it; omitting the clause leaves placement alone | | Move to module root | `move page Module.Name to module;` | Removes from folder | | Move across modules | `move page Old.Name to NewModule;` | **Breaks by-name references** — use `show impact of` first | | Move to folder in other module | `move page Old.Name to folder 'path' in NewModule;` | | @@ -555,7 +558,7 @@ Nested folders use `/` separator: `'Parent/Child/Grandchild'`. Missing folders a |-----------|--------|-------| | Show workflows | `show workflows [in module];` | List all or filter by module | | Describe workflow | `describe workflow Module.Name;` | Full MDL output | -| Create workflow | `create [or modify] workflow Module.Name parameter $Ctx: Module.Entity begin ... end workflow;` | See activity types below | +| Create workflow | `create [or modify] workflow Module.Name [folder 'path'] parameter $Ctx: Module.Entity begin ... end workflow;` | See activity types below | | Drop workflow | `drop workflow Module.Name;` | | **Workflow Activity Types:** @@ -814,7 +817,7 @@ Respond in {{Language}}.$$, |-----------|--------|-------| | Show collections | `show image collection [in module];` | List all or filter by module | | Describe collection | `describe image collection Module.Name;` | Full MDL output with embedded images | -| Create collection | `create image collection Module.Name [export level 'Hidden'\|'Public'] [comment 'text'] [(image Name from file 'path', ...)];` | With or without images | +| Create collection | `create image collection Module.Name [folder 'path'] [export level 'Hidden'\|'Public'] [comment 'text'] [(image Name from file 'path', ...)];` | With or without images | | Create or modify | `create or modify image collection Module.Name [...];` | Preserves UUID — preferred for AI agents | | Drop collection | `drop image collection Module.Name;` | Removes collection and all embedded images | @@ -993,6 +996,7 @@ source json '{"latitude": 51.9, "current": {"temp": 12.8}}' | Describe mapping | `describe import mapping Module.Name;` | Re-executable CREATE statement | | Create mapping | See below | Assignment syntax: `attr = jsonField`, or `attr = a/b/c` to reach a nested leaf with **no entity per level** — the shape Studio Pro produces. The path may not cross a `0..*` element (CE0256) | | Create or modify | `create or modify import mapping Module.Name ...;` | Updates existing mapping, preserves UUID | +| Place in a folder | `create [or modify] import mapping Module.Name folder 'path' ...;` | Clause goes after the name. On `or modify` it **moves** the mapping; omitting it leaves placement alone | | Drop mapping | `drop import mapping Module.Name;` | | ```sql @@ -1025,6 +1029,7 @@ create Module.OrderResponse_CustomerInfo/Module.CustomerInfo = customer { | Describe mapping | `describe export mapping Module.Name;` | Re-executable CREATE statement | | Create mapping | See below | Assignment syntax: `jsonField = attr`. **No nested `a/b/c` form**: an export has to produce the intermediate node, so Mendix rejects a collapsed member with CE5015 — give it its own element | | Create or modify | `create or modify export mapping Module.Name ...;` | Updates existing mapping, preserves UUID | +| Place in a folder | `create [or modify] export mapping Module.Name folder 'path' ...;` | Clause goes after the name. On `or modify` it **moves** the mapping; omitting it leaves placement alone | | Drop mapping | `drop export mapping Module.Name;` | | ```sql @@ -1054,7 +1059,7 @@ Module.OrderResponse_CustomerInfo/Module.CustomerInfo as customer { |-----------|--------|-------| | Show Java actions | `show java actions [in module];` | List all or filtered by module | | Describe Java action | `describe java action Module.Name;` | Full MDL output with signature | -| Create Java action | `create [or modify] java action Module.Name(params) returns type as $$ ... $$;` | OR MODIFY updates signature/body, preserves UUID | +| Create Java action | `create [or modify] java action Module.Name [folder 'path'](params) returns type as $$ ... $$;` | OR MODIFY updates signature/body, preserves UUID | | Create with type params | `create java action Module.Name(EntityType: entity , Obj: pEntity) ...;` | Generic type parameters | | Create exposed action | `... exposed as 'caption' in 'Category' as $$ ... $$;` | Toolbox-visible in Studio Pro | | Rename Java action | `rename java action Module.Old to New;` | Renames BSON unit and .java source file | @@ -1064,7 +1069,7 @@ Module.OrderResponse_CustomerInfo/Module.CustomerInfo as customer { | Empty argument | `call java action Module.Name(Param = empty);` | Unbound code-action parameter preserved as empty mapping | | Show JavaScript actions | `show javascript actions [in module];` | List all or filtered by module | | Describe JavaScript action | `describe javascript action Module.Name;` | Re-executable MDL with signature + body | -| Create JavaScript action | `create [or modify] javascript action Module.Name(params) returns type [platform Web] as $$ ... $$;` | Writes the unit + `javascriptsource//actions/.js`; OR MODIFY preserves UUID | +| Create JavaScript action | `create [or modify] javascript action Module.Name [folder 'path'](params) returns type [platform Web] as $$ ... $$;` | Writes the unit + `javascriptsource//actions/.js`; OR MODIFY preserves UUID | | Create exposed/native | `... exposed as 'caption' in 'Category' platform Native as $$ ... $$;` | `platform` is Web (default), Native, Hybrid, or All | | Drop JavaScript action | `drop javascript action Module.Name;` | Deletes MPR unit and .js source file | | Call from nanoflow | `$Result = call javascript action Module.Name(Param = value);` | Inside a nanoflow | @@ -1124,10 +1129,10 @@ MDL uses explicit property declarations for pages: | Describe snippet | `describe snippet Module.Name;` | Round-trippable MDL output | | List building blocks | `show building blocks [in module];` | Read-only; cannot be authored via MDL | | Describe building block | `describe building block Module.Name;` | Informational (header comment + widget tree), not a `create` statement | -| Create menu | `create [or modify] menu Module.Name ( );` | Standalone `Menus$MenuDocument`. Full replacement: the item list is the document's complete contents | +| Create menu | `create [or modify] menu Module.Name [folder 'path'] ( );` | Standalone `Menus$MenuDocument`. Full replacement: the item list is the document's complete contents | | Describe menu | `describe menu Module.Name;` | Round-trippable MDL. Not the navigation-profile menu — see `show navigation menu` | | Drop menu | `drop menu Module.Name;` | | -| Create menu | `create [or modify] menu Module.Name ( );` | Standalone `Menus$MenuDocument`. Full replacement: the item list is the document's complete contents | +| Create menu | `create [or modify] menu Module.Name [folder 'path'] ( );` | Standalone `Menus$MenuDocument`. Full replacement: the item list is the document's complete contents | | Describe menu | `describe menu Module.Name;` | Round-trippable MDL. Not the navigation-profile menu — see `show navigation menu` | | Drop menu | `drop menu Module.Name;` | | @@ -1185,6 +1190,50 @@ create page MyModule.Customer_Edit - Layout: `layoutgrid`, `row`, `column`, `container`, `customcontainer` - Input: `textbox`, `textarea`, `checkbox`, `radiobuttons`, `datepicker`, `combobox` - Display: `dynamictext`, `datagrid`, `gallery`, `listview`, `image`, `staticimage`, `dynamicimage` + +### List View specialization templates + +A List View over a generalization can render a different body per specialization. +The template is identified by the **entity** it renders — it has no name: + +```sql +listview vehicleListView (DataSource: database from Pages.Vehicle) { + dynamictext defaultVehicle (Content: '{1}', ContentParams: [{1} = Brand]) + + template for Pages.Bus { + dynamictext busLabel (Content: 'Bus, capacity {1}', ContentParams: [{1} = PassengerCapacity]) + } + template for Pages.Truck { + dynamictext truckLabel (Content: 'Truck, max load {1} kg', ContentParams: [{1} = MaxLoadKg]) + } +} +``` + +Widgets in the list view body are the **default** rendering, used for an object no +template matches. Templates keep their source order — Mendix stores and matches in +that order, so it is authored, not derived. + +`template for Module.Entity` is not the same statement as a Gallery's +`template `, which is a named content slot. The entity must be the list +view's entity or a specialization of it, and at most one template per entity is +allowed. Inside a template the context object is the specialization, so an +attribute only that specialization has still resolves. + +Editing them with `alter page` — adding reuses `insert into` with the same block; +removing needs its own form, because a template has no name: + +```sql +alter page Pages.Vehicle_Overview { + insert into vehicleListView { + template for Pages.Motorcycle { dynamictext mcLabel (Content: 'M') } + }; + drop template for Pages.SUV in vehicleListView +}; +``` + +Naming the list view in the `drop` is required: one page can hold two list views +with a template for the same entity. Widgets inside a template are ordinary named +widgets, so `set … on busLabel` and `insert after busLabel { … }` need nothing new. - Actions: `actionbutton`, `linkbutton`, `navigationlist` - Structure: `dataview`, `header`, `footer`, `controlbar`, `snippetcall` diff --git a/docs/11-proposals/PROPOSAL_microflow_inheritance_split_statement.md b/docs/11-proposals/PROPOSAL_microflow_inheritance_split_statement.md index 108c9f2ee..2c032be96 100644 --- a/docs/11-proposals/PROPOSAL_microflow_inheritance_split_statement.md +++ b/docs/11-proposals/PROPOSAL_microflow_inheritance_split_statement.md @@ -1,11 +1,14 @@ --- title: Microflow Inheritance Split And Cast Statements -status: draft +status: done +date: 2026-08-06 +related: + - PROPOSAL_split_statement_syntax_alignment.md --- # Proposal: Microflow Inheritance Split And Cast Statements -Status: Draft +Status: Done ## Summary @@ -13,10 +16,12 @@ Add round-trip MDL support for type-based microflow decisions and cast actions: ```mdl split type $Input -case Sample.SpecializedInput - cast $SpecificInput; -else - return false; + when Sample.SpecializedInput then + cast $SpecificInput; + when Sample.BaseInput then + return false; + when (empty) then + return false; end split; ``` @@ -26,7 +31,9 @@ Studio Pro represents specialization/type decisions as `InheritanceSplit` object ## Semantics -`split type $Var` evaluates the runtime specialization of an object variable. Each `case Module.Entity` branch corresponds to an outgoing sequence flow with an `InheritanceCase`. The optional `else` branch maps to the outgoing flow without an inheritance case. +`split type $Var` evaluates the runtime specialization of an object variable. Each `when Module.Entity then` branch corresponds to an outgoing sequence flow with an `InheritanceCase`. + +The `when (empty) then` branch maps to the outgoing flow with **no** inheritance case — which on a Mendix object-type decision is the `(empty)` flow, taken when the object is **null**. It is **not** a default for unmatched types: mxbuild still requires a flow for every subtype and for the base entity (CE0090), and omitting the empty branch is CE0089. It was originally spelled `else`, which read as a default and never was one; that spelling still parses and warns MDL065 (mxcli #913). `cast $Output` emits a `CastAction` that produces the downcast variable. `$Output = cast $Input` is accepted for source-preserving authoring, but current Mendix BSON stores the generated cast variable as the primary persisted field. diff --git a/docs/11-proposals/README.md b/docs/11-proposals/README.md index 2599436f0..89f2547c3 100644 --- a/docs/11-proposals/README.md +++ b/docs/11-proposals/README.md @@ -26,10 +26,10 @@ for display in this README): -## Active Proposals (93) +## Active Proposals (97) -### In Progress (partial) (11) +### In Progress (partial) (12) | Proposal | Status | Summary | |----------|--------|---------| @@ -44,11 +44,13 @@ for display in this README): | [Navigation Support in MDL](navigation-support.md) | Partial | Every Mendix project has exactly one navigation$NavigationDocument — a project-level singleton containing navigation profiles (Responsive, P | | [Podman Support as Docker Alternative](PROPOSAL_podman_support.md) | Partial | Docker Desktop requires a paid subscription for larger organizations. | | [SHOW/DESCRIBE/USE Building Blocks](show-describe-building-blocks.md) | Partial | Document type: Pages$BuildingBlock (NOT Forms$BuildingBlock — the reader now | +| [Skill packs — shipping a skill that carries more than prose](PROPOSAL_skill_packs.md) | Partial | mxcli ships 65 skills today and every one of them is a single Markdown file. | -### Proposed (38) +### Proposed (39) | Proposal | Status | Summary | |----------|--------|---------| +| [Bootstrap hooks that fetch the mxcli the project was built with](PROPOSAL_bootstrap_source.md) | Proposed | mxcli init writes .claude/bootstrap-mxcli.sh with a hard-coded download of | | [Comprehensive MDL Syntax and Grammar Improvements (v2)](PROPOSAL_mdl_syntax_improvements_v2.md) | Proposed | This document consolidates and expands upon previous proposals, offering a unified set of syntax and grammar improvements for all major Mend | | [Concurrent Access Safety for mxcli](PROPOSAL_concurrent_access.md) | Proposed | When Claude Code spawns multiple subagents, each may execute mxcli commands against the same Mendix project simultaneously. | | [Consumed REST Services (SHOW / DESCRIBE / CREATE)](show-describe-consumed-rest-services.md) | Proposed | Document type: rest$ConsumedRestService | @@ -115,7 +117,6 @@ for display in this README): | [Microflow CHANGE Refresh Modifier](PROPOSAL_microflow_change_refresh_modifier.md) | Draft | Status: Draft | | [Microflow Download File Statement](PROPOSAL_microflow_download_file_statement.md) | Draft | Status: Draft | | [Microflow Free Annotation](PROPOSAL_microflow_free_annotation.md) | Draft | Status: Draft | -| [Microflow Inheritance Split And Cast Statements](PROPOSAL_microflow_inheritance_split_statement.md) | Draft | Status: Draft | | [Module Maven Dependencies (JAR Dependencies)](PROPOSAL_module_maven_dependencies.md) | Draft | ModuleSettings document; this proposal focuses specifically on the Maven/JAR dependency subset | | [Multi-Agent Merge for Mendix Projects](PROPOSAL_multi_agent_merge.md) | Draft | Related: PROPOSAL_concurrent_access.md | | [Multi-Project Tree View](PROPOSAL_multi_project_tree.md) | Draft | Mendix solutions increasingly span multiple applications: a frontend and a backend, or a microservices landscape (product catalog, orders, f | @@ -130,12 +131,13 @@ for display in this README): | [RENAME with Reference Refactoring](PROPOSAL_rename_refactoring.md) | Draft | Renaming entities, microflows, pages, and modules is one of the most common refactoring operations. | | [Replace Generated Playwright Tests with playwright-cli](proposal-playwright-cli.md) | Draft | The current approach (documented in proposal-playwright-testing.md) has Claude Code generate TypeScript test files (.spec.ts), then run them | | [Self-Describing Syntax Feature Registry](syntax-feature-registry.md) | Draft | Branch: research/recursive-help-discovery | +| [Structured description of irreducible microflow graphs](PROPOSAL_structured_microflow_description.md) | Draft | DESCRIBE MICROFLOW renders a microflow's control flow as nested if/then/else. | | [Version-Aware Agent Support](PROPOSAL_version_aware_agent_support.md) | Draft | Three use cases require mxcli to be version-aware at the MDL level: | | [warm dev loop — Docker-free run and iPad split-screen preview](PROPOSAL_mxcli_dev_warm_loop.md) | Draft | Relates to: PROPOSAL_check_mxbuild_gap_heuristics.md (the static-check gate that | | [Workflow / Microflow Syntax Alignment](PROPOSAL_workflow_microflow_syntax_alignment.md) | Draft | MDL spells the same concept differently depending on which document type you are | -## Archived (25) +## Archived (26) Terminal-status proposals. See [archive/](archive/). @@ -143,6 +145,7 @@ Terminal-status proposals. See [archive/](archive/). | Proposal | Status | Summary | |----------|--------|---------| | [Unified Schema Registry](archive/UNIFIED_SCHEMA_REGISTRY.md) | Abandoned | Mendix project metadata (BSON document schemas, widget property structures, MDL keyword | +| [Align the two microflow split statements on one branch syntax](archive/PROPOSAL_split_statement_syntax_alignment.md) | Done | MDL spells Mendix's two multi-way decisions in unrelated shapes, and one of the | | [Architecture Improvements for Agentic Development and Reduced Merge Conflicts](archive/PROPOSAL_agentic_architecture_improvements.md) | Done | Two related friction points have been identified: | | [Augment Widget Templates from .mpk at Runtime](archive/PROPOSAL_mpk_widget_augmentation.md) | Done | When mxcli creates pluggable widgets (ComboBox, DataGrid2, etc.), it uses static JSON templates extracted from a Mendix 11.6.0 project. | | [Business Events Support in mxcli](archive/PROPOSAL_business_events_support.md) | Done | Business Events is a Mendix feature for asynchronous event-driven integration, allowing applications to publish and subscribe to business ev | diff --git a/docs/11-proposals/archive/PROPOSAL_microflow_enum_split_statement.md b/docs/11-proposals/archive/PROPOSAL_microflow_enum_split_statement.md index a9c0a9c66..46c0c21b1 100644 --- a/docs/11-proposals/archive/PROPOSAL_microflow_enum_split_statement.md +++ b/docs/11-proposals/archive/PROPOSAL_microflow_enum_split_statement.md @@ -15,20 +15,26 @@ Round-trip MDL support for enumeration decisions using SQL/OQL-style `CASE WHEN case $Status when Open, Pending then return true; - when (empty) then + when Closed then return false; - else + when (empty) then return false; end case; ``` +> The original version of this example carried an `else` branch. **MDL008 rejects +> that** — a Mendix enum split is exclusive, with one outgoing flow per value and +> no default. Cover every value explicitly; `(empty)` may share a branch with a +> real value. Corrected while investigating mxcli #913, where the stale example +> was one of the surfaces contradicting the shipped behaviour. + ## Motivation Studio Pro represents enumeration decisions as exclusive splits whose outgoing sequence flows carry enumeration case values. Without a first-class MDL statement, describe/exec round-trips collapse those structures into boolean-looking decisions or unsupported comments. ## Semantics -`case` evaluates an enumeration variable or attribute path. Each `when` lists one or more enumeration values (bare identifiers, consistent with all other enum value references in MDL) that enter the same branch. `(empty)` represents the Mendix empty enumeration case. `else` is optional and maps to the outgoing flow without an explicit case value. The enum type is inferred from the variable's declared type — no explicit type annotation is needed at the call site. Maximum 16 cases are supported; a clear error is raised if exceeded. +`case` evaluates an enumeration variable or attribute path. Each `when` lists one or more enumeration values (bare identifiers, consistent with all other enum value references in MDL) that enter the same branch. `(empty)` represents the Mendix empty enumeration case, and a branch for it is **required** (MDL056; mxbuild reports CE0079 without one). There is **no** `else`: MDL008 rejects it, because a Mendix enum split is exclusive with one outgoing flow per value and no default. The enum type is inferred from the variable's declared type — no explicit type annotation is needed at the call site. Maximum 16 cases are supported; a clear error is raised if exceeded. ## Tests And Examples diff --git a/docs/11-proposals/archive/PROPOSAL_split_statement_syntax_alignment.md b/docs/11-proposals/archive/PROPOSAL_split_statement_syntax_alignment.md new file mode 100644 index 000000000..8647dd191 --- /dev/null +++ b/docs/11-proposals/archive/PROPOSAL_split_statement_syntax_alignment.md @@ -0,0 +1,233 @@ +--- +title: Align the two microflow split statements on one branch syntax +status: done +date: 2026-08-20 +shipped-in: unreleased +related: + - mendixlabs/mxcli#913 + - docs/12-bug-reports/2026-08-20-issue-913-split-statement-syntax.md + - docs/13-decisions/0003-mdl-is-sql-shaped.md + - PROPOSAL_workflow_microflow_syntax_alignment.md + - PROPOSAL_structured_microflow_description.md +--- + +# Proposal: Align the two microflow split statements on one branch syntax + +> **Implemented.** The type split now takes `when then` branches and a +> `when (empty) then` empty branch, matching the enumeration split. The legacy +> `case`/`else` spelling still parses and warns **MDL065**; both spellings are +> pinned to produce the same AST (`TestInheritanceSplit_BothSpellingsProduceTheSameAST`) +> and were measured to describe identically. Branch-body indentation was fixed in +> both emitters and in both splits. Open questions 1, 2 and 4 below were **not** +> taken and remain open. + +## Problem Statement + +MDL spells Mendix's two multi-way decisions in unrelated shapes, and one of the +two uses a keyword that means the opposite of what it says. + +```mdl +case $Status split type $Animal + when Open, Pending then case Zoo.Dog + … … + when (empty) then case Zoo.Cat + … … +end case; else + … + end split; +``` + +Three problems, established in the +[#913 investigation](../12-bug-reports/2026-08-20-issue-913-split-statement-syntax.md): + +1. **`else` on a `split type` is not a default branch.** It is the `(empty)` + flow — taken when the object is **null**. Proven on mxbuild 11.13.0: a split + with `case Zoo.Dog` plus `else` still fails **CE0090** demanding flows for + `Zoo.Cat` and the base `Zoo.Animal`; adding those cases (keeping the `else`) + gives 0 errors. An author who reads `else` as `if`'s `else` writes a branch + that never fires for a real object, and nothing warns them. +2. **`case` has two meanings.** Subject introducer in the enum split and in + `caseExpression`; branch introducer in the type split. Two of three agree. +3. **Nothing transfers.** Knowing one split teaches you nothing about the other: + different opener, different branch keyword, different catch-all, different + terminator. + +The reporter's own summary is accurate: *"use case for cases (not when…), and +not use case where switch/split is meant"*. + +### Relationship to other proposals + +[PROPOSAL_workflow_microflow_syntax_alignment.md](PROPOSAL_workflow_microflow_syntax_alignment.md) +makes the same argument one level up — MDL spells one concept differently +depending on the document type, against `design-mdl-syntax` principle 2 +("never create a second syntax for the same concept") — and observes that +defects cluster in the divergent surface, because a construct with no second +consumer has nothing keeping it honest. The two splits are that pattern inside a +single document type: the inheritance split's branch handling is the surface with +no sibling, and it is where finding 1's silent `else` semantics live. The two +proposals are independent and complementary; neither blocks the other. + +[PROPOSAL_structured_microflow_description.md](PROPOSAL_structured_microflow_description.md) +(#923) owns how DESCRIBE renders graphs that do not nest. This proposal owns what +the statements are *spelled* like. Open question 4 is the one place they meet. + +## Why not `switch` + +The issue proposes aligning on a Java/TypeScript `switch`. Rejecting that, for +three reasons: + +1. **[ADR-0003](../13-decisions/0003-mdl-is-sql-shaped.md) is Accepted and + explicit**: MDL is SQL-shaped, for citizen developers, in the PL/SQL lineage. + `CASE x WHEN a THEN … END CASE` is PL/SQL's statement-CASE verbatim. The + enum split is not an accident — it is the ADR being followed. +2. **MDL already has a second `case … when … then … else … end`** — the + `caseExpression` at `MDLSettings.g4:312`, SQL's searched CASE. A `switch` + statement sitting next to a `case WHEN` expression is *less* consistent than + what exists today, not more. +3. **It would make three spellings, not one.** Existing scripts use `case` and + `split type`; adding `switch` without removing them is the outcome the issue + is complaining about, one iteration later. + +The reporter's *consistency* argument survives all of this. It is the +inheritance split, not the enum split, that is out of line — so that is what +this proposal changes. + +## Proposed MDL Syntax + +Move the type split onto the enum split's branch syntax. `split type` and +`end split` stay (they are descriptive, already shipped, and keep the two +terminators distinguishable in error messages); only the branch keyword and the +catch-all change. + +```mdl +split type $Animal + when Zoo.Dog then + return 'woof'; + when Zoo.Cat then + return 'meow'; + when Zoo.Animal then + return 'generic'; + when (empty) then + return 'no animal'; +end split; +``` + +Against the enum split, now the same statement with a different subject: + +```mdl +case $Status + when Open, Pending then + return 'active'; + when Closed then + return 'done'; + when (empty) then + return 'unset'; +end case; +``` + +What this buys: + +| | before | after | +|---|---|---| +| branch keyword | `case` / `when` | `when` in both | +| catch-all | `else` / none | `when (empty) then` in both | +| meaning of the last branch | hidden | stated | +| `case` distinct meanings | 2 | 1 | +| indentation rule | 2 (both wrong) | 1 | + +`when (empty) then` is not a cosmetic rename. It is the accurate name for the +flow, it matches what Studio Pro shows, and it removes the "any other type" +misreading that `else` invites — a misreading mxbuild catches with CE0090 only +because Mendix separately requires every subtype to be covered. + +### Backwards compatibility + +`case Module.Entity` and `else` keep parsing, indefinitely — scripts in the wild +use them, and this is a readability change, not a correctness one. They become +deprecated spellings: + +- `mxcli check` emits a **warning** (new rule, e.g. `MDL0xx`) naming the + replacement, with `else` → `when (empty) then` called out as a *meaning* + clarification rather than a rename. +- `DESCRIBE` emits only the new form. +- No behavioural difference: both spellings build the identical flow. + +## Implementation Plan + +### Files to modify + +| File | Change | +|---|---| +| `mdl/grammar/domains/MDLMicroflow.g4` | `inheritanceSplitCase`: accept `WHEN qualifiedName THEN microflowBody` alongside today's `CASE qualifiedName microflowBody`; accept `WHEN LPAREN EMPTY RPAREN THEN microflowBody` as the empty branch alongside `ELSE microflowBody`. Unambiguous — the rule is only reachable inside `inheritanceSplitStatement`. | +| `mdl/visitor/visitor_microflow*.go` | Map the new forms onto the existing `ast.InheritanceSplitStmt` (`Cases`, `ElseBody`). No AST change. | +| `mdl/ast/ast_microflow.go` | Record which spelling was parsed, for the deprecation warning only. | +| `mdl/executor/validate_microflow.go` | New warning rule for the deprecated spellings. Register in `ValidateProgram` so `check` and `exec` agree. | +| `mdl/executor/cmd_microflows_show_helpers.go` | `emitInheritanceSplitStatement`: emit `when … then`, emit the empty branch as `when (empty) then`, keep the existing suppression when its body is empty. Fix both emitters' indentation (see below). | +| `.claude/skills/mendix/write-microflows.md`, `docs/01-project/MDL_QUICK_REFERENCE.md`, `cmd/mxcli/syntax/features_*.go` | New spelling; state what the empty branch means. | +| `docs/11-proposals/PROPOSAL_microflow_inheritance_split_statement.md` | Correct the `else` description; status is stale at `draft`. | +| `docs/11-proposals/archive/PROPOSAL_microflow_enum_split_statement.md` | Its example shows an `else` on a `case`, which MDL008 rejects. | + +The builder (`cmd_microflows_builder_actions.go`) does **not** change: both +spellings already funnel into `s.ElseBody` → `addBranch("", …)`, which is the +`(empty)` flow that CE0089 requires. + +### Indentation (can land first, independently) + +Both emitters are wrong, in opposite directions, and no test pins either: + +- enum split — `when` at `indentStr+" "`, body traversed at `indent+1`: the + same column. Body should traverse at `indent+2`. +- inheritance split — branch at `indentStr`, flush with `split type` and + `end split`. Branch should be at `indentStr+" "` and its body at `indent+2`. + +Target, matching `if/else` and the repo's hand-written fixtures: + +``` + case $Status + when Open then + return 'open'; + end case; +``` + +This is a two-line change with no grammar or AST impact. It is worth landing on +its own, ahead of the syntax change, because it fixes the unreadable output in +finding 2 of the bug report without waiting on a language decision. + +## Version Compatibility + +None. This is an MDL surface change with no Mendix version dependency — both +spellings produce byte-identical documents. No `sdk/versions/*.yaml` entry, no +`checkFeature()` gate. + +## Test Plan + +| Test | Where | Asserts | +|---|---|---| +| `913-split-type-when-syntax.mdl` | `mdl-examples/bug-tests/` | New spelling parses and executes; `mx check` 0 errors | +| `913-split-type-legacy-syntax.mdl` | `mdl-examples/bug-tests/` | Old spelling still parses and executes | +| equivalence test | `mdl/backend/modelsdk/` | Both spellings produce the same flows and case values | +| `913-split-type-else-is-empty.mdl` | `mdl-examples/bug-tests/` | Pins finding 1: `case Dog` + `else` fails CE0090 for `Cat` and the base; the same split with every type covered passes. **The control is the point** — without the second run the CE0090s could be read as an unrelated coverage bug | +| deprecation warning | `mdl/executor/` | Old spelling warns, new spelling does not; warning is a warning, so `exec` still runs | +| describe roundtrip | `mdl/executor/` | DESCRIBE emits the new spelling; describe → exec → describe is stable | +| indentation | `mdl/executor/` | Branch bodies indent one level from their branch keyword, in both splits and matching `if`. No such test exists today, which is why the bug shipped | + +Per the repo checklist, the indentation test must be shown to fail against the +current emitters before the fix lands. + +## Open Questions + +1. **Should `split type` become `case type`?** It would make the two statements + textually identical apart from the subject (`case type $x … end case`). + Rejected here because a shared `end case` makes a mismatched terminator + ambiguous to diagnose, and `split type` is descriptive and already shipped — + but it is the cleaner endpoint if churn is acceptable. +2. **Should the enum split accept `else` as an alias for `when (empty) then`?** + No, as proposed — MDL008 rejects `else` on a `case` today and the error is + good. Noted only because the archived enum-split proposal's example uses it. +3. **Does the deprecation ever end?** Suggest: never remove. The parse cost is + one alternative and the warning does the teaching. +4. **Multi-value type branches** — `when Zoo.Dog, Zoo.Cat then` is expressible in + the new grammar and is what the enum split allows. Not proposed, because it + would render as the non-nesting graph described in + [#923](https://github.com/mendixlabs/mxcli/issues/923). Worth revisiting once + that is resolved. diff --git a/docs/12-bug-reports/2026-08-20-issue-913-split-statement-syntax.md b/docs/12-bug-reports/2026-08-20-issue-913-split-statement-syntax.md new file mode 100644 index 000000000..5987c72b4 --- /dev/null +++ b/docs/12-bug-reports/2026-08-20-issue-913-split-statement-syntax.md @@ -0,0 +1,276 @@ +# #913 — MDL decision (split) syntax is inconsistent + +- **Reported**: 2026-08-17 by @mkrouwel, against mxcli v0.18 +- **Investigated**: 2026-08-20, against `766cdf6` (also reproduced on `c134251`) +- **Upstream**: [mendixlabs/mxcli#913](https://github.com/mendixlabs/mxcli/issues/913) +- **Proposal**: [PROPOSAL_split_statement_syntax_alignment.md](../11-proposals/archive/PROPOSAL_split_statement_syntax_alignment.md) — **implemented** + +## What was reported + +MDL renders Mendix's three decision types in three unrelated shapes: + +| Mendix decision | MDL | +|---|---| +| Boolean | `if … then … else … end if;` | +| Enumeration | `case $x when _a then (…) when (empty) then (…) end case;` | +| Object type | `split type $x case Module.EntityA (…) else return; end split;` | + +The reporter asks for alignment on a Java/TypeScript `switch`, with `case` for +branches, `default`/`empty` for the catch-all, and "right indentation". + +## Verdict + +The complaint is valid; the proposed remedy is not. Investigation found **three +defects**, only one of which the issue names, plus one interaction worth routing +elsewhere. The `switch` proposal is rejected — see the proposal document for why +and for the alternative that satisfies the same complaint. + +## Reproduction + +Environment: mxbuild 11.13.0, blank project, `bin/mxcli` at `766cdf6`. + +```bash +mxcli exec i913.mdl -p Repro.mpr # author the splits +mxcli -p Repro.mpr -c "describe microflow I913.X" +~/.mxcli/mxbuild/11.13.0/modeler/mx check Repro.mpr +``` + +Fixtures used are reproduced inline below; each finding names the exact command +that produced its evidence. + +--- + +## Finding 1 — `else` on a `split type` is not a default branch (it is `(empty)`) + +**Severity: high.** Silently wrong semantics, no diagnostic, contradicts the +keyword's meaning everywhere else in MDL. + +`else` on an inheritance split maps to the outgoing flow with **no** +`InheritanceCase` — which on a Mendix object-type decision is the `(empty)` +flow, taken when the object is **null**. It is not "any other type". An author +who reads `else` the way `if`'s `else` reads gets a branch that never fires for +a real object. + +This is stated in the builder +(`mdl/executor/cmd_microflows_builder_actions.go`, the `addBranch("", s.ElseBody)` +call and its comment) but nowhere a user would look. + +**Proof.** One case plus `else`, on a three-entity hierarchy: + +```mdl +split type $Animal + case I913.Dog + return 'dog'; + else + return 'other'; +end split; +``` + +`mx check` on the resulting project: + +``` +[error] [CE0090] "The 'I913.Animal' value should be configured for an outgoing flow." at Object type decision '' +[error] [CE0090] "The 'I913.Cat' value should be configured for an outgoing flow." at Object type decision '' +The app contains: 2 errors. +``` + +mxbuild demands a flow for `Cat` and for the base `Animal` **even though an +`else` is present**. If `else` were a default, neither error would be raised. + +**Positive control.** Give every type its own case, keep the `else`: + +```mdl +split type $Animal + case I913.Dog return 'dog'; + case I913.Cat return 'cat'; + case I913.Animal return 'animal'; + else return 'other'; +end split; +``` + +``` +The app contains: 0 errors. +``` + +The `else` contributes nothing to type coverage in either run. It is the +`(empty)` flow and only that. + +Related: [`.claude/skills/fix-issue.md`](../../.claude/skills/fix-issue.md) already +records that this flow is load-bearing (dropping it fails **CE0089**) and that +`else` cannot substitute for the base entity's case (**CE0090**). What is missing +is that MDL spells it `else`, which reads as the opposite. + +--- + +## Finding 2 — DESCRIBE indents the two splits inconsistently, and neither matches `if` + +**Severity: medium.** Output is ambiguous to read and contradicts mxcli's own +documented examples. No test pins the current behaviour, so it is a free fix. + +Column-annotated `describe microflow`, single-value branches (nothing else in +play): + +``` +col2 | case $Status +col4 | when Open then +col4 | return 'open'; <- body at the SAME column as `when` +col4 | when Pending then +col4 | return 'pending'; +col2 | end case; +``` + +``` +col2 | split type $Animal +col2 | case I913.Dog <- branch at the SAME column as `split type` +col4 | declare $y String = 'woof'; +col2 | case I913.Cat +col2 | else +col4 | return 'other'; +col2 | end split; +``` + +``` +col2 | if $Flag then <- the correct reference +col4 | return 'yes'; +col2 | else +col4 | return 'no'; +col2 | end if; +``` + +Two bugs pointing opposite ways: + +- **enum split** — `when` is indented from `case`, but the branch body is *not* + indented from `when`. `mdl/executor/cmd_microflows_show_helpers.go:1575` writes + `when` at `indentStr+" "` and then traverses the body at `indent+1`, which is + the same column. +- **inheritance split** — `case` is *not* indented from `split type`, sitting + flush with `split type` and `end split`, while its body is indented. + Same file, line 1612 vs 1626. + +Neither matches `if/else`. The enum form also contradicts every hand-written +example in the repo — `mdl-examples/bug-tests/907-case-enum-split-is-supported.mdl` +puts `case` at 2, `when` at 4, body at 6. + +**Why it matters beyond taste.** Nest an `if` inside a case branch and the +output cannot be parsed by eye: + +``` +col2 | case $Status +col4 | when Open then +col4 | if $Flag then +col6 | return 'open-yes'; +col4 | else <- reads as the case's else, which is MDL008 +col6 | return 'open-no'; +col4 | end if; +col4 | when (empty) then +col4 | return 'unset'; <- (empty) body, or a sibling of the `when`s? +col2 | end case; +``` + +The `else` on line 4 belongs to the nested `if`, but sits at exactly the column +a reader expects a `case` branch — and an `else` on a `case` is an **MDL008** +error. DESCRIBE output is the template LLM authors copy from; this shape teaches +a spelling `mxcli check` rejects. + +--- + +## Finding 3 — `case` has two meanings + +**Severity: low on its own, but it is the issue's actual point and it is correct.** + +| Site | Role of `case` | Branch keyword | +|---|---|---| +| `caseExpression` (`MDLSettings.g4:312`) | subject / searched introducer | `when … then` | +| enum split (`MDLMicroflow.g4:186`) | subject introducer | `when … then` | +| inheritance split (`MDLMicroflow.g4:208`) | **branch** introducer | — | + +Two of three agree. The inheritance split is the outlier, and it is the one the +reporter singles out ("not use case where switch/split is meant"). The enum +split's `case … when … then … end case` is SQL's statement-`CASE` verbatim and +is what [ADR-0003](../13-decisions/0003-mdl-is-sql-shaped.md) commits the language to. + +--- + +## Observation — multi-value `when` branches render as a non-nesting graph + +**Not in scope here; routing to [#923](https://github.com/mendixlabs/mxcli/issues/923).** + +`when Pending, Closed then` — a form mxcli documents and pins in +`907-case-enum-split-is-supported.mdl` — produces two flows to one destination. +DESCRIBE renders the branch **empty** and hoists its body past `end case;`: + +```mdl +case $Status + when Open then return 'open'; + when Pending, Closed then return 'other'; + when (empty) then return 'unset'; +end case; +``` + +describes as + +``` + -- WARNING: the decision at (360, 200) has 4 branches that do not nest … + -- This description is NOT equivalent to the microflow and must not + -- be re-executed over it … (mxcli #923, recombinable) + case $Status + when Open then + return 'open'; + when Pending, Closed then + when (empty) then + return 'unset'; + end case; + return 'other'; +``` + +Two notes for whoever owns #923: + +1. Replacing the multi-value branch with three single-value branches removes the + warning entirely, so the trigger is the shared destination, not the split. +2. The warning says the description "must not be re-executed". Measured, it + round-trips: describe → exec → describe differs only in the split's + `@position` (360,200 → 530,200), with identical statement content. For this + shape the wording looks stronger than the behaviour. Layout drift across a + roundtrip is its own question. + +--- + +## Documentation drift found along the way + +| File | Problem | +|---|---| +| `docs/11-proposals/archive/PROPOSAL_microflow_enum_split_statement.md` | Its worked example includes an `else` branch on a `case`. MDL008 rejects that. Proposal is `status: done`, so the example ships as the reference. | +| `docs/11-proposals/PROPOSAL_microflow_inheritance_split_statement.md` | "The optional `else` branch maps to the outgoing flow without an inheritance case" — true and useless. Does not say that flow is `(empty)`, i.e. the null-object case. Still `status: draft` though the feature shipped. | +| `.claude/skills/mendix/write-microflows.md` | `split type` example puts `case` flush with `split type`, matching the buggy emitter rather than `if`. | +| `docs/01-project/MDL_QUICK_REFERENCE.md` | Type split row says "Runtime specialization branches" with no mention of what `else` means. | + +## Resolution + +All three findings are fixed. The type split now takes the enumeration split's +branch syntax: + +```mdl +split type $Animal + when Zoo.Dog then … + when Zoo.Animal then … + when (empty) then … -- what `else` always was: the null-object flow +end split; +``` + +| Finding | Fix | +|---|---| +| 1 — `else` means `(empty)` | Branch renamed to `when (empty) then`, which says what it does. `else` still parses and warns **MDL065**, whose message states the `(empty)`/null semantics and the CE0090 consequence rather than reading as a rename. | +| 2 — indentation | Branch bodies now render one level in from their branch keyword, in **both** splits and in **both** emitters (`cmd_microflows_show_helpers.go` for DESCRIBE, `cmd_diff_mdl.go` for diff — the second had the identical bug and was found only by grepping for the pattern). | +| 3 — `case` overloaded | `case` now introduces a subject everywhere and never a branch. | + +The old spelling is kept indefinitely: scripts in the wild use it, and both +spellings build the identical flow — measured by describing two projects built +from the two spellings and diffing (identical), with `mx check` 0 errors on each. + +**Controls.** The indentation test was run against reverted emitters and fails +with the reported symptom (`branch body indent = 4, want 6`) before passing +after. The three DESCRIBE tests that pinned the old spelling failed on the +change and were updated deliberately rather than loosened. + +Not taken: `switch` (rejected, see the proposal), `case type` as a replacement +for `split type`, multi-value type branches, and `else` as an enum-split alias. diff --git a/mdl-examples/bug-tests/913-split-type-empty-is-not-a-default.mdl b/mdl-examples/bug-tests/913-split-type-empty-is-not-a-default.mdl new file mode 100644 index 000000000..90c0df688 --- /dev/null +++ b/mdl-examples/bug-tests/913-split-type-empty-is-not-a-default.mdl @@ -0,0 +1,58 @@ +-- ============================================================================ +-- #913 — `when (empty) then` is the NULL-object branch, not a default +-- ============================================================================ +-- +-- POSITIVE TEST: `mxcli check` MUST accept this file. +-- +-- The single most misleading thing about the old `split type ... else` spelling +-- was that `else` reads as "any other type" and is not that. It maps to +-- Mendix's `(empty)` outgoing flow, taken when the object is NULL. Renaming it +-- to `when (empty) then` (MDL065 warns on the old name) makes the branch say +-- what it does. +-- +-- MEASURED on mxbuild 11.13.0, three-entity hierarchy, one case plus an else: +-- +-- split type $Animal +-- case Zoo.Dog return 'dog'; +-- else return 'other'; +-- end split; +-- +-- [error] [CE0090] "The 'Zoo.Animal' value should be configured for an outgoing flow." +-- [error] [CE0090] "The 'Zoo.Cat' value should be configured for an outgoing flow." +-- The app contains: 2 errors. +-- +-- The CONTROL is the point: give every type its own branch, KEEP the empty +-- branch, and it is 0 errors. So the empty branch contributes nothing to type +-- coverage — which is exactly what its name now promises. Without that second +-- run the CE0090s read as an unrelated coverage bug. +-- +-- Removing the empty branch is not the fix either: it is CE0089 "The '(empty)' +-- value should be configured for an outgoing flow". mxcli's builder emits the +-- flow unconditionally for that reason, so MDL cannot express a split without +-- one, and DESCRIBE suppresses it only when its body is empty. +-- +-- To handle "any subtype I did not name", branch on the BASE entity — that is +-- what `when Zoo.Animal then` does below. + +CREATE MODULE Bug913E; + +CREATE OR MODIFY PERSISTENT ENTITY Bug913E.Animal ( "Name": String(50) ); +CREATE OR MODIFY PERSISTENT ENTITY Bug913E.Dog GENERALIZATION Bug913E.Animal (); +CREATE OR MODIFY PERSISTENT ENTITY Bug913E.Cat GENERALIZATION Bug913E.Animal (); + +CREATE OR MODIFY MICROFLOW Bug913E.EveryBranchCovered ($Animal: Bug913E.Animal) +RETURNS String +BEGIN + split type $Animal + when Bug913E.Dog then + return 'dog'; + when Bug913E.Cat then + return 'cat'; + -- The base entity: this is the "none of the more specific ones" branch. + when Bug913E.Animal then + return 'some other animal'; + -- NOT a default. Taken only when $Animal is null. + when (empty) then + return 'no animal at all'; + end split; +END; diff --git a/mdl-examples/bug-tests/913-split-type-legacy-syntax.mdl b/mdl-examples/bug-tests/913-split-type-legacy-syntax.mdl new file mode 100644 index 000000000..000aa69cd --- /dev/null +++ b/mdl-examples/bug-tests/913-split-type-legacy-syntax.mdl @@ -0,0 +1,53 @@ +-- ============================================================================ +-- #913 — the pre-unification type split spelling still parses +-- ============================================================================ +-- +-- POSITIVE TEST: `mxcli check` MUST accept this file. It warns MDL065 twice +-- (once for the `case` branches, once for `else`) and must NOT error — scripts +-- in the wild use this spelling, and both spellings build the identical flow. +-- +-- Pinned separately from the current spelling so that "we kept the old form +-- working" is a claim with a test behind it rather than an intention. +-- +-- The equivalence itself is pinned in Go: +-- mdl/visitor — TestInheritanceSplit_BothSpellingsProduceTheSameAST +-- mdl/executor — TestMDL065_DoesNotBlockExecution + +CREATE MODULE Bug913L; + +CREATE OR MODIFY PERSISTENT ENTITY Bug913L.Animal ( "Name": String(50) ); +CREATE OR MODIFY PERSISTENT ENTITY Bug913L.Dog GENERALIZATION Bug913L.Animal (); +CREATE OR MODIFY PERSISTENT ENTITY Bug913L.Cat GENERALIZATION Bug913L.Animal (); + +CREATE OR MODIFY MICROFLOW Bug913L.OnType ($Animal: Bug913L.Animal) +RETURNS String +BEGIN + split type $Animal + case Bug913L.Dog + return 'woof'; + case Bug913L.Cat + return 'meow'; + case Bug913L.Animal + return 'generic'; + else + return 'no animal'; + end split; +END; + +-- Mixing the two spellings in one statement also parses. Nothing depends on +-- it, but a grammar change making the alternatives mutually exclusive would +-- reject scripts mid-migration. +CREATE OR MODIFY MICROFLOW Bug913L.MixedSpelling ($Animal: Bug913L.Animal) +RETURNS String +BEGIN + split type $Animal + case Bug913L.Dog + return 'woof'; + when Bug913L.Cat then + return 'meow'; + when Bug913L.Animal then + return 'generic'; + when (empty) then + return 'no animal'; + end split; +END; diff --git a/mdl-examples/bug-tests/913-split-type-unified-syntax.mdl b/mdl-examples/bug-tests/913-split-type-unified-syntax.mdl new file mode 100644 index 000000000..c67395d65 --- /dev/null +++ b/mdl-examples/bug-tests/913-split-type-unified-syntax.mdl @@ -0,0 +1,75 @@ +-- ============================================================================ +-- #913 — the type split uses the same branch syntax as the enum split +-- ============================================================================ +-- +-- POSITIVE TEST: `mxcli check` MUST accept this file with no warnings. +-- +-- Before #913 the two multi-way splits shared nothing. `case` introduced the +-- SUBJECT in an enum split (and in MDL's caseExpression) but a BRANCH in a type +-- split, so the word meant two things depending on the statement. The legacy +-- spelling still parses — see 913-split-type-legacy-syntax.mdl — but warns +-- MDL065, so it is kept out of this file. +-- +-- Read the two microflows below side by side: one statement, two subjects. + +CREATE MODULE Bug913; + +CREATE OR MODIFY ENUMERATION Bug913.Status ( + Open 'Open', + Pending 'Pending', + Closed 'Closed' +); + +CREATE OR MODIFY PERSISTENT ENTITY Bug913.Animal ( "Name": String(50) ); +CREATE OR MODIFY PERSISTENT ENTITY Bug913.Dog GENERALIZATION Bug913.Animal (); +CREATE OR MODIFY PERSISTENT ENTITY Bug913.Cat GENERALIZATION Bug913.Animal (); + +-- Enumeration split: unchanged by #913, shown here as the shape being matched. +CREATE OR MODIFY MICROFLOW Bug913.OnStatus ($Status: Enumeration(Bug913.Status)) +RETURNS String +BEGIN + case $Status + when Open, Pending then + return 'active'; + when Closed then + return 'done'; + when (empty) then + return 'unset'; + end case; +END; + +-- Type split: same `when ... then` branches, same `when (empty) then`. +-- +-- Every concrete subtype AND the base entity needs its own branch — Mendix +-- fails the build with CE0090 otherwise. The `(empty)` branch does NOT cover +-- them; see 913-split-type-empty-is-not-a-default.mdl. +CREATE OR MODIFY MICROFLOW Bug913.OnType ($Animal: Bug913.Animal) +RETURNS String +BEGIN + split type $Animal + when Bug913.Dog then + return 'woof'; + when Bug913.Cat then + return 'meow'; + when Bug913.Animal then + return 'generic'; + when (empty) then + return 'no animal'; + end split; +END; + +-- Nanoflows take the same form. +CREATE OR MODIFY NANOFLOW Bug913.OnTypeInNanoflow ($Animal: Bug913.Animal) +RETURNS String +BEGIN + split type $Animal + when Bug913.Dog then + return 'woof'; + when Bug913.Cat then + return 'meow'; + when Bug913.Animal then + return 'generic'; + when (empty) then + return 'no animal'; + end split; +END; diff --git a/mdl-examples/bug-tests/925-show-access-on-entity.mdl b/mdl-examples/bug-tests/925-show-access-on-entity.mdl new file mode 100644 index 000000000..1a4462dbd --- /dev/null +++ b/mdl-examples/bug-tests/925-show-access-on-entity.mdl @@ -0,0 +1,57 @@ +-- upstream #925 — `SHOW ACCESS ON ENTITY Mod.Entity` was a parse error, while +-- the MICROFLOW and PAGE spellings parsed. +-- +-- $ mxcli -p app.mpr -c "SHOW ACCESS ON ENTITY eShop.CatalogBrand" +-- Parse error: line 1:60 extraneous input '.' expecting the start of a statement +-- +-- Cause: ENTITY is in the grammar's `keyword` rule, so it is a legal +-- identifierOrKeyword. `showOrList ACCESS ON qualifiedName` therefore matched +-- the word ENTITY as the whole name, the statement ended there, and the real +-- qualified name became extraneous input. (The reported column is offset by the +-- `CONNECT LOCAL ''; ` prefix `-p` prepends to a `-c` string — 27 in the +-- script itself.) +-- +-- The command has always existed: the bare form below is the same statement and +-- the same handler. Only the explicit spelling — the one the CLAUDE.md written +-- by `mxcli init` documents, `SHOW ACCESS ON MICROFLOW|PAGE|ENTITY Mod.Name` — +-- was unreachable. +-- +-- Found while fixing it, and fixed with it: `SHOW ACCESS ON PAGE` PARSED but +-- answered the wrong question. The singular `SHOW PAGE` → DESCRIBE PAGE alias +-- branch sits ~200 lines earlier in the visitor and matched first, so the +-- command printed the page's entire definition instead of its allowed module +-- roles. That is why the issue reports the PAGE variant as working. +-- +-- Run against any project with a persistent entity and a page (names below are +-- from a blank app + Administration). All five must answer in kind: +-- +-- mxcli -p .mpr -c "SHOW ACCESS ON ENTITY Administration.Account" +-- → Access rules for Administration.Account: … +-- mxcli -p .mpr -c "SHOW ACCESS ON PAGE Administration.Account_Overview" +-- → Allowed module roles for Administration.Account_Overview: … +-- +-- Unit coverage: mdl/visitor/visitor_show_access_test.go + +-- The regression: this line is the whole issue. +show access on entity Administration.Account; +/ + +-- Its synonym, which always worked. The two must print the same report. +show access on Administration.Account; +/ + +-- LIST is the other half of showOrList, so it must reach the same statement. +list access on entity Administration.Account; +/ + +-- The second defect: this printed the page's widget tree before the fix. +show access on page Administration.Account_Overview; +/ + +-- Controls — the guards must not steal the plain spellings, which still +-- describe the document rather than reporting access. +show entity Administration.Account; +/ + +show page Administration.Account_Overview; +/ diff --git a/mdl-examples/bug-tests/931-accordion-hidden-property.mdl b/mdl-examples/bug-tests/931-accordion-hidden-property.mdl new file mode 100644 index 000000000..4458dd410 --- /dev/null +++ b/mdl-examples/bug-tests/931-accordion-hidden-property.mdl @@ -0,0 +1,149 @@ +-- ============================================================================ +-- Issue mendixlabs/mxcli#931 — a hidden pluggable-widget property fails the build +-- ============================================================================ +-- +-- A pluggable widget's compiled editorConfig.js can HIDE a property under some +-- configurations of the same widget, and mxbuild evaluates that logic: a hidden +-- property must hold its DEFAULT value. A non-default one is CE0463 "the +-- definition of this widget has changed", which fails `mx check` and makes +-- `mx create-module-package` refuse the entire module. +-- +-- The reporter's Accordion (verbatim from the issue) is that shape: +-- +-- PLUGGABLEWIDGET 'com.mendix.widget.web.accordion.Accordion' accordion1 ( +-- advancedMode: false, collapsible: false, ... +-- ) { +-- GROUP group1 (..., InitialCollapsedState: 'expanded', +-- InitiallyCollapsed: 'false') +-- } +-- +-- With `collapsible: false` the widget hides its whole State group, so both +-- values are non-defaults on hidden properties. Measured on mxbuild 11.13.0 +-- against a blank app — one property changed per run, everything else at its +-- default: +-- +-- collapsible: false + InitialCollapsedState: 'expanded' → CE0463 +-- collapsible: false + InitiallyCollapsed: 'false' → CE0463 +-- collapsible: false + expandBehavior: multipleExpanded → CE0463 +-- collapsible: false + animate: false → CE0463 +-- collapsible: TRUE + every one of those values → 0 errors +-- collapsible: false + every value left at its default → 0 errors +-- +-- So the widget IS authorable; the combination is not. Before the fix mxcli +-- emitted four MDL-WIDGET10 warnings — all on properties the script had left at +-- their defaults, i.e. all harmless — and said nothing about the two that break +-- the build. +-- +-- Verify (needs -p: the rules are lifted from the project's installed .mpk, so a +-- project-less `mxcli check` cannot see them): +-- +-- mxcli check 931-accordion-hidden-property.mdl -p app.mpr +-- → the two commented-out statements below each report MDL-WIDGET10 as an +-- ERROR naming the property and its default +-- mxcli exec 931-accordion-hidden-property.mdl -p app.mpr +-- mx check app.mpr # 0 errors +-- mx create-module-package app.mpr M931 # Export successful +-- ============================================================================ + +create module M931; +/ + +-- The reporter's page, corrected: `collapsible: true` shows the State group, so +-- the same group values are legal. This is the statement that must execute and +-- build cleanly. +create or modify page M931.AccordionOK ( + Title: 'Accordion', + Layout: Atlas_Core.Atlas_Default +) +{ + PLUGGABLEWIDGET 'com.mendix.widget.web.accordion.Accordion' accordion1 ( + advancedMode: false, collapsible: true, expandBehavior: singleExpanded, + animate: true, showIcon: right, animateIcon: true + ) { + GROUP group1 (HeaderRenderMode: 'text', HeaderText: 'Odoo Connector', + HeaderHeading: 'headingThree', Visible: 'true', + LoadContent: 'always', InitialCollapsedState: 'expanded', + InitiallyCollapsed: 'true') + } +} +/ + +-- The failing shapes, kept as documentation. Uncomment either one and re-run +-- `mxcli check ... -p app.mpr` to see the error; `mx check` reports CE0463 on +-- the same widget. +-- +-- (1) hidden by a WIDGET property — `collapsible` is off: +-- +-- create or modify page M931.AccordionCE0463 ( Title: 'x', Layout: Atlas_Core.Atlas_Default ) { +-- PLUGGABLEWIDGET 'com.mendix.widget.web.accordion.Accordion' accordion1 ( +-- advancedMode: false, collapsible: false, expandBehavior: singleExpanded, +-- animate: true, showIcon: right, animateIcon: true +-- ) { +-- GROUP group1 (HeaderRenderMode: 'text', HeaderText: 'Odoo Connector', +-- HeaderHeading: 'headingThree', Visible: 'true', +-- LoadContent: 'always', InitialCollapsedState: 'expanded', +-- InitiallyCollapsed: 'false') +-- } +-- } +-- +-- (2) hidden by the GROUP's OWN property — `initiallyCollapsed` applies only +-- while `initialCollapsedState` is 'dynamic', so a non-default value here is +-- CE0463 even with `collapsible: true`: +-- +-- create or modify page M931.AccordionItemScope ( Title: 'x', Layout: Atlas_Core.Atlas_Default ) { +-- PLUGGABLEWIDGET 'com.mendix.widget.web.accordion.Accordion' accordion1 ( +-- advancedMode: false, collapsible: true, expandBehavior: singleExpanded, +-- animate: true, showIcon: right, animateIcon: true +-- ) { +-- GROUP group1 (HeaderRenderMode: 'text', HeaderText: 'H', +-- HeaderHeading: 'headingThree', Visible: 'true', +-- LoadContent: 'always', InitialCollapsedState: 'expanded', +-- InitiallyCollapsed: 'false') +-- } +-- } + +-- ---------------------------------------------------------------------------- +-- The headline crash of #931, which the Accordion did not cause. +-- +-- `mx create-module-package` aborted on any module mxcli created, with +-- +-- Exception occurred: Unable to cast object of type +-- 'Newtonsoft.Json.Linq.JValue' to type 'Newtonsoft.Json.Linq.JObject' +-- at MprProperty.Init → MprUnit.get_Contents → MprDocumentHasher.Write +-- +-- while `mx check` reported 0 errors. `Forms$Page.AllowedModuleRoles` was written +-- with typed-array marker 3; the marker tells Mendix's reader what the entries +-- are, and the entries are qualified-name STRINGS. An empty list never crashed — +-- which is why it took a page carrying a role, and `create module` grants one. +-- +-- Creating the module above is therefore already the reproduction. The check is +-- the export, not the build: +-- +-- mx check app.mpr # 0 errors, before AND after +-- mx create-module-package app.mpr M931 # crashed before, exports after +-- ---------------------------------------------------------------------------- + +-- Second half of the same issue: a design property set through ALTER STYLING on +-- a widget mxcli itself created used to report success and store nothing. The +-- widget carried no `Appearance.DesignProperties` array, and `bsonnav.DSet` +-- cannot add an absent key — so the write silently did nothing. +-- +-- After the fix `describe page M931.StylingOK` shows +-- `layoutgrid lg1 (DesignProperties: ['Row gap': 'Large'])`. +create or modify page M931.StylingOK ( + Title: 'Styling', + Layout: Atlas_Core.Atlas_Default +) +{ + LAYOUTGRID lg1 { + ROW r1 { + COLUMN c1 (DesktopWidth: AutoFill) { + DYNAMICTEXT t1 (Content: 'design property target', RenderMode: P) + } + } + } +} +/ + +ALTER STYLING ON PAGE M931.StylingOK WIDGET lg1 + SET 'Row gap' = 'Large'; diff --git a/mdl-examples/bug-tests/932-folder-on-modify.mdl b/mdl-examples/bug-tests/932-folder-on-modify.mdl new file mode 100644 index 000000000..93cfab0f7 --- /dev/null +++ b/mdl-examples/bug-tests/932-folder-on-modify.mdl @@ -0,0 +1,86 @@ +-- Issue #932: a FOLDER clause on a document that already exists was accepted, +-- reported as a success, and silently dropped. +-- +-- The placement lives in the unit's containment row, not in the document's +-- contents, and every Update* rewrites contents only — so this was true of +-- EVERY doctype with a FOLDER clause, not just the JSON structures reported. +-- `resolveFolder` still ran, so the folder was created: the model was left with +-- a new empty folder and the document exactly where it started. +-- +-- Run twice. The second run must report every statement as "Unchanged" and +-- leave the .mpr byte-identical (ADR-0008) — a move to the container a document +-- already occupies is not a write. + +create module Bug932; +/ + +-- Everything is created at the module root first, so the folder clauses below +-- are genuinely moves rather than initial placements. +create or modify json structure Bug932.JSON_Example + snippet '{"result": [{"id": 1}]}'; +/ + +create or modify microflow Bug932.ACT_Example() +returns boolean as $ok +begin + declare $ok boolean = true; + return $ok; +end; +/ + +create or modify enumeration Bug932.ENUM_Example (Draft 'Draft', Published 'Published'); +/ + +create or modify constant Bug932.CONST_Example type string default 'x'; +/ + +-- Now file each one away. Before the fix every one of these printed a success +-- and moved nothing. +create or modify json structure Bug932.JSON_Example + folder 'Private/JSON Structures' + snippet '{"result": [{"id": 1}]}'; +/ + +create or modify microflow Bug932.ACT_Example() +returns boolean as $ok +folder 'Private/Microflows' +begin + declare $ok boolean = true; + return $ok; +end; +/ + +create or modify enumeration Bug932.ENUM_Example (Draft 'Draft', Published 'Published') + folder 'Private'; +/ + +create or modify constant Bug932.CONST_Example type string default 'x' + folder 'Private'; +/ + +-- Expected layout: nothing left at the module root, and no empty folders. +-- Doctypes that had no folder clause at all before #932: they could only ever +-- be created at the module root, and MOVE did not name them either. +create queue Bug932.Q_Example folder 'Private/Queues' ( Parallelism: 3 ); +/ + +create regular expression Bug932.RE_Digits folder 'Private/Patterns' ( Expression: '[0-9]+' ); +/ + +create java action Bug932.JA_Example folder 'Private/Java' () + returns string as $$return null;$$; +/ + +show folders in Bug932; +/ + +-- The control for the fix, and the one that decides whether it is a fix or a +-- new bug: a statement that says nothing about folders must leave placement +-- alone. If this filed JSON_Example back into the module root, every +-- CREATE OR MODIFY in every existing script would unfile its document. +create or modify json structure Bug932.JSON_Example + snippet '{"result": [{"id": 1}, {"id": 2}]}'; +/ + +show folders in Bug932; +/ diff --git a/mdl-examples/bug-tests/941-describe-page-datasources.mdl b/mdl-examples/bug-tests/941-describe-page-datasources.mdl new file mode 100644 index 000000000..860cb1fbe --- /dev/null +++ b/mdl-examples/bug-tests/941-describe-page-datasources.mdl @@ -0,0 +1,103 @@ +-- Issue #941: DESCRIBE PAGE emitted invalid MDL for non-database datasources. +-- +-- Three defects, one cause: the datasource switch was copied per widget family +-- (five copies on the read side, six on the write side) and the copies drifted. +-- +-- 1. A chart series bound to a MICROFLOW described as +-- `DataSource: database from Module.TheMicroflow`, because the object-list +-- item emitter had no type switch at all. Re-executing that reported +-- "entity not found: Module.TheMicroflow". +-- 2. A pluggable list whose datasource carried no entity described as +-- `DataSource: database from ` — an empty slot, which the parser reads as +-- an entity called "from" ("entity not found: from"). +-- 3. A GALLERY bound over an association lost its datasource entirely, while +-- a LISTVIEW bound the same way kept it. Silent loss, not a parse error. +-- +-- Run, then `describe page` each page below and re-run `mxcli check` on the +-- output: every datasource must come back with the kind it went in with. + +create module Bug941; +/ + +create entity Bug941.Item ( Title: string(200), Qty: integer ); +/ + +create entity Bug941.Bucket ( Label: string(100) ); +/ + +create association Bug941.Item_Bucket from Bug941.Item to Bug941.Bucket; +/ + +create microflow Bug941.DS_GetItems () +returns list of Bug941.Item as $Items +begin + retrieve $Items from Bug941.Item; + return $Items; +end; +/ + +-- Defect 3: the gallery's association source used to be dropped on describe, +-- the listview's survived. Both are in one page so the asymmetry is visible. +create or modify page Bug941.PgContext + ( Title: 'Context sources', Layout: Atlas_Core.PopupLayout, + Params: { $Bucket: Bug941.Bucket } ) +{ + dataview dvBucket (DataSource: $Bucket) { + listview lvAssoc (DataSource: $currentObject/Bug941.Item_Bucket) { + textbox tbA (Label: 'Title', Attribute: Title) + } + gallery gAssoc (DataSource: $currentObject/Bug941.Item_Bucket) { + textbox tbB (Label: 'Title', Attribute: Title) + } + } +}; +/ + +-- Defect 1: the second series is bound to a microflow. Before the fix DESCRIBE +-- rendered BOTH series as `database from`, and the microflow one then failed +-- reference validation as a missing entity. +create or modify page Bug941.PgChart + ( Title: 'Chart sources', Layout: Atlas_Core.Atlas_Default ) +{ + pluggablewidget 'com.mendix.widget.web.barchart.BarChart' chart1 { + series sDb ( + DataSet: 'static', + DataSource: database from Bug941.Item, + StaticXAttribute: Title, + StaticYAttribute: Qty, + StaticName: 'From database' + ) + series sMf ( + DataSet: 'static', + DataSource: microflow Bug941.DS_GetItems, + StaticXAttribute: Title, + StaticYAttribute: Qty, + StaticName: 'From microflow' + ) + } +}; +/ + +-- The kinds that always worked, kept as the control: a fix that broke these +-- while fixing the others would not be a fix. +create or modify page Bug941.PgOther + ( Title: 'Other sources', Layout: Atlas_Core.Atlas_Default ) +{ + gallery gDb (DataSource: database from Bug941.Item) { + textbox tbC (Label: 'Title', Attribute: Title) + } + gallery gMf (DataSource: microflow Bug941.DS_GetItems) { + textbox tbD (Label: 'Title', Attribute: Title) + } + dataview dvSel (DataSource: selection gDb) { + textbox tbE (Label: 'Title', Attribute: Title) + } +}; +/ + +describe page Bug941.PgContext; +/ +describe page Bug941.PgChart; +/ +describe page Bug941.PgOther; +/ diff --git a/mdl-examples/bug-tests/expect-vacuous-assertions.test.mdl b/mdl-examples/bug-tests/expect-vacuous-assertions.test.mdl index 309aa5b99..0b8b4e291 100644 --- a/mdl-examples/bug-tests/expect-vacuous-assertions.test.mdl +++ b/mdl-examples/bug-tests/expect-vacuous-assertions.test.mdl @@ -34,6 +34,20 @@ -- otherwise. test_3 is an error because its @verify sits on a test using the -- rollback default — see the @verify section below. -- +-- ISSUE #926 -- and it has to be an ERROR on EVERY runner. +-- +-- --require-assertions was honoured only by the endpoint runner, so the two +-- commands below disagreed about the same file: the first errored on test_2, the +-- second exited 0. Whether a test is vacuous is a property of the parsed test +-- case, so both must agree. +-- +-- mxcli test -p app.mpr --local --require-assertions # errored +-- mxcli test -p app.mpr --require-assertions # exited 0 +-- +-- Not reproducible from MDL alone: the defect is in which runner reads the flag, +-- not in anything the script can say. The unit control is +-- TestRequireAssertionsIsHonouredOnTheLogRunner. +-- -- The microflows referenced below do not need to exist for --list. /** diff --git a/mdl-examples/doctype-tests/08-security-examples.mdl b/mdl-examples/doctype-tests/08-security-examples.mdl index 7ef8b386b..58ca53eb4 100644 --- a/mdl-examples/doctype-tests/08-security-examples.mdl +++ b/mdl-examples/doctype-tests/08-security-examples.mdl @@ -150,10 +150,14 @@ show access on microflow SecTest.ACT_Customer_Create; show access on page SecTest.Customer_Overview; / --- Show access rules on a specific entity (CRUD permissions) +-- Show access rules on a specific entity (CRUD permissions). The bare name and +-- the explicit ENTITY spelling are synonyms; #925 had only the bare one. show access on SecTest.Customer; / +show access on entity SecTest.Customer; +/ + -- ============================================================================ -- Level 1.6: Security Matrix -- ============================================================================ diff --git a/mdl-examples/doctype-tests/18-folder-examples.mdl b/mdl-examples/doctype-tests/18-folder-examples.mdl index 06afa13fc..00a6fd8aa 100644 --- a/mdl-examples/doctype-tests/18-folder-examples.mdl +++ b/mdl-examples/doctype-tests/18-folder-examples.mdl @@ -151,3 +151,51 @@ show folders in FolderTest; list folders; drop module FolderTest; + +-- --------------------------------------------------------------------------- +-- #932: MOVE accepts every top-level document type, and a FOLDER clause on +-- CREATE OR MODIFY places an existing document. +-- --------------------------------------------------------------------------- + +create module FolderMove932; +/ + +create entity FolderMove932.Order ( + OrderId: integer, + OrderName: string(200) +); +/ + +create json structure FolderMove932.JSON_Order + snippet '{"id": 1, "name": "Widget"}'; +/ + +create import mapping FolderMove932.IMM_Order + with json structure FolderMove932.JSON_Order +{ + create FolderMove932.Order { + OrderId = id, + OrderName = name + } +}; +/ + +-- Doctypes that MOVE could not name before: the statement was a parse error +-- ("no viable alternative at input 'MOVEIMPORT'"). +move json structure FolderMove932.JSON_Order to folder 'Private/JSON structures'; +/ +move import mapping FolderMove932.IMM_Order to folder 'Private/Import mappings'; +/ + +-- A folder clause on CREATE OR MODIFY is the other way to place a document, +-- and re-running it must be a no-op rather than a second move. +create or modify json structure FolderMove932.JSON_Order + folder 'Private/JSON structures' + snippet '{"id": 1, "name": "Widget"}'; +/ + +show folders in FolderMove932; +/ + +drop module FolderMove932; +/ diff --git a/mdl-examples/doctype-tests/29-listview-specialization-templates.mdl b/mdl-examples/doctype-tests/29-listview-specialization-templates.mdl new file mode 100644 index 000000000..b53cbee9a --- /dev/null +++ b/mdl-examples/doctype-tests/29-listview-specialization-templates.mdl @@ -0,0 +1,128 @@ +-- List View specialization templates (issue mendixlabs/mxcli#940) +-- +-- A List View over a generalization can render a different body per +-- specialization. Studio Pro stores these as Forms$ListViewTemplate elements in +-- a Templates array, alongside — not inside — the list view's own Widgets. +-- +-- Before this syntax existed there was no way to author one, and no way to READ +-- one either: DESCRIBE PAGE dropped them silently, so a describe → exec round +-- trip destroyed them. Measured on ako/TestApp's Pages.Vehicle_Overview: +-- 4 templates before, 0 after, with `mx check` reporting 0 errors both times. +-- +-- Run: +-- mxcli exec mdl-examples/doctype-tests/29-listview-specialization-templates.mdl -p app.mpr +-- mxcli -p app.mpr -c "describe page Vehicles.Vehicle_Overview" +-- +-- The describe output re-executes to the same document, templates included. + +create or modify entity Vehicles.Vehicle ( + Brand: String(100), + Model: String(100), + Year: Integer +); + +create or modify entity Vehicles.Bus extends Vehicles.Vehicle ( + PassengerCapacity: Integer, + RouteNumber: String(20) +); + +create or modify entity Vehicles.Truck extends Vehicles.Vehicle ( + MaxLoadKg: Integer, + HasTrailer: Boolean +); + +create or modify entity Vehicles.Car extends Vehicles.Vehicle ( + NumberOfDoors: Integer +); + +create or modify page Vehicles.Vehicle_Overview ( + Title: 'Vehicle Overview', + Layout: Atlas_Core.Atlas_Default +) { + listview vehicleListView (DataSource: database from Vehicles.Vehicle) { + -- The default body: what a Vehicle that is none of the specializations + -- below renders as. This is the list view's own Widgets array. + dynamictext defaultVehicle ( + Content: '{1} {2} ({3})', + ContentParams: [{1} = Brand, {2} = Model, {3} = Year] + ) + + -- One body per specialization. The template has no name — the entity it + -- renders is what identifies it. Order is preserved as written, because + -- Mendix stores and matches in that order. + -- + -- Note each template reads attributes that exist ONLY on its own + -- specialization: inside a template the context object is that entity. + template for Vehicles.Bus { + dynamictext busLabel ( + Content: 'Bus {1} - capacity {2}, route {3}', + ContentParams: [{1} = Brand, {2} = PassengerCapacity, {3} = RouteNumber] + ) + } + template for Vehicles.Truck { + dynamictext truckLabel ( + Content: 'Truck {1} - max load {2} kg, trailer: {3}', + ContentParams: [{1} = Brand, {2} = MaxLoadKg, {3} = HasTrailer] + ) + } + template for Vehicles.Car { + dynamictext carLabel ( + Content: 'Car {1} - {2} doors', + ContentParams: [{1} = Brand, {2} = NumberOfDoors] + ) + } + } +}; + +-- ALTER PAGE: adding a template reuses INSERT INTO with the same block, so a +-- template has one spelling everywhere. Removing one needs its own form, because +-- a template has no name to put in a widget ref. + +create or modify entity Vehicles.Motorcycle extends Vehicles.Vehicle ( + HasSidecar: Boolean +); + +alter page Vehicles.Vehicle_Overview { + insert into vehicleListView { + template for Vehicles.Motorcycle { + dynamictext mcLabel ( + Content: 'Motorcycle {1} - sidecar: {2}', + ContentParams: [{1} = Brand, {2} = HasSidecar] + ) + } + }; + drop template for Vehicles.Car in vehicleListView +}; + +-- Naming the list view in the drop is required, not optional: one page can hold +-- two list views with a template for the same entity. +-- +-- Most template edits need none of this. Widgets inside a template are ordinary +-- named widgets, so this works with no new syntax and lands in the right +-- template: +-- +-- alter page Vehicles.Vehicle_Overview { +-- set Content = 'Bus {1}' on busLabel +-- }; + +-- Refused, each with a message naming the problem: +-- +-- template for Vehicles.Unrelated { ... } +-- -> is not Vehicles.Vehicle or a specialization of it, so the template can +-- never match an object the list view shows +-- +-- two `template for Vehicles.Bus` blocks in one list view +-- -> more than one template for Vehicles.Bus +-- +-- a `template for` nested inside another +-- -> list view templates cannot nest +-- +-- insert before/after a template +-- -> only INSERT INTO: templates are not siblings of the list view's body +-- +-- templates mixed with ordinary widgets in one INSERT +-- -> they go to different places (Templates vs the default body) +-- +-- drop template for an entity with no template +-- -> names the templates that ARE there, rather than dropping nothing and +-- reporting success diff --git a/mdl/ast/ast.go b/mdl/ast/ast.go index 256d2cfd3..005c1a2d4 100644 --- a/mdl/ast/ast.go +++ b/mdl/ast/ast.go @@ -5,6 +5,8 @@ // associations, enumerations, and view entities. package ast +import "strings" + // Statement represents any MDL statement that can be executed. type Statement interface { isStatement() @@ -45,19 +47,102 @@ type Program struct { // DocumentType represents the type of document being moved. type DocumentType string +// The doctypes MOVE accepts. The value is the MDL spelling, which is also what +// the executor prints back — so an unrecognised one cannot be mistaken for a +// recognised one in output. +// +// Only ENTITY is not a top-level document unit: it lives inside a domain model +// and its move converts associations rather than reparenting a row. Everything +// else here reduces to one containment update, which is why doctypes can be +// added to this list without a handler each. const ( - DocumentTypePage DocumentType = "PAGE" - DocumentTypeMicroflow DocumentType = "MICROFLOW" - DocumentTypeSnippet DocumentType = "SNIPPET" - DocumentTypeNanoflow DocumentType = "NANOFLOW" - DocumentTypeEntity DocumentType = "ENTITY" - DocumentTypeEnumeration DocumentType = "ENUMERATION" - DocumentTypeConstant DocumentType = "CONSTANT" - DocumentTypeDatabaseConnection DocumentType = "DATABASE CONNECTION" - DocumentTypeJavaAction DocumentType = "JAVA ACTION" - DocumentTypeODataService DocumentType = "ODATA SERVICE" + DocumentTypePage DocumentType = "PAGE" + DocumentTypeMicroflow DocumentType = "MICROFLOW" + DocumentTypeSnippet DocumentType = "SNIPPET" + DocumentTypeNanoflow DocumentType = "NANOFLOW" + DocumentTypeEntity DocumentType = "ENTITY" + DocumentTypeEnumeration DocumentType = "ENUMERATION" + DocumentTypeConstant DocumentType = "CONSTANT" + DocumentTypeDatabaseConnection DocumentType = "DATABASE CONNECTION" + DocumentTypeJavaAction DocumentType = "JAVA ACTION" + DocumentTypeODataService DocumentType = "ODATA SERVICE" + DocumentTypeBuildingBlock DocumentType = "BUILDING BLOCK" + DocumentTypeLayout DocumentType = "LAYOUT" + DocumentTypeMenu DocumentType = "MENU" + DocumentTypeWorkflow DocumentType = "WORKFLOW" + DocumentTypeQueue DocumentType = "QUEUE" + DocumentTypeScheduledEvent DocumentType = "SCHEDULED EVENT" + DocumentTypeRegularExpression DocumentType = "REGULAR EXPRESSION" + DocumentTypeJsonStructure DocumentType = "JSON STRUCTURE" + DocumentTypeImportMapping DocumentType = "IMPORT MAPPING" + DocumentTypeExportMapping DocumentType = "EXPORT MAPPING" + DocumentTypeJavaScriptAction DocumentType = "JAVASCRIPT ACTION" + DocumentTypeDataTransformer DocumentType = "DATA TRANSFORMER" + DocumentTypeImageCollection DocumentType = "IMAGE COLLECTION" + DocumentTypeIconCollection DocumentType = "ICON COLLECTION" + DocumentTypeRestClient DocumentType = "REST CLIENT" + DocumentTypePublishedRestService DocumentType = "PUBLISHED REST SERVICE" + DocumentTypeODataClient DocumentType = "ODATA CLIENT" + DocumentTypeBusinessEventService DocumentType = "BUSINESS EVENT SERVICE" + DocumentTypeModel DocumentType = "MODEL" + DocumentTypeAgent DocumentType = "AGENT" + DocumentTypeKnowledgeBase DocumentType = "KNOWLEDGE BASE" + DocumentTypeConsumedMCPService DocumentType = "CONSUMED MCP SERVICE" ) +// MoveDocumentTypeByKeyword maps a moveDocumentType grammar rule's text to its +// DocumentType. The rule's GetText() concatenates its tokens with no separator, +// so `IMPORT MAPPING` arrives as "IMPORTMAPPING". +// +// The single registry for the doctypes MOVE accepts: the visitor reads it to +// build the statement, and the executor reads its values to decide whether a +// document it found is the kind the statement named. Keeping one list is the +// point — a second copy is how the MOVE FOLDER discriminator went stale before. +var MoveDocumentTypeByKeyword = map[string]DocumentType{ + "PAGE": DocumentTypePage, + "MICROFLOW": DocumentTypeMicroflow, + "NANOFLOW": DocumentTypeNanoflow, + "SNIPPET": DocumentTypeSnippet, + "BUILDINGBLOCK": DocumentTypeBuildingBlock, + "LAYOUT": DocumentTypeLayout, + "MENU": DocumentTypeMenu, + "ENUMERATION": DocumentTypeEnumeration, + "CONSTANT": DocumentTypeConstant, + "WORKFLOW": DocumentTypeWorkflow, + "QUEUE": DocumentTypeQueue, + "SCHEDULEDEVENT": DocumentTypeScheduledEvent, + "REGULAREXPRESSION": DocumentTypeRegularExpression, + "JSONSTRUCTURE": DocumentTypeJsonStructure, + "IMPORTMAPPING": DocumentTypeImportMapping, + "EXPORTMAPPING": DocumentTypeExportMapping, + "JAVAACTION": DocumentTypeJavaAction, + "JAVASCRIPTACTION": DocumentTypeJavaScriptAction, + "DATABASECONNECTION": DocumentTypeDatabaseConnection, + "DATATRANSFORMER": DocumentTypeDataTransformer, + "IMAGECOLLECTION": DocumentTypeImageCollection, + "ICONCOLLECTION": DocumentTypeIconCollection, + "RESTCLIENT": DocumentTypeRestClient, + "PUBLISHEDRESTSERVICE": DocumentTypePublishedRestService, + "ODATACLIENT": DocumentTypeODataClient, + "ODATASERVICE": DocumentTypeODataService, + "BUSINESSEVENTSERVICE": DocumentTypeBusinessEventService, + "MODEL": DocumentTypeModel, + "AGENT": DocumentTypeAgent, + "KNOWLEDGEBASE": DocumentTypeKnowledgeBase, + "CONSUMEDMCPSERVICE": DocumentTypeConsumedMCPService, +} + +// IsMoveDocumentType reports whether spelling (lower-cased, spaced, e.g. +// "json structure") names a doctype MOVE accepts. +func IsMoveDocumentType(spelling string) bool { + for _, docType := range MoveDocumentTypeByKeyword { + if strings.EqualFold(string(docType), spelling) { + return true + } + } + return false +} + // MoveStmt represents: MOVE PAGE/MICROFLOW/SNIPPET/NANOFLOW/ENTITY/ENUMERATION Module.Name TO FOLDER 'path' IN Module type MoveStmt struct { DocumentType DocumentType // PAGE, MICROFLOW, SNIPPET, NANOFLOW, ENTITY, ENUMERATION diff --git a/mdl/ast/ast_agenteditor.go b/mdl/ast/ast_agenteditor.go index b04264b2a..a744350d9 100644 --- a/mdl/ast/ast_agenteditor.go +++ b/mdl/ast/ast_agenteditor.go @@ -16,6 +16,7 @@ package ast // [, DeepLinkURL: '...'] // ); type CreateModelStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName Documentation string Provider string // "MxCloudGenAI" by default @@ -55,6 +56,7 @@ func (s *AlterModelStmt) isStatement() {} // Documentation: '...' // ); type CreateConsumedMCPServiceStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName OuterDocumentation string // /** ... */ doc comment ProtocolVersion string @@ -89,6 +91,7 @@ func (s *AlterConsumedMCPServiceStmt) isStatement() {} // Key: Module.SomeConstant // ); type CreateKnowledgeBaseStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName Documentation string Provider string @@ -122,6 +125,7 @@ func (s *AlterKnowledgeBaseStmt) isStatement() {} // CreateAgentStmt represents CREATE AGENT Module.Name (...) [{ body }]. type CreateAgentStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName Documentation string UsageType string // "Task" or "Conversational" diff --git a/mdl/ast/ast_alter_page.go b/mdl/ast/ast_alter_page.go index c857084b0..d978efd9c 100644 --- a/mdl/ast/ast_alter_page.go +++ b/mdl/ast/ast_alter_page.go @@ -66,6 +66,19 @@ type DropWidgetOp struct { func (s *DropWidgetOp) isAlterPageOperation() {} +// DropListViewTemplateOp represents: +// DROP TEMPLATE FOR Module.Specialization IN listViewName +// +// A List View specialization template has no name, so it is addressed by the +// entity it renders plus the list view holding it — a page can carry two list +// views with a template for the same entity. +type DropListViewTemplateOp struct { + Specialization string + ListView string +} + +func (s *DropListViewTemplateOp) isAlterPageOperation() {} + // ReplaceWidgetOp represents: REPLACE widgetRef WITH { widgets } type ReplaceWidgetOp struct { Target WidgetRef diff --git a/mdl/ast/ast_datatransformer.go b/mdl/ast/ast_datatransformer.go index 3691e1096..4b7cd504f 100644 --- a/mdl/ast/ast_datatransformer.go +++ b/mdl/ast/ast_datatransformer.go @@ -6,6 +6,7 @@ package ast // // CREATE DATA TRANSFORMER Module.Name SOURCE JSON '...' { JSLT '...'; }; type CreateDataTransformerStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName SourceType string // "JSON" or "XML" SourceJSON string // the source content diff --git a/mdl/ast/ast_imagecollection.go b/mdl/ast/ast_imagecollection.go index 3f25c93e8..926d07ad3 100644 --- a/mdl/ast/ast_imagecollection.go +++ b/mdl/ast/ast_imagecollection.go @@ -12,6 +12,7 @@ type ImageItem struct { // // CREATE IMAGE COLLECTION Module.Name [EXPORT LEVEL 'Public'] [COMMENT '...'] [(IMAGE "name" FROM FILE 'path', ...)] type CreateImageCollectionStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName CreateOrModify bool ExportLevel string // "Hidden" (default) or "Public" diff --git a/mdl/ast/ast_import_export_mapping.go b/mdl/ast/ast_import_export_mapping.go index b075e50a3..0a89f9586 100644 --- a/mdl/ast/ast_import_export_mapping.go +++ b/mdl/ast/ast_import_export_mapping.go @@ -18,6 +18,7 @@ package ast // }; type CreateImportMappingStmt struct { Name QualifiedName + Folder string // Folder path within module (empty = leave placement alone) SchemaKind string // "JSON_STRUCTURE" or "XML_SCHEMA" or "" SchemaRef QualifiedName // qualified name of the schema source RootElement *ImportMappingElementDef @@ -69,6 +70,7 @@ type ImportMappingElementDef struct { // }; type CreateExportMappingStmt struct { Name QualifiedName + Folder string // Folder path within module (empty = leave placement alone) SchemaKind string // "JSON_STRUCTURE" or "XML_SCHEMA" or "" SchemaRef QualifiedName // qualified name of the schema source NullValueOption string // "LeaveOutElement" or "SendAsNil" (default: "LeaveOutElement") diff --git a/mdl/ast/ast_javaaction.go b/mdl/ast/ast_javaaction.go index 17540ddfa..991ed91b0 100644 --- a/mdl/ast/ast_javaaction.go +++ b/mdl/ast/ast_javaaction.go @@ -22,6 +22,7 @@ type JavaActionParam struct { // EXPOSED AS 'caption' IN 'category' // AS $$ ... $$; type CreateJavaActionStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName // Qualified name (Module.ActionName) Parameters []JavaActionParam // Input parameters ReturnType DataType // Return type (can be nil for void) @@ -56,6 +57,7 @@ func (s *DropJavaActionStmt) isStatement() {} // It mirrors CreateJavaActionStmt with an added Platform (Web/Native/Hybrid/All, // default Web). The inline source is JavaScript rather than Java. type CreateJavaScriptActionStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName // Qualified name (Module.ActionName) Parameters []JavaActionParam // Input parameters ReturnType DataType // Return type (can be nil for void) diff --git a/mdl/ast/ast_microflow.go b/mdl/ast/ast_microflow.go index e8fb480f5..784686f7c 100644 --- a/mdl/ast/ast_microflow.go +++ b/mdl/ast/ast_microflow.go @@ -123,11 +123,23 @@ type InheritanceSplitCase struct { } // InheritanceSplitStmt represents: SPLIT TYPE $Var ... END SPLIT +// +// ElseBody is Mendix's `(empty)` outgoing flow — the branch taken when the +// object is NULL. It is NOT a default for unmatched types: mxbuild demands a +// flow for every subtype and for the base entity regardless (CE0090), and +// omitting this one is CE0089. It is spelled `when (empty) then`; `else` is +// the legacy spelling that reads as a default and is not one (mxcli #913). type InheritanceSplitStmt struct { Variable string // Variable name without $ prefix Cases []InheritanceSplitCase ElseBody []MicroflowStatement Annotations *ActivityAnnotations // Optional @position, @caption, @color, @annotation + + // Which spelling the source used, for the MDL065 deprecation warning only. + // Both build the identical flow, so nothing downstream of the validator + // may branch on these. + LegacyCaseKeyword bool // at least one branch used `case X` instead of `when X then` + LegacyElseKeyword bool // the empty branch used `else` instead of `when (empty) then` } func (s *EnumSplitStmt) isMicroflowStatement() {} diff --git a/mdl/ast/ast_navigation.go b/mdl/ast/ast_navigation.go index 20232428d..dde4b3255 100644 --- a/mdl/ast/ast_navigation.go +++ b/mdl/ast/ast_navigation.go @@ -39,6 +39,7 @@ type NavMenuItemDef struct { // Like CREATE NAVIGATION, this is a full replacement: the item list given is the // document's complete contents, so an omitted item is a removed item. type CreateMenuStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName Items []NavMenuItemDef CreateOrModify bool // CREATE OR MODIFY / OR REPLACE diff --git a/mdl/ast/ast_page_v3.go b/mdl/ast/ast_page_v3.go index eb18154d1..eb35c4236 100644 --- a/mdl/ast/ast_page_v3.go +++ b/mdl/ast/ast_page_v3.go @@ -81,6 +81,13 @@ type WidgetV3 struct { Name string // Required widget name Properties map[string]any // All properties as key-value pairs Children []*WidgetV3 // Child widgets + + // Specialization is the entity a List View template renders, set only for + // `template for Module.Entity { ... }`. It is a separate field rather than a + // Property because it identifies the template — a Forms$ListViewTemplate has + // no name, and carries nothing but this entity and its widgets. Empty for + // every other widget, including a Gallery's named `template ` slot. + Specialization string } // DataSourceV3 represents a V3 datasource expression. diff --git a/mdl/ast/ast_queue.go b/mdl/ast/ast_queue.go index 1ad13ee0b..51b767d67 100644 --- a/mdl/ast/ast_queue.go +++ b/mdl/ast/ast_queue.go @@ -6,6 +6,7 @@ package ast // // CREATE [OR REPLACE|MODIFY] QUEUE Module.Name ( Parallelism: 3, ClusterWide: true ); type CreateQueueStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName Documentation string // Parallelism is kept as written. Mendix stores it as an expression string diff --git a/mdl/ast/ast_regularexpression.go b/mdl/ast/ast_regularexpression.go index d82da7e49..94c193542 100644 --- a/mdl/ast/ast_regularexpression.go +++ b/mdl/ast/ast_regularexpression.go @@ -8,6 +8,7 @@ package ast // Expression: '^[a-z]+$' // ); type CreateRegularExpressionStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName Documentation string // Expression is the pattern, as written (unquoted). diff --git a/mdl/ast/ast_scheduledevent.go b/mdl/ast/ast_scheduledevent.go index 7e97bf0ca..38d35e4ec 100644 --- a/mdl/ast/ast_scheduledevent.go +++ b/mdl/ast/ast_scheduledevent.go @@ -13,6 +13,7 @@ package ast // "not mentioned" stays distinguishable from "mentioned as 0" — 0 is a real // hour, minute and month offset. type CreateScheduledEventStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName Documentation string // Microflow is the qualified name of the microflow to run. diff --git a/mdl/ast/ast_sql.go b/mdl/ast/ast_sql.go index d20103841..357873770 100644 --- a/mdl/ast/ast_sql.go +++ b/mdl/ast/ast_sql.go @@ -104,6 +104,7 @@ type DatabaseQueryParamDef struct { // CreateDatabaseConnectionStmt represents: CREATE DATABASE CONNECTION Module.Name ... type CreateDatabaseConnectionStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName DatabaseType string // "PostgreSQL", "MSSQL", "Oracle" ConnectionString string // constant ref or string literal for connection string diff --git a/mdl/ast/ast_workflow.go b/mdl/ast/ast_workflow.go index 82a4561cf..123f481ef 100644 --- a/mdl/ast/ast_workflow.go +++ b/mdl/ast/ast_workflow.go @@ -4,6 +4,7 @@ package ast // CreateWorkflowStmt represents: CREATE WORKFLOW Module.Name ... type CreateWorkflowStmt struct { + Folder string // Folder path within module (empty = leave placement alone) Name QualifiedName CreateOrModify bool Documentation string diff --git a/mdl/backend/backend.go b/mdl/backend/backend.go index 1e7337359..1eb4a3395 100644 --- a/mdl/backend/backend.go +++ b/mdl/backend/backend.go @@ -13,6 +13,7 @@ type FullBackend interface { ModuleBackend ModuleSettingsBackend FolderBackend + DocumentPlacementBackend DomainModelBackend MicroflowBackend PageBackend diff --git a/mdl/backend/bsonnav/bsonnav.go b/mdl/backend/bsonnav/bsonnav.go index 5912844be..6ccd99358 100644 --- a/mdl/backend/bsonnav/bsonnav.go +++ b/mdl/backend/bsonnav/bsonnav.go @@ -48,6 +48,14 @@ func DGetString(doc bson.D, key string) string { // is absent, the mutation is silently skipped — this is intentional for // optional fields (e.g. Appearance, DataSource) that may not be present // on every widget type. +// +// DSet CANNOT ADD an absent key: doc is a bson.D held by value, so appending +// would not be visible to the caller. A mutation that has to work on a document +// which may not carry the key yet must go through DSetArrayIn (or write the +// grown child back into its parent itself) and check the result — otherwise the +// write is a silent no-op. That is what made ALTER STYLING report success and +// store nothing when the target widget had no DesignProperties array +// (upstream #931). func DSet(doc bson.D, key string, value any) bool { for i := range doc { if doc[i].Key == key { @@ -87,26 +95,70 @@ func ToBsonA(v any) []any { } // DSetArray sets a Mendix-style BSON array field, preserving the int32 marker. -func DSetArray(doc bson.D, key string, elements []any) { - existing := ToBsonA(DGet(doc, key)) - var marker any - if len(existing) > 0 { - if _, ok := existing[0].(int32); ok { - marker = existing[0] - } else if _, ok := existing[0].(int); ok { - marker = existing[0] +func DSetArray(doc bson.D, key string, elements []any) bool { + marker, ok := arrayMarker(DGet(doc, key)) + if !ok { + // No marker to preserve. Writing the elements bare would produce a list + // whose first entry Mendix reads as the typed-array marker — measured on + // 11.13, a DesignProperties array written that way makes the project + // unloadable ("Type OptionDesignPropertyValue does not contain a + // constructor with a parameter of type Appearance") and + // create-module-package die with "Unknown export error". Refusing is the + // safe direction; a caller that legitimately creates the property passes + // the marker explicitly via DSetArrayIn. + if len(elements) > 0 { + return false } + return DSet(doc, key, bson.A{}) } - var result bson.A - if marker != nil { - result = make(bson.A, 0, len(elements)+1) - result = append(result, marker) - result = append(result, elements...) - } else { - result = make(bson.A, len(elements)) - copy(result, elements) + return DSet(doc, key, withMarker(marker, elements)) +} + +// DSetArrayIn sets an array property on a child document of parent, CREATING it +// with the given typed-array marker when the child does not carry it yet. +// +// It exists because DSet cannot add an absent key to a bson.D held by value: the +// grown child has to be written back into its parent, which needs the parent. +// Returns false when parent has no such child document — nothing was written. +func DSetArrayIn(parent bson.D, childKey, arrayKey string, elements []any, marker int32) bool { + child := DGetDoc(parent, childKey) + if child == nil { + return false + } + existing, ok := arrayMarker(DGet(child, arrayKey)) + if !ok { + existing = marker + } + value := withMarker(existing, elements) + if DSet(child, arrayKey, value) { + return true + } + // The child does not carry the key at all — append and write the grown child + // back into the parent. + return DSet(parent, childKey, append(child, bson.E{Key: arrayKey, Value: value})) +} + +// arrayMarker returns the leading typed-array marker of a Mendix array value. +func arrayMarker(val any) (int32, bool) { + arr := ToBsonA(val) + if len(arr) == 0 { + return 0, false } - DSet(doc, key, result) + switch m := arr[0].(type) { + case int32: + return m, true + case int: + return int32(m), true + } + return 0, false +} + +// withMarker builds a Mendix array value: the typed-array marker followed by the +// elements. +func withMarker(marker int32, elements []any) bson.A { + result := make(bson.A, 0, len(elements)+1) + result = append(result, marker) + return append(result, elements...) } // ExtractBinaryIDFromDoc extracts a binary ID string from a bson.D field. diff --git a/mdl/backend/mcp/page_mutator.go b/mdl/backend/mcp/page_mutator.go index f06cb1de2..df882dc33 100644 --- a/mdl/backend/mcp/page_mutator.go +++ b/mdl/backend/mcp/page_mutator.go @@ -482,6 +482,15 @@ func (m *mcpPageMutator) InsertColumns(gridRef, afterColumnRef string, _ backend return fmt.Errorf("inserting columns into %s is not yet supported by the MCP backend", gridRef) } +func (m *mcpPageMutator) InsertListViewTemplates(listViewRef string, _ []*pages.ListViewTemplate) error { + return fmt.Errorf("adding specialization templates to %s is not yet supported by the MCP backend", listViewRef) +} + +func (m *mcpPageMutator) DropListViewTemplate(listViewRef, specialization string) error { + return fmt.Errorf("dropping the %s template from %s is not yet supported by the MCP backend", + specialization, listViewRef) +} + func (m *mcpPageMutator) ReplaceColumn(gridRef, columnRef string, _ []*backend.DataGridColumnSpec) error { return fmt.Errorf("replacing column %s.%s is not yet supported by the MCP backend", gridRef, columnRef) } diff --git a/mdl/backend/mcp/unsupported_gen.go b/mdl/backend/mcp/unsupported_gen.go index 0e8e53241..761a9c484 100644 --- a/mdl/backend/mcp/unsupported_gen.go +++ b/mdl/backend/mcp/unsupported_gen.go @@ -474,6 +474,11 @@ func (unsupportedBackend) FindCustomWidgetType(_ string) (r0 *types.RawCustomWid return } +func (unsupportedBackend) FindDocumentUnit(_ string, _ string) (r0 *types.DocumentUnit, err1 error) { + err1 = errUnsupported("FindDocumentUnit") + return +} + func (unsupportedBackend) FindViewEntitySourceDocumentID(_ string, _ string) (r0 model.ID, err1 error) { err1 = errUnsupported("FindViewEntitySourceDocumentID") return @@ -692,6 +697,11 @@ func (unsupportedBackend) ListDatabaseConnections() (r0 []*model.DatabaseConnect return } +func (unsupportedBackend) ListDocumentUnits() (r0 []*types.DocumentUnit, err1 error) { + err1 = errUnsupported("ListDocumentUnits") + return +} + func (unsupportedBackend) ListDomainModels() (r0 []*domainmodel.DomainModel, err1 error) { err1 = errUnsupported("ListDomainModels") return @@ -862,6 +872,11 @@ func (unsupportedBackend) MoveDatabaseConnection(_ *model.DatabaseConnection) (e return } +func (unsupportedBackend) MoveDocument(_ model.ID, _ model.ID) (err0 error) { + err0 = errUnsupported("MoveDocument") + return +} + func (unsupportedBackend) MoveEntity(_ *domainmodel.Entity, _ model.ID, _ model.ID, _ string, _ string) (r0 []string, err1 error) { err1 = errUnsupported("MoveEntity") return diff --git a/mdl/backend/mock/backend.go b/mdl/backend/mock/backend.go index 56a222519..1b700ae35 100644 --- a/mdl/backend/mock/backend.go +++ b/mdl/backend/mock/backend.go @@ -54,6 +54,11 @@ type MockBackend struct { DeleteFolderFunc func(id model.ID) error MoveFolderFunc func(id model.ID, newContainerID model.ID) error + // DocumentPlacementBackend + MoveDocumentFunc func(unitID, containerID model.ID) error + FindDocumentUnitFunc func(moduleName, name string) (*types.DocumentUnit, error) + ListDocumentUnitsFunc func() ([]*types.DocumentUnit, error) + // DomainModelBackend ListDomainModelsFunc func() ([]*domainmodel.DomainModel, error) GetDomainModelFunc func(moduleID model.ID) (*domainmodel.DomainModel, error) diff --git a/mdl/backend/mock/mock_page_mutator.go b/mdl/backend/mock/mock_page_mutator.go index fefd01693..610e7211f 100644 --- a/mdl/backend/mock/mock_page_mutator.go +++ b/mdl/backend/mock/mock_page_mutator.go @@ -29,6 +29,8 @@ type MockPageMutator struct { DropWidgetFunc func(refs []backend.WidgetRef) error ReplaceWidgetFunc func(widgetRef string, columnRef string, widgets []pages.Widget) error InsertColumnsFunc func(gridRef, afterColumnRef string, position backend.InsertPosition, columns []*backend.DataGridColumnSpec) error + InsertListViewTemplatesFunc func(listViewRef string, templates []*pages.ListViewTemplate) error + DropListViewTemplateFunc func(listViewRef, specialization string) error ReplaceColumnFunc func(gridRef, columnRef string, columns []*backend.DataGridColumnSpec) error FindWidgetFunc func(name string) bool AddVariableFunc func(name, dataType, defaultValue string) error @@ -127,6 +129,20 @@ func (m *MockPageMutator) InsertColumns(gridRef, afterColumnRef string, position return fmt.Errorf("MockBackend.InsertColumns not configured") } +func (m *MockPageMutator) InsertListViewTemplates(listViewRef string, templates []*pages.ListViewTemplate) error { + if m.InsertListViewTemplatesFunc != nil { + return m.InsertListViewTemplatesFunc(listViewRef, templates) + } + return fmt.Errorf("MockBackend.InsertListViewTemplates not configured") +} + +func (m *MockPageMutator) DropListViewTemplate(listViewRef, specialization string) error { + if m.DropListViewTemplateFunc != nil { + return m.DropListViewTemplateFunc(listViewRef, specialization) + } + return fmt.Errorf("MockBackend.DropListViewTemplate not configured") +} + func (m *MockPageMutator) ReplaceColumn(gridRef, columnRef string, columns []*backend.DataGridColumnSpec) error { if m.ReplaceColumnFunc != nil { return m.ReplaceColumnFunc(gridRef, columnRef, columns) diff --git a/mdl/backend/mock/mock_placement.go b/mdl/backend/mock/mock_placement.go new file mode 100644 index 000000000..cd1d520f0 --- /dev/null +++ b/mdl/backend/mock/mock_placement.go @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mock + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" +) + +// MoveDocument records nothing by default and refuses, so a test exercising a +// folder change has to say what it expects to happen rather than passing +// against a stub that silently succeeds. +func (m *MockBackend) MoveDocument(unitID, containerID model.ID) error { + if m.MoveDocumentFunc != nil { + return m.MoveDocumentFunc(unitID, containerID) + } + return fmt.Errorf("MockBackend.MoveDocument not configured") +} + +// FindDocumentUnit returns "no such document" by default. Nil-with-nil-error is +// the honest default here — it is what the real backends return for a name that +// is not in the module — and it keeps an unconfigured mock from inventing a +// document for a handler to move. +func (m *MockBackend) FindDocumentUnit(moduleName, name string) (*types.DocumentUnit, error) { + if m.FindDocumentUnitFunc != nil { + return m.FindDocumentUnitFunc(moduleName, name) + } + return nil, nil +} + +// ListDocumentUnits returns nothing by default. An empty project is the honest +// default for a mock: it makes a folder listing show only what the test set up +// through the typed list functions, rather than inventing documents. +func (m *MockBackend) ListDocumentUnits() ([]*types.DocumentUnit, error) { + if m.ListDocumentUnitsFunc != nil { + return m.ListDocumentUnitsFunc() + } + return nil, nil +} diff --git a/mdl/backend/modelsdk/design_property_test.go b/mdl/backend/modelsdk/design_property_test.go index 942d0d200..7528fd6d5 100644 --- a/mdl/backend/modelsdk/design_property_test.go +++ b/mdl/backend/modelsdk/design_property_test.go @@ -6,6 +6,8 @@ import ( "bytes" "testing" + "go.mongodb.org/mongo-driver/bson" + "github.com/mendixlabs/mxcli/modelsdk/codec" "github.com/mendixlabs/mxcli/sdk/pages" ) @@ -55,3 +57,29 @@ func TestAppearanceDynamicClasses(t *testing.T) { t.Errorf("encoded appearance missing DynamicClasses expression %q\nBSON: %x", expr, out) } } + +// Studio Pro writes a DesignProperties list on every Forms$Appearance, empty ones +// included (measured: 16 of 16 pages in a blank 11.13 app, and the list `mx +// update-widgets` writes back into an mxcli-authored page). The codec omits an +// empty, never-appended PartList, so mxcli's widgets carried no DesignProperties +// key at all — which is also why ALTER STYLING's design-property write was a +// silent no-op: bsonnav.DSet cannot add a key that is not there. (upstream #931) +func TestAppearanceAlwaysEmitsDesignProperties(t *testing.T) { + out, err := (&codec.Encoder{}).Encode(newAppearance("card", "", "", nil)) + if err != nil { + t.Fatalf("encode appearance: %v", err) + } + + var doc bson.D + if err := bson.Unmarshal(out, &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + val, ok := lookupKey(doc, "DesignProperties") + if !ok { + t.Fatalf("no DesignProperties key on an appearance with none set: %v", doc) + } + arr, ok := val.(bson.A) + if !ok || len(arr) != 1 || arr[0] != int32(3) { + t.Errorf("DesignProperties = %v, want the empty typed-array marker [3]", val) + } +} diff --git a/mdl/backend/modelsdk/placement.go b/mdl/backend/modelsdk/placement.go new file mode 100644 index 000000000..4a09febdc --- /dev/null +++ b/mdl/backend/modelsdk/placement.go @@ -0,0 +1,120 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + "go.mongodb.org/mongo-driver/bson" +) + +// MoveDocument reparents a top-level document unit. See +// backend.DocumentPlacementBackend for why this is type-agnostic; the writer +// handles the idempotence the interface requires. +func (b *Backend) MoveDocument(unitID, containerID model.ID) error { + if b.writer == nil { + return fmt.Errorf("MoveDocument: not connected for writing") + } + if unitID == "" || containerID == "" { + return fmt.Errorf("MoveDocument: unit and container are both required") + } + return b.writer.MoveUnit(string(unitID), string(containerID)) +} + +// FindDocumentUnit locates a document by module and name through the unit +// table, whatever its type. +// +// Only units contained as "Documents" are considered. A module also holds its +// domain model, its security and its settings, and folders sit in the same +// table — restricting to the document containment is what keeps a folder named +// like a document from being returned as one. +func (b *Backend) FindDocumentUnit(moduleName, name string) (*types.DocumentUnit, error) { + if b.reader == nil { + return nil, fmt.Errorf("FindDocumentUnit: not connected") + } + modules, err := b.reader.ListModules() + if err != nil { + return nil, fmt.Errorf("FindDocumentUnit: list modules: %w", err) + } + var moduleID string + for _, m := range modules { + if m.Name == moduleName { + moduleID = m.ID + break + } + } + if moduleID == "" { + return nil, nil + } + containers := b.containerSetForModule(moduleID) + + var found *types.DocumentUnit + err = b.eachDocumentUnit(func(doc *types.DocumentUnit) bool { + if doc.Name != name || !containers[string(doc.ContainerID)] { + return true + } + found = doc + return false + }) + if err != nil { + return nil, fmt.Errorf("FindDocumentUnit: %w", err) + } + return found, nil +} + +// ListDocumentUnits returns every top-level document in the project. +func (b *Backend) ListDocumentUnits() ([]*types.DocumentUnit, error) { + if b.reader == nil { + return nil, fmt.Errorf("ListDocumentUnits: not connected") + } + var out []*types.DocumentUnit + if err := b.eachDocumentUnit(func(doc *types.DocumentUnit) bool { + out = append(out, doc) + return true + }); err != nil { + return nil, fmt.Errorf("ListDocumentUnits: %w", err) + } + return out, nil +} + +// eachDocumentUnit walks every "Documents" unit, decoding just enough of each +// to name it, and stops early when visit returns false. +// +// A unit whose contents will not decode is skipped rather than failing the +// walk: a listing missing one unreadable document is more useful than no +// listing, and the alternative would let one damaged unit hide a whole project. +func (b *Backend) eachDocumentUnit(visit func(*types.DocumentUnit) bool) error { + units, err := b.reader.ListUnits() + if err != nil { + return fmt.Errorf("list units: %w", err) + } + for _, u := range units { + if u.ContainmentName != "Documents" { + continue + } + raw, err := b.reader.GetRawUnitBytes(u.ID) + if err != nil || len(raw) == 0 { + continue + } + var doc bson.D + if err := bson.Unmarshal(raw, &doc); err != nil { + continue + } + name := docNameOf(doc) + if name == "" { + continue + } + if !visit(&types.DocumentUnit{ + ID: model.ID(u.ID), + ContainerID: model.ID(u.ContainerID), + Name: name, + Type: u.Type, + Kind: types.DocumentKind(u.Type), + }) { + return nil + } + } + return nil +} diff --git a/mdl/backend/modelsdk/placement_test.go b/mdl/backend/modelsdk/placement_test.go new file mode 100644 index 000000000..066499f2f --- /dev/null +++ b/mdl/backend/modelsdk/placement_test.go @@ -0,0 +1,145 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/model" +) + +// firstDocument returns some document unit in the module, to move around. +func firstDocument(t *testing.T, b *Backend, moduleName string) (name string, container model.ID) { + t.Helper() + mfs, err := b.ListMicroflows() + if err != nil { + t.Fatalf("ListMicroflows: %v", err) + } + for _, mf := range mfs { + if b.moduleNameFor(mf.ID) == moduleName { + return mf.Name, mf.ContainerID + } + } + t.Skipf("fixture has no microflow in %s to move", moduleName) + return "", "" +} + +// TestFindDocumentUnitResolvesByNameWithoutAType is the half of #932 that makes +// a placement fix general: a document is located through the unit table, so a +// doctype nobody wrote a finder for is still reachable. +func TestFindDocumentUnitResolvesByNameWithoutAType(t *testing.T) { + b := New() + if err := b.Connect(copyFixture(t)); err != nil { + t.Fatalf("connect: %v", err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + name, _ := firstDocument(t, b, "MyFirstModule") + got, err := b.FindDocumentUnit("MyFirstModule", name) + if err != nil { + t.Fatalf("FindDocumentUnit: %v", err) + } + if got == nil { + t.Fatalf("FindDocumentUnit(MyFirstModule, %s) found nothing", name) + } + if got.Name != name || got.ID == "" || got.Type == "" { + t.Errorf("got %+v, want the named document with an id and a type", got) + } + if got.Kind == "" { + t.Errorf("got no human-readable kind for %q", got.Type) + } + + // A name that is not there must be reported as absent, not invented: the + // MOVE handler distinguishes the two to produce a "not found" instead of + // reparenting whatever came first. + missing, err := b.FindDocumentUnit("MyFirstModule", "NoSuchDocument_932") + if err != nil { + t.Fatalf("FindDocumentUnit(missing): %v", err) + } + if missing != nil { + t.Errorf("FindDocumentUnit for an absent name returned %+v", missing) + } +} + +// TestMoveDocumentPersistsAndIsIdempotent pins both halves of the reparent +// primitive against real storage: the row really changes, and moving a document +// to where it already is neither writes nor counts as a write. +// +// The write counters matter as much as the row does. A move changes no byte of +// the document, so it is invisible to content-based no-op elision — without the +// count, ReportMutation calls a real move "Unchanged", which is the same class +// of lie as the silent no-op this fixes. +func TestMoveDocumentPersistsAndIsIdempotent(t *testing.T) { + proj := copyFixture(t) + b := New() + if err := b.Connect(proj); err != nil { + t.Fatalf("connect: %v", err) + } + var _ backend.DocumentPlacementBackend = b + + mod, err := b.GetModuleByName("MyFirstModule") + if err != nil || mod == nil { + t.Fatalf("GetModuleByName: %v, %v", mod, err) + } + name, origin := firstDocument(t, b, "MyFirstModule") + doc, err := b.FindDocumentUnit("MyFirstModule", name) + if err != nil || doc == nil { + t.Fatalf("FindDocumentUnit(%s): %v, %v", name, doc, err) + } + + folder := &model.Folder{Name: "Placement932", ContainerID: mod.ID} + if err := b.CreateFolder(folder); err != nil { + t.Fatalf("CreateFolder: %v", err) + } + if folder.ID == origin { + t.Fatal("fixture folder collided with the document's current container") + } + + offeredBefore, writtenBefore := b.writer.WriteStats() + if err := b.MoveDocument(doc.ID, folder.ID); err != nil { + t.Fatalf("MoveDocument: %v", err) + } + offeredAfter, writtenAfter := b.writer.WriteStats() + if writtenAfter != writtenBefore+1 || offeredAfter != offeredBefore+1 { + t.Errorf("a real move counted %d offered / %d written, want one of each", + offeredAfter-offeredBefore, writtenAfter-writtenBefore) + } + + // Same move again: offered, but elided. + if err := b.MoveDocument(doc.ID, folder.ID); err != nil { + t.Fatalf("MoveDocument (repeat): %v", err) + } + offeredRepeat, writtenRepeat := b.writer.WriteStats() + if writtenRepeat != writtenAfter { + t.Errorf("moving a document to its current container wrote (written %d → %d)", + writtenAfter, writtenRepeat) + } + if offeredRepeat != offeredAfter+1 { + t.Errorf("the repeat move was not offered to storage, so it cannot have been judged elided") + } + + if err := b.Disconnect(); err != nil { + t.Fatalf("disconnect: %v", err) + } + + // Reopen: the placement must have reached disk, not just the cache. + b2 := New() + if err := b2.Connect(proj); err != nil { + t.Fatalf("reconnect: %v", err) + } + t.Cleanup(func() { _ = b2.Disconnect() }) + + again, err := b2.FindDocumentUnit("MyFirstModule", name) + if err != nil || again == nil { + t.Fatalf("FindDocumentUnit after reopen: %v, %v", again, err) + } + if again.ContainerID != folder.ID { + t.Errorf("after reopen the document sits in %q, want the folder %q", again.ContainerID, folder.ID) + } + // Still in the module: a move that lost the module qualification would be + // the orphaning failure #892 documents, not a move. + if b2.moduleNameFor(again.ID) != "MyFirstModule" { + t.Errorf("the moved document is no longer resolvable in its module") + } +} diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index e526caa45..45756a8c3 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -145,10 +145,25 @@ func init() { }) codec.RegisterListMarker("Forms$TextBox", 2) // Pluggable widget container: null visibility/editability slots; marker 2. + // LabelTemplate is null too — Studio Pro writes the key on every CustomWidget, + // mxcli omitted it. Measured by handing an mxcli-authored Accordion page to + // `mx update-widgets` and diffing the unit it wrote back (upstream #931). codec.RegisterTypeDefaults("CustomWidgets$CustomWidget", codec.TypeDefaults{ - NullFields: []string{"ConditionalVisibilitySettings", "ConditionalEditabilitySettings"}, + NullFields: []string{ + "ConditionalVisibilitySettings", "ConditionalEditabilitySettings", + "LabelTemplate", + }, }) codec.RegisterListMarker("CustomWidgets$CustomWidget", 2) + // Every Forms$Appearance carries a DesignProperties list, emitted as the empty + // typed-array marker [3] when the widget has no design properties. The codec + // omits an empty, never-appended PartList, so an mxcli-authored widget had no + // DesignProperties key at all — which is also what made ALTER STYLING's design + // property write a silent no-op (bsonnav.DSet cannot add an absent key). + // Same measurement as above (upstream #931). + codec.RegisterTypeDefaults("Forms$Appearance", codec.TypeDefaults{ + MandatoryLists: []string{"DesignProperties"}, + }) // RadioButtonGroup (the MDL `radiobuttons` widget): same null-slot set as TextBox. codec.RegisterTypeDefaults("Forms$RadioButtonGroup", codec.TypeDefaults{ NullFields: []string{ @@ -563,6 +578,7 @@ func widgetToGen(w pages.Widget) (element.Element, error) { for _, t := range x.Templates { tg := genPg.NewListViewTemplate() assignID(tg) + tg.SetSpecializationQualifiedName(t.Specialization) for _, w := range t.Widgets { wg, err := widgetToGen(w) if err != nil { diff --git a/mdl/backend/mpr/backend.go b/mdl/backend/mpr/backend.go index 85ba986d9..354557fd7 100644 --- a/mdl/backend/mpr/backend.go +++ b/mdl/backend/mpr/backend.go @@ -154,6 +154,22 @@ func (b *MprBackend) MoveFolder(id model.ID, newContainerID model.ID) error { return b.writer.MoveFolder(id, newContainerID) } +// --------------------------------------------------------------------------- +// DocumentPlacementBackend +// --------------------------------------------------------------------------- + +func (b *MprBackend) MoveDocument(unitID, containerID model.ID) error { + return b.writer.MoveDocument(unitID, containerID) +} + +func (b *MprBackend) FindDocumentUnit(moduleName, name string) (*types.DocumentUnit, error) { + return b.writer.FindDocumentUnit(moduleName, name) +} + +func (b *MprBackend) ListDocumentUnits() ([]*types.DocumentUnit, error) { + return b.writer.ListDocumentUnits() +} + // --------------------------------------------------------------------------- // DomainModelBackend // --------------------------------------------------------------------------- diff --git a/mdl/backend/mutation.go b/mdl/backend/mutation.go index 932da1df6..82654a4f1 100644 --- a/mdl/backend/mutation.go +++ b/mdl/backend/mutation.go @@ -123,6 +123,21 @@ type PageMutator interface { // Columns are serialized as CustomWidgets$WidgetObject, not as form widgets. InsertColumns(gridRef string, afterColumnRef string, position InsertPosition, columns []*DataGridColumnSpec) error + // InsertListViewTemplates appends specialization templates to a List View. + // + // Templates live in the list view's Templates array, NOT in its Widgets array + // (which holds the default body), and a Forms$ListViewTemplate is not a + // widget — the same reason DataGrid2 columns get their own method above. + // Routing them through InsertWidget would append a non-widget to the widget + // list, which Studio Pro cannot open. + InsertListViewTemplates(listViewRef string, templates []*pages.ListViewTemplate) error + + // DropListViewTemplate removes the template rendering the given + // specialization from a List View. A template has no name, so it is addressed + // by entity. Returns an error naming the templates that ARE present when the + // entity does not match one — silently dropping nothing would read as success. + DropListViewTemplate(listViewRef string, specialization string) error + // ReplaceColumn replaces a single DataGrid2 column with new columns. // Columns are serialized as CustomWidgets$WidgetObject, not as form widgets. ReplaceColumn(gridRef string, columnRef string, columns []*DataGridColumnSpec) error diff --git a/mdl/backend/pagemutator/listview_templates.go b/mdl/backend/pagemutator/listview_templates.go new file mode 100644 index 000000000..3967de354 --- /dev/null +++ b/mdl/backend/pagemutator/listview_templates.go @@ -0,0 +1,183 @@ +// SPDX-License-Identifier: Apache-2.0 + +package pagemutator + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" + "github.com/mendixlabs/mxcli/mdl/bsonutil" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/sdk/pages" + "go.mongodb.org/mongo-driver/bson" +) + +// listViewTemplateType is the stored $Type of a List View specialization +// template. The entity it renders is stored under "Entity" — the SDK calls that +// property Specialization, but no document does; see pages.ListViewTemplate. +const listViewTemplateType = "Forms$ListViewTemplate" + +// findListView locates a List View by name and returns it with its parent slot, +// so a caller that has to append a missing field can store the grown document +// back (a bson.D is a slice: appending to it reallocates). +func (m *Mutator) findListView(listViewRef string) (*bsonWidgetResult, error) { + result := m.widgetFinder(m.rawData, listViewRef) + if result == nil { + return nil, fmt.Errorf("widget %q not found", listViewRef) + } + if t := widgetTypeName(result.widget); t != "Forms$ListView" && t != "Pages$ListView" { + return nil, fmt.Errorf("widget %q is a %s, not a list view — specialization templates "+ + "exist only on a list view (a gallery's `template ` is a named content slot, "+ + "a different construct)", listViewRef, t) + } + return result, nil +} + +// storedTemplateEntities returns the specializations a list view already renders, +// in stored order. +func storedTemplateEntities(listView bson.D) []string { + var out []string + for _, el := range bsonnav.DGetArrayElements(bsonnav.DGet(listView, "Templates")) { + if doc, ok := el.(bson.D); ok { + out = append(out, bsonnav.DGetString(doc, "Entity")) + } + } + return out +} + +// InsertListViewTemplates appends specialization templates to a list view. +// +// They go in the Templates array, never in Widgets: Widgets holds the default +// body, and a Forms$ListViewTemplate is not a widget. Appending one to the +// widget list produces a document Studio Pro cannot open, which is the same +// reason DataGrid2 columns have their own path rather than going through +// InsertWidget. +func (m *Mutator) InsertListViewTemplates(listViewRef string, templates []*pages.ListViewTemplate) error { + result, err := m.findListView(listViewRef) + if err != nil { + return err + } + + // One template per specialization, matching what CREATE PAGE enforces. Mendix + // renders the first match, so a second template for the same entity is dead + // weight the author cannot see. + existing := storedTemplateEntities(result.widget) + for _, tpl := range templates { + for _, have := range existing { + if strings.EqualFold(have, tpl.Specialization) { + return fmt.Errorf("list view %q already has a template for %s", listViewRef, tpl.Specialization) + } + } + existing = append(existing, tpl.Specialization) + } + + newDocs := make([]any, 0, len(templates)) + for _, tpl := range templates { + children, err := m.serializeWidgets(tpl.Widgets) + if err != nil { + return err + } + // The child list carries Mendix's list marker like any other widget list: + // 3 when empty, 2 when populated. + childArr := bson.A{int32(3)} + if len(children) > 0 { + childArr = bson.A{int32(2)} + childArr = append(childArr, children...) + } + id := string(tpl.ID) + if id == "" { + id = types.GenerateID() + } + // Key order matches Studio Pro's documents. + newDocs = append(newDocs, bson.D{ + {Key: "$ID", Value: bsonutil.IDToBsonBinary(id)}, + {Key: "$Type", Value: listViewTemplateType}, + {Key: "Entity", Value: tpl.Specialization}, + {Key: "Widgets", Value: childArr}, + }) + } + + stored := bsonnav.ToBsonA(bsonnav.DGet(result.widget, "Templates")) + var out bson.A + switch { + case len(stored) > 0 && isListMarker(stored[0]): + // Appending to an empty list flips the marker from 3 to 2, the same + // distinction Studio Pro makes. + out = append(out, int32(2)) + out = append(out, stored[1:]...) + out = append(out, newDocs...) + case len(stored) == 0: + out = append(out, int32(2)) + out = append(out, newDocs...) + default: + out = append(out, stored...) + out = append(out, newDocs...) + } + + if bsonnav.DSet(result.widget, "Templates", out) { + return nil + } + // The field was absent: appending grows the bson.D, so the widget has to be + // stored back into its parent slot or the change is written to a copy. + grown := append(result.widget, bson.E{Key: "Templates", Value: out}) + if result.parentArr == nil || result.index < 0 || result.index >= len(result.parentArr) { + return fmt.Errorf("cannot add templates to list view %q: its parent slot could not be located", listViewRef) + } + result.parentArr[result.index] = grown + bsonnav.DSetArray(result.parentDoc, result.parentKey, result.parentArr) + return nil +} + +// DropListViewTemplate removes the template rendering a given specialization. +// +// A template has no name, so it is addressed by entity. An entity that matches +// nothing is an error naming what IS there: dropping nothing and reporting +// success is how a typo in a specialization name becomes a silent no-op. +func (m *Mutator) DropListViewTemplate(listViewRef, specialization string) error { + result, err := m.findListView(listViewRef) + if err != nil { + return err + } + + stored := bsonnav.ToBsonA(bsonnav.DGet(result.widget, "Templates")) + var out bson.A + var marker []any + rest := []any(stored) + if len(stored) > 0 && isListMarker(stored[0]) { + marker = []any{stored[0]} + rest = stored[1:] + } + + dropped := false + for _, el := range rest { + doc, ok := el.(bson.D) + if ok && strings.EqualFold(bsonnav.DGetString(doc, "Entity"), specialization) { + dropped = true + continue + } + out = append(out, el) + } + if !dropped { + have := storedTemplateEntities(result.widget) + if len(have) == 0 { + return fmt.Errorf("list view %q has no specialization templates", listViewRef) + } + return fmt.Errorf("list view %q has no template for %s (it has: %s)", + listViewRef, specialization, strings.Join(have, ", ")) + } + + // An emptied list reverts to Mendix's empty-list marker. + final := bson.A{} + if len(marker) > 0 && len(out) == 0 { + final = bson.A{int32(3)} + } else { + final = append(final, marker...) + final = append(final, out...) + } + bsonnav.DSetArray(result.parentDoc, result.parentKey, result.parentArr) + if !bsonnav.DSet(result.widget, "Templates", final) { + return fmt.Errorf("list view %q has no Templates array to drop from", listViewRef) + } + return nil +} diff --git a/mdl/backend/pagemutator/listview_templates_test.go b/mdl/backend/pagemutator/listview_templates_test.go new file mode 100644 index 000000000..03cb0d758 --- /dev/null +++ b/mdl/backend/pagemutator/listview_templates_test.go @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: Apache-2.0 + +package pagemutator + +import ( + "strings" + "testing" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// stubWidgetDeps serializes a widget to a marker document, so these tests assert +// the Templates plumbing without pulling in an engine's serializer. +type stubWidgetDeps struct { + Deps +} + +func (d *stubWidgetDeps) SerializeWidget(w pages.Widget) bson.D { + return bson.D{{Key: "$Type", Value: "Forms$TextBox"}, {Key: "Name", Value: w.GetName()}} +} + +func listViewDoc(entities ...string) bson.D { + templates := bson.A{int32(3)} + if len(entities) > 0 { + templates = bson.A{int32(2)} + for _, e := range entities { + templates = append(templates, bson.D{ + {Key: "$Type", Value: "Forms$ListViewTemplate"}, + {Key: "Entity", Value: e}, + {Key: "Widgets", Value: bson.A{int32(3)}}, + }) + } + } + return bson.D{ + {Key: "$Type", Value: "Forms$ListView"}, + {Key: "Name", Value: "vehicleListView"}, + {Key: "Widgets", Value: bson.A{int32(3)}}, + {Key: "Templates", Value: templates}, + } +} + +func templateEntitiesOf(t *testing.T, m *Mutator) []string { + t.Helper() + lv := m.widgetFinder(m.rawData, "vehicleListView") + if lv == nil { + t.Fatal("list view not found") + } + return storedTemplateEntities(lv.widget) +} + +// TestInsertListViewTemplates_AppendsToTemplatesNotWidgets is the load-bearing +// assertion: a Forms$ListViewTemplate is NOT a widget, and the list view's +// Widgets array is its default body. Routing a template through the widget path +// would append a non-widget to the widget list, producing a page Studio Pro +// cannot open — the same reason DataGrid2 columns have their own path. +func TestInsertListViewTemplates_AppendsToTemplatesNotWidgets(t *testing.T) { + m := New(makeRawPage(listViewDoc("Pages.Bus")), model.ID("u"), &stubWidgetDeps{}) + + err := m.InsertListViewTemplates("vehicleListView", []*pages.ListViewTemplate{{ + Specialization: "Pages.Truck", + Widgets: []pages.Widget{&pages.TextBox{BaseWidget: pages.BaseWidget{Name: "truckLabel"}}}, + }}) + if err != nil { + t.Fatalf("InsertListViewTemplates: %v", err) + } + + if got := templateEntitiesOf(t, m); len(got) != 2 || got[0] != "Pages.Bus" || got[1] != "Pages.Truck" { + t.Errorf("templates = %v, want [Pages.Bus Pages.Truck] — appended, in order", got) + } + + lv := m.widgetFinder(m.rawData, "vehicleListView") + body := bsonnav.DGetArrayElements(bsonnav.DGet(lv.widget, "Widgets")) + if len(body) != 0 { + t.Errorf("the list view's default body gained %d widget(s); a template must never land in Widgets", len(body)) + } + + // The new template carries its children and Studio Pro's key order. + tplArr := bsonnav.DGetArrayElements(bsonnav.DGet(lv.widget, "Templates")) + last, ok := tplArr[len(tplArr)-1].(bson.D) + if !ok { + t.Fatalf("template is %T, want bson.D", tplArr[len(tplArr)-1]) + } + var keys []string + for _, e := range last { + keys = append(keys, e.Key) + } + if len(keys) != 4 || keys[0] != "$ID" || keys[1] != "$Type" || keys[2] != "Entity" || keys[3] != "Widgets" { + t.Errorf("template keys = %v, want [$ID $Type Entity Widgets]", keys) + } + if n := len(bsonnav.DGetArrayElements(bsonnav.DGet(last, "Widgets"))); n != 1 { + t.Errorf("template has %d child widget(s), want 1", n) + } +} + +// TestInsertListViewTemplates_FlipsTheEmptyMarker pins the list marker: Mendix +// writes 3 for an empty list and 2 for a populated one. Appending to an empty +// Templates array must flip it, or the document disagrees with itself. +func TestInsertListViewTemplates_FlipsTheEmptyMarker(t *testing.T) { + m := New(makeRawPage(listViewDoc()), model.ID("u"), &stubWidgetDeps{}) + + before := bsonnav.ToBsonA(bsonnav.DGet(m.widgetFinder(m.rawData, "vehicleListView").widget, "Templates")) + if len(before) != 1 || before[0] != int32(3) { + t.Fatalf("fixture is not an empty list: %v", before) + } + + if err := m.InsertListViewTemplates("vehicleListView", []*pages.ListViewTemplate{{Specialization: "Pages.Bus"}}); err != nil { + t.Fatalf("InsertListViewTemplates: %v", err) + } + + after := bsonnav.ToBsonA(bsonnav.DGet(m.widgetFinder(m.rawData, "vehicleListView").widget, "Templates")) + if len(after) != 2 || after[0] != int32(2) { + t.Errorf("Templates = %v, want marker 2 followed by one template", after) + } +} + +// TestDropListViewTemplate covers the removal and the two ways it can be asked +// for something that is not there. A drop that matches nothing must NOT report +// success: that is how a typo in a specialization name becomes a silent no-op. +func TestDropListViewTemplate(t *testing.T) { + t.Run("drops one and keeps the order of the rest", func(t *testing.T) { + m := New(makeRawPage(listViewDoc("Pages.Bus", "Pages.Truck", "Pages.Car")), model.ID("u"), &stubWidgetDeps{}) + if err := m.DropListViewTemplate("vehicleListView", "Pages.Truck"); err != nil { + t.Fatalf("DropListViewTemplate: %v", err) + } + got := templateEntitiesOf(t, m) + if len(got) != 2 || got[0] != "Pages.Bus" || got[1] != "Pages.Car" { + t.Errorf("templates = %v, want [Pages.Bus Pages.Car]", got) + } + }) + + t.Run("emptying the list restores the empty marker", func(t *testing.T) { + m := New(makeRawPage(listViewDoc("Pages.Bus")), model.ID("u"), &stubWidgetDeps{}) + if err := m.DropListViewTemplate("vehicleListView", "Pages.Bus"); err != nil { + t.Fatalf("DropListViewTemplate: %v", err) + } + arr := bsonnav.ToBsonA(bsonnav.DGet(m.widgetFinder(m.rawData, "vehicleListView").widget, "Templates")) + if len(arr) != 1 || arr[0] != int32(3) { + t.Errorf("Templates = %v, want the empty-list marker 3", arr) + } + }) + + t.Run("no such template names the ones that are there", func(t *testing.T) { + m := New(makeRawPage(listViewDoc("Pages.Bus", "Pages.Car")), model.ID("u"), &stubWidgetDeps{}) + err := m.DropListViewTemplate("vehicleListView", "Pages.SUV") + if err == nil { + t.Fatal("dropping a template that does not exist reported success") + } + for _, want := range []string{"Pages.SUV", "Pages.Bus", "Pages.Car"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not mention %q", err.Error(), want) + } + } + }) + + t.Run("a list view with no templates says so", func(t *testing.T) { + m := New(makeRawPage(listViewDoc()), model.ID("u"), &stubWidgetDeps{}) + err := m.DropListViewTemplate("vehicleListView", "Pages.SUV") + if err == nil || !strings.Contains(err.Error(), "no specialization templates") { + t.Errorf("err = %v, want it to say the list view has no templates", err) + } + }) +} + +// TestListViewTemplateOpsRefuseNonListViews pins that both operations check the +// target's type. A gallery's `template ` is a named content slot, not a +// per-specialization body, and must not be reachable through these. +func TestListViewTemplateOpsRefuseNonListViews(t *testing.T) { + grid := bson.D{ + {Key: "$Type", Value: "Forms$DataGrid"}, + {Key: "Name", Value: "vehicleListView"}, + } + m := New(makeRawPage(grid), model.ID("u"), &stubWidgetDeps{}) + + if err := m.InsertListViewTemplates("vehicleListView", []*pages.ListViewTemplate{{Specialization: "Pages.Bus"}}); err == nil || + !strings.Contains(err.Error(), "not a list view") { + t.Errorf("insert err = %v, want a refusal naming the widget type", err) + } + if err := m.DropListViewTemplate("vehicleListView", "Pages.Bus"); err == nil || + !strings.Contains(err.Error(), "not a list view") { + t.Errorf("drop err = %v, want a refusal naming the widget type", err) + } +} diff --git a/mdl/backend/pagemutator/mutator.go b/mdl/backend/pagemutator/mutator.go index a53417d4f..e0e4bc57b 100644 --- a/mdl/backend/pagemutator/mutator.go +++ b/mdl/backend/pagemutator/mutator.go @@ -2545,8 +2545,7 @@ func setDesignPropertyMut(widget bson.D, key, valueType, option string) error { continue } bsonnav.DSet(entry, "Value", buildDesignPropertyValueDoc(valueType, option)) - bsonnav.DSetArray(appearance, "DesignProperties", elements) - return nil + return writeDesignProperties(widget, key, elements) } entry := bson.D{ @@ -2555,7 +2554,25 @@ func setDesignPropertyMut(widget bson.D, key, valueType, option string) error { {Key: "Key", Value: key}, {Key: "Value", Value: buildDesignPropertyValueDoc(valueType, option)}, } - bsonnav.DSetArray(appearance, "DesignProperties", append(elements, entry)) + return writeDesignProperties(widget, key, append(elements, entry)) +} + +// designPropertiesMarker is the typed-array marker Studio Pro writes on a +// Forms$Appearance's DesignProperties list — measured on every page of a blank +// 11.13 app, empty lists included. +const designPropertiesMarker = 3 + +// writeDesignProperties stores the widget's design-property entries, creating the +// Appearance.DesignProperties array when the widget does not carry one yet. +// +// It goes through DSetArrayIn rather than DSetArray because a widget mxcli +// authored may have no DesignProperties key at all, and DSet cannot add one: the +// write was a silent no-op while ALTER STYLING still reported success +// (upstream #931). +func writeDesignProperties(widget bson.D, key string, elements []any) error { + if !bsonnav.DSetArrayIn(widget, "Appearance", "DesignProperties", elements, designPropertiesMarker) { + return fmt.Errorf("could not store design property %q on this widget", key) + } return nil } @@ -2573,8 +2590,7 @@ func removeDesignPropertyMut(widget bson.D, key string) error { } kept = append(kept, el) } - bsonnav.DSetArray(appearance, "DesignProperties", kept) - return nil + return writeDesignProperties(widget, key, kept) } // clearDesignPropertiesMut removes all design properties from the widget, @@ -2584,8 +2600,7 @@ func clearDesignPropertiesMut(widget bson.D) error { if appearance == nil { return nil } - bsonnav.DSetArray(appearance, "DesignProperties", nil) - return nil + return writeDesignProperties(widget, "", nil) } // buildDesignPropertyValueDoc builds the typed Value sub-document for a design diff --git a/mdl/backend/pagemutator/mutator_test.go b/mdl/backend/pagemutator/mutator_test.go index 1aa28670d..72560b4f2 100644 --- a/mdl/backend/pagemutator/mutator_test.go +++ b/mdl/backend/pagemutator/mutator_test.go @@ -520,18 +520,59 @@ func TestDSetArray_PreservesMarker(t *testing.T) { } } -func TestDSetArray_NoMarker(t *testing.T) { +// A Mendix array's first entry is its typed-array marker. With no marker to +// preserve, writing the elements bare produces a list whose first ENTRY the +// reader takes as the marker — measured on mxbuild 11.13, a DesignProperties +// array written that way makes the project fail to load ("Type +// OptionDesignPropertyValue does not contain a constructor with a parameter of +// type Appearance") and create-module-package die with "Unknown export error". +// So DSetArray refuses instead of writing, and says so (upstream #931). +func TestDSetArray_NoMarker_Refuses(t *testing.T) { parent := bson.D{ {Key: "Widgets", Value: bson.A{"a", "b"}}, } - bsonnav.DSetArray(parent, "Widgets", []any{"x"}) + if bsonnav.DSetArray(parent, "Widgets", []any{"x"}) { + t.Error("DSetArray reported success with no marker to write") + } result := bsonnav.ToBsonA(bsonnav.DGet(parent, "Widgets")) - if len(result) != 1 { - t.Fatalf("Expected 1 element, got %d", len(result)) + if len(result) != 2 || result[0] != "a" || result[1] != "b" { + t.Errorf("refused write must leave the value untouched, got %v", result) + } +} + +// DSetArrayIn is the caller-supplies-the-marker form, and the one that can CREATE +// the property: DSet cannot add a key to a bson.D held by value, so the grown +// child has to be written back into its parent. +func TestDSetArrayIn_CreatesWithMarker(t *testing.T) { + widget := bson.D{ + {Key: "Appearance", Value: bson.D{{Key: "Class", Value: ""}}}, + } + if !bsonnav.DSetArrayIn(widget, "Appearance", "DesignProperties", []any{"entry"}, 3) { + t.Fatal("DSetArrayIn reported failure on a widget that has an Appearance") + } + + appearance := bsonnav.DGetDoc(widget, "Appearance") + result := bsonnav.ToBsonA(bsonnav.DGet(appearance, "DesignProperties")) + if len(result) != 2 || result[0] != int32(3) || result[1] != "entry" { + t.Errorf("got %v, want [3 entry]", result) + } +} + +// An existing marker wins over the caller's default: the stored document is the +// authority on how its own list is versioned. +func TestDSetArrayIn_PreservesExistingMarker(t *testing.T) { + widget := bson.D{ + {Key: "Appearance", Value: bson.D{{Key: "DesignProperties", Value: bson.A{int32(2)}}}}, + } + if !bsonnav.DSetArrayIn(widget, "Appearance", "DesignProperties", []any{"entry"}, 3) { + t.Fatal("DSetArrayIn reported failure") } - if result[0] != "x" { - t.Errorf("Expected [x], got %v", result) + + appearance := bsonnav.DGetDoc(widget, "Appearance") + result := bsonnav.ToBsonA(bsonnav.DGet(appearance, "DesignProperties")) + if len(result) != 2 || result[0] != int32(2) { + t.Errorf("got %v, want the stored marker 2 preserved", result) } } diff --git a/mdl/backend/placement.go b/mdl/backend/placement.go new file mode 100644 index 000000000..1f80dc6ab --- /dev/null +++ b/mdl/backend/placement.go @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: Apache-2.0 + +package backend + +import ( + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" +) + +// DocumentPlacementBackend moves top-level documents between containers, and +// finds them without being told their type. +// +// Placement is the one property every document type shares and none of them +// stores: a document's folder is its unit's container, not a field in its +// contents. So the typed Move* methods on the per-domain interfaces +// (MoveMicroflow, MoveEnumeration, …) all reduce to the same row update, and a +// new document type inherits a placement bug simply by not having one written +// for it — which is how import mappings, JSON structures, queues, scheduled +// events and a dozen others ended up unable to leave the module root (#932). +// +// These two methods are type-agnostic on purpose. FindDocumentUnit resolves a +// qualified name through the unit table rather than through a per-kind list, so +// it cannot inherit the blind spot that a hand-maintained list of document +// kinds keeps re-introducing (cf. #892, where the same kind of list made a +// non-empty folder render as empty and a destructive drop look safe). The typed +// Move* methods stay for the doctypes whose move does extra work — remapping +// document access roles, rewriting cross-module references. +type DocumentPlacementBackend interface { + // MoveDocument reparents a top-level document unit to containerID, which is + // either a module ID (the module root) or a folder ID. + // + // Implementations must be idempotent: moving a document to the container it + // already occupies must not write. Placement changes nothing inside the + // unit, so it is invisible to content-based no-op elision (ADR-0008) and has + // to account for itself. + MoveDocument(unitID, containerID model.ID) error + + // FindDocumentUnit returns the top-level document named name inside + // moduleName, wherever it sits in that module's folder tree, or nil when + // there is no such document. + FindDocumentUnit(moduleName, name string) (*types.DocumentUnit, error) + + // ListDocumentUnits returns every top-level document in the project with + // its container, name and type. + // + // Type-agnostic for the same reason as FindDocumentUnit, and it matters + // most here: a folder listing built from per-kind list calls can only show + // the kinds someone remembered to add, and an under-count is what made + // dropping a non-empty folder look safe in #892. This one cannot miss a + // kind, because it never asks what kind anything is. + ListDocumentUnits() ([]*types.DocumentUnit, error) +} diff --git a/mdl/catalog/builder_pages.go b/mdl/catalog/builder_pages.go index 99f8f78a9..555feb57a 100644 --- a/mdl/catalog/builder_pages.go +++ b/mdl/catalog/builder_pages.go @@ -301,6 +301,14 @@ type rawWidgetInfo struct { // references (each child is visited separately by extractWidgetsRecursive). var widgetChildKeys = map[string]bool{ "Widgets": true, "Rows": true, "FooterWidgets": true, "TabPages": true, + // A List View's specialization templates. Missing here, the list view's own + // scan descended into them and collected each template's Entity — then + // returned the lexicographically smallest candidate, so a specialization + // sorting before the list view's real datasource silently replaced it. The + // widgets table recorded the wrong entity for the list view, decided by + // alphabetical accident. Each template is indexed as a container in its own + // right instead (see extractWidgetsRecursive). Issue #940. + "Templates": true, } // scanWidgetOwnRefs collects the entity/microflow/nanoflow a widget references in @@ -509,6 +517,22 @@ func extractWidgetsRecursive(w map[string]any) []rawWidgetInfo { } } + // Handle List View specialization templates. A template is not a widget, but + // it is a container that carries its own entity — the specialization it + // renders — so it is walked as one: the row it produces holds that entity, + // and its children are indexed like any other nested widgets. + // + // Without this, every widget inside a template was absent from the index, and + // because the refs projection is built from the widgets table, an entity used + // only inside a template reported 0 pages and 0 widgets — so anything using + // reference counts to decide "unused, safe to delete" would delete a document + // in active use. Issue #940. + for _, tpl := range getBsonArrayElements(w["Templates"]) { + if tplMap, ok := tpl.(map[string]any); ok { + result = append(result, extractWidgetsRecursive(tplMap)...) + } + } + // Handle DataView/ListView footer widgets footerWidgets := getBsonArrayElements(w["FooterWidgets"]) for _, fw := range footerWidgets { diff --git a/mdl/catalog/builder_pages_listview_templates_test.go b/mdl/catalog/builder_pages_listview_templates_test.go new file mode 100644 index 000000000..8c78e3bf5 --- /dev/null +++ b/mdl/catalog/builder_pages_listview_templates_test.go @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: Apache-2.0 + +package catalog + +import "testing" + +// listViewWithTemplates builds the shape from issue #940: a List View over a +// generalization, with one template per specialization. Each template holds a +// data container over a child entity and a button calling a microflow — the +// content that reported "0 pages, 0 widgets". +// +// The List View's own Widgets array is empty on purpose. Studio Pro also +// populates it, and that content WAS indexed; leaving it empty isolates the +// templates so the test cannot pass on the strength of the sibling array. +func listViewWithTemplates(firstSpecialization string) map[string]any { + tmpl := func(specialization, viewName, buttonName, microflow string) map[string]any { + return map[string]any{ + "$Type": "Forms$ListViewTemplate", + "Entity": specialization, + "Widgets": []any{ + map[string]any{ + "$Type": "Forms$DataView", + "Name": viewName, + "DataSource": map[string]any{ + "$Type": "Forms$DataViewSource", + "EntityRef": map[string]any{ + "$Type": "DomainModels$DirectEntityRef", + "Entity": "MyModule.ChildRecord", + }, + }, + "Widgets": []any{ + map[string]any{ + "$Type": "Forms$ActionButton", + "Name": buttonName, + "Action": map[string]any{ + "$Type": "Forms$MicroflowClientAction", + "Microflow": microflow, + }, + }, + }, + }, + }, + } + } + return map[string]any{ + "$Type": "Forms$ListView", + "Name": "listView1", + "DataSource": map[string]any{ + "$Type": "Forms$ListViewXPathSource", + "EntityRef": map[string]any{ + "$Type": "DomainModels$DirectEntityRef", + "Entity": "MyModule.BaseItem", + }, + }, + "Widgets": []any{}, + "Templates": []any{ + tmpl(firstSpecialization, "typeAView", "btnA", "MyModule.ACT_A_DoThing"), + tmpl("MyModule.TypeB", "typeBView", "btnB", "MyModule.ACT_TypeB_DoThing"), + }, + } +} + +func widgetByName(rows []rawWidgetInfo, name string) *rawWidgetInfo { + for i := range rows { + if rows[i].Name == name { + return &rows[i] + } + } + return nil +} + +// TestListViewTemplateContentsAreIndexed is the regression test for #940. +// +// extractWidgetsRecursive knew Widgets, Rows/Columns, FooterWidgets, TabPages, +// pluggable Object.Properties and NavigationList Items — but not Templates. So +// every widget inside a List View specialization template was absent from the +// widgets table, and since the refs projection is built from that table, +// `SHOW REFERENCES TO ` reported 0 pages and +// 0 widgets. Anything using reference counts to decide "unused, safe to delete" +// would delete a document in active use. +func TestListViewTemplateContentsAreIndexed(t *testing.T) { + rows := extractWidgetsRecursive(listViewWithTemplates("MyModule.TypeA")) + + for _, name := range []string{"typeAView", "typeBView", "btnA", "btnB"} { + if widgetByName(rows, name) == nil { + t.Errorf("widget %q inside a list view template is missing from the index (#940)", name) + } + } + + if w := widgetByName(rows, "typeBView"); w != nil && w.EntityRef != "MyModule.ChildRecord" { + t.Errorf("template data view EntityRef = %q, want MyModule.ChildRecord "+ + "(this is the ref that made SHOW REFERENCES report 0 pages)", w.EntityRef) + } + if w := widgetByName(rows, "btnB"); w != nil && w.MicroflowRef != "MyModule.ACT_TypeB_DoThing" { + t.Errorf("template button MicroflowRef = %q, want MyModule.ACT_TypeB_DoThing", w.MicroflowRef) + } + + // The specialization each template renders is itself a reference the page + // makes. It is carried by the template's own row rather than folded into the + // list view, which is what the next test pins. + var specializations []string + for _, r := range rows { + if r.WidgetType == "Forms$ListViewTemplate" { + specializations = append(specializations, r.EntityRef) + } + } + if len(specializations) != 2 { + t.Errorf("got %d list view template row(s) %v, want 2", len(specializations), specializations) + } +} + +// TestListViewDataSourceSurvivesATemplateEntity pins the defect the report did +// not find. Templates was missing from widgetChildKeys as well, so the list +// view's own ref scan descended into its templates, collected each one's +// Entity, and returned the lexicographically smallest of the candidates. A +// specialization that sorts before the list view's real datasource silently +// replaced it — the widgets table then recorded the wrong entity for the list +// view, decided by alphabetical accident. +func TestListViewDataSourceSurvivesATemplateEntity(t *testing.T) { + // "MyModule.AAA_First" sorts before "MyModule.BaseItem". + rows := extractWidgetsRecursive(listViewWithTemplates("MyModule.AAA_First")) + + lv := widgetByName(rows, "listView1") + if lv == nil { + t.Fatal("the list view itself is missing from the index") + } + if lv.EntityRef != "MyModule.BaseItem" { + t.Errorf("list view EntityRef = %q, want MyModule.BaseItem — a template's "+ + "specialization displaced the widget's own datasource (#940)", lv.EntityRef) + } +} diff --git a/mdl/executor/cmd_agenteditor_agents.go b/mdl/executor/cmd_agenteditor_agents.go index 58dd5dfcf..8761cddd4 100644 --- a/mdl/executor/cmd_agenteditor_agents.go +++ b/mdl/executor/cmd_agenteditor_agents.go @@ -85,7 +85,7 @@ func describeAgentEditorAgent(ctx *ExecContext, name ast.QualifiedName) error { fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", a.Documentation) } - fmt.Fprintf(ctx.Output, "create agent %s (\n", qualifiedName) + fmt.Fprintf(ctx.Output, "create agent %s%s (\n", qualifiedName, describeFolderClause(ctx, a.ContainerID)) // Build property lines. User-set properties are emitted in a stable // order; empty values are omitted. diff --git a/mdl/executor/cmd_agenteditor_kbs.go b/mdl/executor/cmd_agenteditor_kbs.go index 588344248..f2c9bbf2f 100644 --- a/mdl/executor/cmd_agenteditor_kbs.go +++ b/mdl/executor/cmd_agenteditor_kbs.go @@ -83,7 +83,7 @@ func describeAgentEditorKnowledgeBase(ctx *ExecContext, name ast.QualifiedName) fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", k.Documentation) } - fmt.Fprintf(ctx.Output, "create knowledge base %s (\n", qualifiedName) + fmt.Fprintf(ctx.Output, "create knowledge base %s%s (\n", qualifiedName, describeFolderClause(ctx, k.ContainerID)) var lines []string if k.Provider != "" { diff --git a/mdl/executor/cmd_agenteditor_mcpservices.go b/mdl/executor/cmd_agenteditor_mcpservices.go index 888411ab6..eb9b159c1 100644 --- a/mdl/executor/cmd_agenteditor_mcpservices.go +++ b/mdl/executor/cmd_agenteditor_mcpservices.go @@ -79,7 +79,7 @@ func describeAgentEditorConsumedMCPService(ctx *ExecContext, name ast.QualifiedN fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", c.Documentation) } - fmt.Fprintf(ctx.Output, "create consumed mcp service %s (\n", qualifiedName) + fmt.Fprintf(ctx.Output, "create consumed mcp service %s%s (\n", qualifiedName, describeFolderClause(ctx, c.ContainerID)) var lines []string if c.ProtocolVersion != "" { diff --git a/mdl/executor/cmd_agenteditor_models.go b/mdl/executor/cmd_agenteditor_models.go index 6e3deab14..a09f0faf1 100644 --- a/mdl/executor/cmd_agenteditor_models.go +++ b/mdl/executor/cmd_agenteditor_models.go @@ -13,6 +13,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/agenteditor" ) @@ -86,7 +87,7 @@ func describeAgentEditorModel(ctx *ExecContext, name ast.QualifiedName) error { fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", m.Documentation) } - fmt.Fprintf(ctx.Output, "create model %s (\n", qualifiedName) + fmt.Fprintf(ctx.Output, "create model %s%s (\n", qualifiedName, describeFolderClause(ctx, m.ContainerID)) // Emit properties in stable order. User-set properties (Provider, Key) // come first; Portal-populated metadata comes last and only if non-empty. @@ -169,8 +170,17 @@ func execCreateAgentEditorModel(ctx *ExecContext, s *ast.CreateModelStmt) error provider = "MxCloudGenAI" } + var existingContainerM model.ID + if existing != nil { + existingContainerM = existing.ContainerID + } + containerIDM, err := containerForDocument(ctx, module.ID, s.Folder, existingContainerM) + if err != nil { + return err + } + m := &agenteditor.Model{ - ContainerID: module.ID, + ContainerID: containerIDM, Name: s.Name.Name, Documentation: s.Documentation, Provider: provider, @@ -191,6 +201,9 @@ func execCreateAgentEditorModel(ctx *ExecContext, s *ast.CreateModelStmt) error return mdlerrors.NewBackend("update model", err) } invalidateHierarchy(ctx) + if _, err := applyDocumentFolder(ctx, m.ID, existingContainerM, containerIDM); err != nil { + return err + } ctx.ReportMutation("Modified", "model: %s", s.Name) return nil } diff --git a/mdl/executor/cmd_agenteditor_write.go b/mdl/executor/cmd_agenteditor_write.go index bca2b476a..32bd18878 100644 --- a/mdl/executor/cmd_agenteditor_write.go +++ b/mdl/executor/cmd_agenteditor_write.go @@ -9,6 +9,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/agenteditor" ) @@ -41,8 +42,17 @@ func execCreateConsumedMCPService(ctx *ExecContext, s *ast.CreateConsumedMCPServ return err } + var existingContainerC model.ID + if existing != nil { + existingContainerC = existing.ContainerID + } + containerIDC, err := containerForDocument(ctx, module.ID, s.Folder, existingContainerC) + if err != nil { + return err + } + c := &agenteditor.ConsumedMCPService{ - ContainerID: module.ID, + ContainerID: containerIDC, Name: s.Name.Name, Documentation: s.OuterDocumentation, ProtocolVersion: s.ProtocolVersion, @@ -59,6 +69,9 @@ func execCreateConsumedMCPService(ctx *ExecContext, s *ast.CreateConsumedMCPServ return mdlerrors.NewBackend("update consumed mcp service", err) } invalidateHierarchy(ctx) + if _, err := applyDocumentFolder(ctx, c.ID, existingContainerC, containerIDC); err != nil { + return err + } ctx.ReportMutation("Modified", "consumed mcp service: %s", s.Name) return nil } @@ -128,8 +141,17 @@ func execCreateKnowledgeBase(ctx *ExecContext, s *ast.CreateKnowledgeBaseStmt) e provider = "MxCloudGenAI" } + var existingContainerK model.ID + if existing != nil { + existingContainerK = existing.ContainerID + } + containerIDK, err := containerForDocument(ctx, module.ID, s.Folder, existingContainerK) + if err != nil { + return err + } + k := &agenteditor.KnowledgeBase{ - ContainerID: module.ID, + ContainerID: containerIDK, Name: s.Name.Name, Documentation: s.Documentation, Provider: provider, @@ -150,6 +172,9 @@ func execCreateKnowledgeBase(ctx *ExecContext, s *ast.CreateKnowledgeBaseStmt) e return mdlerrors.NewBackend("update knowledge base", err) } invalidateHierarchy(ctx) + if _, err := applyDocumentFolder(ctx, k.ID, existingContainerK, containerIDK); err != nil { + return err + } ctx.ReportMutation("Modified", "knowledge base: %s", s.Name) return nil } @@ -206,8 +231,17 @@ func execCreateAgent(ctx *ExecContext, s *ast.CreateAgentStmt) error { return err } + var existingContainerA model.ID + if existingAgent != nil { + existingContainerA = existingAgent.ContainerID + } + containerIDA, err := containerForDocument(ctx, module.ID, s.Folder, existingContainerA) + if err != nil { + return err + } + a := &agenteditor.Agent{ - ContainerID: module.ID, + ContainerID: containerIDA, Name: s.Name.Name, Documentation: s.Documentation, Description: s.Description, @@ -301,6 +335,9 @@ func execCreateAgent(ctx *ExecContext, s *ast.CreateAgentStmt) error { return mdlerrors.NewBackend("update agent", err) } invalidateHierarchy(ctx) + if _, err := applyDocumentFolder(ctx, a.ID, existingContainerA, containerIDA); err != nil { + return err + } ctx.ReportMutation("Modified", "agent: %s", s.Name) return nil } diff --git a/mdl/executor/cmd_alter_page.go b/mdl/executor/cmd_alter_page.go index 6adc34e7e..bbfce980f 100644 --- a/mdl/executor/cmd_alter_page.go +++ b/mdl/executor/cmd_alter_page.go @@ -10,6 +10,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" "github.com/mendixlabs/mxcli/mdl/backend" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/pages" ) @@ -74,6 +75,10 @@ func execAlterPage(ctx *ExecContext, s *ast.AlterPageStmt) error { if err := applyDropWidgetMutator(mutator, o); err != nil { return mdlerrors.NewBackend("drop", err) } + case *ast.DropListViewTemplateOp: + if err := mutator.DropListViewTemplate(o.ListView, o.Specialization); err != nil { + return mdlerrors.NewBackend("drop TEMPLATE", err) + } case *ast.ReplaceWidgetOp: if err := applyReplaceWidgetMutator(ctx, mutator, o, modName, containerID); err != nil { return mdlerrors.NewBackend("replace", err) @@ -238,6 +243,30 @@ func applyInsertWidgetMutator(ctx *ExecContext, mutator backend.PageMutator, op return mutator.InsertColumns(op.Target.Widget, op.Target.Column, backend.InsertPosition(op.Position), specs) } + // Special path: inserting specialization templates into a List View. A + // Forms$ListViewTemplate lives in the list view's Templates array, not in its + // Widgets array, and is not a widget — the same reason DataGrid2 columns take + // their own path above. Routed through InsertWidget it would append a + // non-widget to the widget list, producing a page Studio Pro cannot open. + if allListViewTemplates(op.Widgets) { + if !strings.EqualFold(op.Position, "INTO") { + return mdlerrors.NewValidation( + "a list view template can only be added with INSERT INTO { template for … }: " + + "templates are not siblings of the widgets in the list view's body") + } + templates, err := buildListViewTemplatesFromAST(ctx, op.Widgets, moduleName, moduleID, mutator, op.Target.Widget) + if err != nil { + return err + } + return mutator.InsertListViewTemplates(op.Target.Widget, templates) + } + if hasListViewTemplate(op.Widgets) { + return mdlerrors.NewValidation( + "mixing `template for …` blocks with ordinary widgets in one INSERT is not supported: " + + "they go to different places (the list view's Templates and its default body). " + + "Use one INSERT for the templates and another for the widgets") + } + // Find entity context for the new widgets. For INSERT BEFORE/AFTER the target // is a sibling, so use its enclosing container's context; for INSERT INTO the // target IS the container, so the children take the target's own context (e.g. @@ -325,6 +354,79 @@ func applyReplaceWidgetMutator(ctx *ExecContext, mutator backend.PageMutator, op // allColumns returns true if all widgets in the slice have type "column". // Used to dispatch ALTER PAGE INSERT/REPLACE into a DataGrid2 column to the // column-specific mutator path. +// allListViewTemplates reports whether every inserted node is a `template for +// Module.Entity { … }` block. +func allListViewTemplates(widgets []*ast.WidgetV3) bool { + if len(widgets) == 0 { + return false + } + for _, w := range widgets { + if w.Specialization == "" { + return false + } + } + return true +} + +// hasListViewTemplate reports whether ANY inserted node is a template, so a +// mixed insert can be refused rather than silently sending the templates down +// the widget path. +func hasListViewTemplate(widgets []*ast.WidgetV3) bool { + for _, w := range widgets { + if w.Specialization != "" { + return true + } + } + return false +} + +// buildListViewTemplatesFromAST builds specialization templates for INSERT INTO. +// +// Each template's children are built in the SPECIALIZATION's entity context, not +// the list view's: an attribute only the specialization has must resolve inside +// its own template. +func buildListViewTemplatesFromAST(ctx *ExecContext, nodes []*ast.WidgetV3, moduleName string, moduleID model.ID, mutator backend.PageMutator, listViewRef string) ([]*pages.ListViewTemplate, error) { + // The specialization check needs the domain model, which the mutator does not + // have — it sees raw BSON. A minimal builder over the same cache answers it. + checker := &pageBuilder{ctx: ctx, backend: ctx.Backend, moduleID: moduleID, moduleName: moduleName, execCache: ctx.Cache} + listEntity := mutator.EnclosingEntityForChildren(listViewRef) + + seen := make(map[string]bool, len(nodes)) + out := make([]*pages.ListViewTemplate, 0, len(nodes)) + for _, node := range nodes { + spec := node.Specialization + if seen[spec] { + return nil, mdlerrors.NewValidation(fmt.Sprintf( + "this INSERT adds two templates for %s", spec)) + } + seen[spec] = true + + // Mendix matches a template against the object's type, so a template for + // an entity outside the list view's hierarchy can never render. Refuse it + // here rather than writing a template nothing will ever reach. + if listEntity != "" && !checker.entityIsOrDescendsFrom(spec, listEntity) { + return nil, mdlerrors.NewValidation(fmt.Sprintf( + "template for %s in list view %s: %s is not %s or a specialization of it, "+ + "so the template can never match an object the list view shows", + spec, listViewRef, spec, listEntity)) + } + + widgets, err := buildWidgetsFromAST(ctx, node.Children, moduleName, moduleID, spec, mutator) + if err != nil { + return nil, mdlerrors.NewBackend("build template widgets", err) + } + out = append(out, &pages.ListViewTemplate{ + BaseElement: model.BaseElement{ + ID: model.ID(types.GenerateID()), + TypeName: "Forms$ListViewTemplate", + }, + Specialization: spec, + Widgets: widgets, + }) + } + return out, nil +} + func allColumns(widgets []*ast.WidgetV3) bool { if len(widgets) == 0 { return false diff --git a/mdl/executor/cmd_alter_page_listview_template_test.go b/mdl/executor/cmd_alter_page_listview_template_test.go new file mode 100644 index 000000000..cc7373c14 --- /dev/null +++ b/mdl/executor/cmd_alter_page_listview_template_test.go @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// alterPageWith runs one ALTER PAGE operation against a mock mutator. +func alterPageWith(t *testing.T, mutator *mock.MockPageMutator, op ast.AlterPageOperation) error { + t.Helper() + mod := mkModule("MyModule") + pg := mkPage(mod.ID, "TestPage") + if mutator.SaveFunc == nil { + mutator.SaveFunc = func() error { return nil } + } + // The real mutator hands back live scopes; the mock returns nil maps unless + // configured, and the builder writes into them. + if mutator.WidgetScopeFunc == nil { + mutator.WidgetScopeFunc = func() map[string]model.ID { return map[string]model.ID{} } + } + if mutator.ParamScopeFunc == nil { + mutator.ParamScopeFunc = func() (map[string]model.ID, map[string]string) { + return map[string]model.ID{}, map[string]string{} + } + } + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListFoldersFunc: func() ([]*types.FolderInfo, error) { return nil, nil }, + ListPagesFunc: func() ([]*pages.Page, error) { return []*pages.Page{pg}, nil }, + OpenPageForMutationFunc: func(unitID model.ID) (backend.PageMutator, error) { + return mutator, nil + }, + } + h := mkHierarchy(mod) + withContainer(h, pg.ContainerID, mod.ID) + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + return execAlterPage(ctx, &ast.AlterPageStmt{ + PageName: ast.QualifiedName{Module: "MyModule", Name: "TestPage"}, + Operations: []ast.AlterPageOperation{op}, + }) +} + +func templateNode(specialization string) *ast.WidgetV3 { + return &ast.WidgetV3{ + Type: "template", + Specialization: specialization, + Properties: map[string]any{}, + Children: []*ast.WidgetV3{{ + Type: "dynamictext", Name: "lbl", Properties: map[string]any{"Content": "x"}, + }}, + } +} + +// TestAlterPageInsertTemplateRoutesToTemplates is the routing assertion. +// +// INSERT INTO normally appends to a container's Widgets. A list view's Widgets +// is its DEFAULT BODY, and a Forms$ListViewTemplate is not a widget — so a +// template routed through InsertWidget would append a non-widget to the widget +// list, producing a page Studio Pro cannot open. It must take the dedicated +// path, the same way DataGrid2 columns do. +func TestAlterPageInsertTemplateRoutesToTemplates(t *testing.T) { + var gotListView string + var gotTemplates []*pages.ListViewTemplate + insertWidgetCalled := false + + mutator := &mock.MockPageMutator{ + EnclosingEntityForChildrenFunc: func(widgetRef string) string { return "" }, + InsertListViewTemplatesFunc: func(listViewRef string, templates []*pages.ListViewTemplate) error { + gotListView = listViewRef + gotTemplates = templates + return nil + }, + InsertWidgetFunc: func(string, string, backend.InsertPosition, []pages.Widget) error { + insertWidgetCalled = true + return nil + }, + } + + err := alterPageWith(t, mutator, &ast.InsertWidgetOp{ + Position: "INTO", + Target: ast.WidgetRef{Widget: "vehicleListView"}, + Widgets: []*ast.WidgetV3{templateNode("Pages.Bus")}, + }) + if err != nil { + t.Fatalf("insert: %v", err) + } + if insertWidgetCalled { + t.Error("a template went through InsertWidget — it would land in the list view's default body") + } + if gotListView != "vehicleListView" { + t.Errorf("listViewRef = %q, want vehicleListView", gotListView) + } + if len(gotTemplates) != 1 || gotTemplates[0].Specialization != "Pages.Bus" { + t.Fatalf("templates = %+v, want one for Pages.Bus", gotTemplates) + } + if gotTemplates[0].TypeName != "Forms$ListViewTemplate" { + t.Errorf("TypeName = %q", gotTemplates[0].TypeName) + } + if len(gotTemplates[0].Widgets) != 1 { + t.Errorf("template has %d widget(s), want 1", len(gotTemplates[0].Widgets)) + } +} + +// TestAlterPageInsertTemplateRefusals covers the shapes that cannot mean what +// they look like. +func TestAlterPageInsertTemplateRefusals(t *testing.T) { + newMutator := func() *mock.MockPageMutator { + return &mock.MockPageMutator{ + EnclosingEntityForChildrenFunc: func(string) string { return "" }, + InsertListViewTemplatesFunc: func(string, []*pages.ListViewTemplate) error { + return nil + }, + } + } + + t.Run("BEFORE/AFTER is refused", func(t *testing.T) { + err := alterPageWith(t, newMutator(), &ast.InsertWidgetOp{ + Position: "AFTER", + Target: ast.WidgetRef{Widget: "someWidget"}, + Widgets: []*ast.WidgetV3{templateNode("Pages.Bus")}, + }) + assertError(t, err) + assertContainsStr(t, err.Error(), "INSERT INTO") + }) + + t.Run("mixing templates and widgets is refused", func(t *testing.T) { + err := alterPageWith(t, newMutator(), &ast.InsertWidgetOp{ + Position: "INTO", + Target: ast.WidgetRef{Widget: "vehicleListView"}, + Widgets: []*ast.WidgetV3{ + {Type: "dynamictext", Name: "plain", Properties: map[string]any{"Content": "x"}}, + templateNode("Pages.Bus"), + }, + }) + assertError(t, err) + assertContainsStr(t, err.Error(), "different places") + }) + + t.Run("two templates for one entity in one INSERT is refused", func(t *testing.T) { + err := alterPageWith(t, newMutator(), &ast.InsertWidgetOp{ + Position: "INTO", + Target: ast.WidgetRef{Widget: "vehicleListView"}, + Widgets: []*ast.WidgetV3{templateNode("Pages.Bus"), templateNode("Pages.Bus")}, + }) + assertError(t, err) + assertContainsStr(t, err.Error(), "two templates for Pages.Bus") + }) +} + +// TestAlterPageDropTemplateReachesTheMutator pins the dispatch for the operation +// that needed new syntax, since a template has no name to put in a widgetRef. +func TestAlterPageDropTemplateReachesTheMutator(t *testing.T) { + var gotListView, gotSpec string + mutator := &mock.MockPageMutator{ + DropListViewTemplateFunc: func(listViewRef, specialization string) error { + gotListView, gotSpec = listViewRef, specialization + return nil + }, + } + err := alterPageWith(t, mutator, &ast.DropListViewTemplateOp{ + Specialization: "Pages.SUV", + ListView: "vehicleListView", + }) + if err != nil { + t.Fatalf("drop: %v", err) + } + if gotListView != "vehicleListView" || gotSpec != "Pages.SUV" { + t.Errorf("DropListViewTemplate(%q, %q), want (vehicleListView, Pages.SUV)", gotListView, gotSpec) + } +} diff --git a/mdl/executor/cmd_businessevents.go b/mdl/executor/cmd_businessevents.go index f96cea7e0..8f6145741 100644 --- a/mdl/executor/cmd_businessevents.go +++ b/mdl/executor/cmd_businessevents.go @@ -281,6 +281,9 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS var existingID model.ID // Target the live service and carry its exclusion forward (#914). existingExcluded := false + // Placement is carried forward the same way when the statement is silent + // about folders (#932). + var existingContainerID model.ID if existing, ok := pickLive(existingServices, func(svc *model.BusinessEventService) bool { return strings.EqualFold(h.GetModuleName(h.FindModuleID(svc.ContainerID)), moduleName) && @@ -293,6 +296,7 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS } existingID = existing.ID existingExcluded = existing.Excluded + existingContainerID = existing.ContainerID } // Resolve folder if specified @@ -372,9 +376,15 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS // Write to project if existingID != "" { + if stmt.Folder == "" { + svc.ContainerID = existingContainerID + } if err := ctx.Backend.UpdateBusinessEventService(svc); err != nil { return mdlerrors.NewBackend("update business event service", err) } + if _, err := applyDocumentFolder(ctx, svc.ID, existingContainerID, svc.ContainerID); err != nil { + return err + } ctx.ReportMutation("Modified", "business event service: %s.%s", moduleName, stmt.Name.Name) } else { if err := ctx.Backend.CreateBusinessEventService(svc); err != nil { diff --git a/mdl/executor/cmd_constants.go b/mdl/executor/cmd_constants.go index b5e5ca222..6a45fb145 100644 --- a/mdl/executor/cmd_constants.go +++ b/mdl/executor/cmd_constants.go @@ -299,6 +299,13 @@ func createConstant(ctx *ExecContext, stmt *ast.CreateConstantStmt) error { if err := ctx.Backend.UpdateConstant(c); err != nil { return mdlerrors.NewBackend("update constant", err) } + target, err := resolveRequestedFolder(ctx, module.ID, stmt.Folder) + if err != nil { + return err + } + if _, err := applyDocumentFolder(ctx, c.ID, c.ContainerID, target); err != nil { + return err + } invalidateHierarchy(ctx) ctx.ReportMutation("Modified", "constant: %s.%s", modName, c.Name) return nil diff --git a/mdl/executor/cmd_datatransformer.go b/mdl/executor/cmd_datatransformer.go index ea2aefef2..9d21c684d 100644 --- a/mdl/executor/cmd_datatransformer.go +++ b/mdl/executor/cmd_datatransformer.go @@ -76,7 +76,7 @@ func describeDataTransformer(ctx *ExecContext, name ast.QualifiedName) error { w := ctx.Output // Emit re-executable MDL - fmt.Fprintf(w, "create data transformer %s.%s\n", modName, dt.Name) + fmt.Fprintf(w, "create data transformer %s.%s%s\n", modName, dt.Name, describeFolderClause(ctx, dt.ContainerID)) // Source — collapse newlines into spaces for single-line string sourceContent := strings.ReplaceAll(dt.SourceJSON, "\n", " ") @@ -125,8 +125,17 @@ func execCreateDataTransformer(ctx *ExecContext, s *ast.CreateDataTransformerStm return mdlerrors.NewNotFound("module", s.Name.Module) } + var existingContainer model.ID + if existing != nil { + existingContainer = existing.ContainerID + } + containerID, err := containerForDocument(ctx, module.ID, s.Folder, existingContainer) + if err != nil { + return err + } + dt := &model.DataTransformer{ - ContainerID: module.ID, + ContainerID: containerID, Name: s.Name.Name, SourceType: s.SourceType, SourceJSON: s.SourceJSON, @@ -148,6 +157,9 @@ func execCreateDataTransformer(ctx *ExecContext, s *ast.CreateDataTransformerStm if err := ctx.Backend.UpdateDataTransformer(dt); err != nil { return mdlerrors.NewBackend("update data transformer", err) } + if _, err := applyDocumentFolder(ctx, dt.ID, existingContainer, containerID); err != nil { + return err + } if !ctx.Quiet { ctx.ReportMutation("Modified", "data transformer: %s.%s (%d steps)", s.Name.Module, s.Name.Name, len(dt.Steps)) diff --git a/mdl/executor/cmd_dbconnection.go b/mdl/executor/cmd_dbconnection.go index 6669cb4b4..78b67b6e2 100644 --- a/mdl/executor/cmd_dbconnection.go +++ b/mdl/executor/cmd_dbconnection.go @@ -32,6 +32,7 @@ func createDatabaseConnection(ctx *ExecContext, stmt *ast.CreateDatabaseConnecti h, _ := getHierarchy(ctx) var existingConnID model.ID + var existingContainer model.ID // Target the live connection and carry its exclusion forward (#914). existingExcluded := false if ex, ok := pickLive(existing, @@ -47,6 +48,7 @@ func createDatabaseConnection(ctx *ExecContext, stmt *ast.CreateDatabaseConnecti } existingConnID = ex.ID existingExcluded = ex.Excluded + existingContainer = ex.ContainerID } // A literal where Mendix stores a ConstantIdentifier writes a project that @@ -72,8 +74,13 @@ func createDatabaseConnection(ctx *ExecContext, stmt *ast.CreateDatabaseConnecti connInputValue = resolveConstantDefault(ctx, connStr) } + containerID, err := containerForDocument(ctx, module.ID, stmt.Folder, existingContainer) + if err != nil { + return err + } + conn := &model.DatabaseConnection{ - ContainerID: module.ID, + ContainerID: containerID, Name: stmt.Name.Name, DatabaseType: stmt.DatabaseType, ConnectionString: connStr, @@ -134,6 +141,9 @@ func createDatabaseConnection(ctx *ExecContext, stmt *ast.CreateDatabaseConnecti if err := ctx.Backend.UpdateDatabaseConnection(conn); err != nil { return mdlerrors.NewBackend("update database connection", err) } + if _, err := applyDocumentFolder(ctx, conn.ID, existingContainer, containerID); err != nil { + return err + } invalidateHierarchy(ctx) ctx.ReportMutation("Modified", "database connection: %s.%s", stmt.Name.Module, stmt.Name.Name) return nil @@ -227,7 +237,7 @@ func describeDatabaseConnection(ctx *ExecContext, name ast.QualifiedName) error // outputDatabaseConnectionMDL outputs a database connection definition in MDL format. func outputDatabaseConnectionMDL(ctx *ExecContext, conn *model.DatabaseConnection, moduleName string) error { - fmt.Fprintf(ctx.Output, "create database connection %s.%s\n", moduleName, conn.Name) + fmt.Fprintf(ctx.Output, "create database connection %s.%s%s\n", moduleName, conn.Name, describeFolderClause(ctx, conn.ContainerID)) fmt.Fprintf(ctx.Output, "type '%s'\n", conn.DatabaseType) // Connection string diff --git a/mdl/executor/cmd_diff_mdl.go b/mdl/executor/cmd_diff_mdl.go index be8f6ffc3..72cbc2c19 100644 --- a/mdl/executor/cmd_diff_mdl.go +++ b/mdl/executor/cmd_diff_mdl.go @@ -431,18 +431,21 @@ func microflowStatementToMDL(ctx *ExecContext, stmt ast.MicroflowStatement, inde } lines = append(lines, indentStr+"end if;") + // Branch bodies render at indent+2, one level in from their `when` at + // indent+1. Both splits rendered them at indent+1 — the same column as the + // branch keyword — which is unreadable once anything nests (#913). case *ast.EnumSplitStmt: lines = append(lines, fmt.Sprintf("%scase $%s", indentStr, s.Variable)) for _, c := range s.Cases { lines = append(lines, fmt.Sprintf("%s when %s then", indentStr, formatEnumSplitCaseValues(enumSplitCaseValues(c)))) for _, caseStmt := range c.Body { - lines = append(lines, microflowStatementToMDL(ctx, caseStmt, indent+1)...) + lines = append(lines, microflowStatementToMDL(ctx, caseStmt, indent+2)...) } } if len(s.ElseBody) > 0 { lines = append(lines, indentStr+" else") for _, elseStmt := range s.ElseBody { - lines = append(lines, microflowStatementToMDL(ctx, elseStmt, indent+1)...) + lines = append(lines, microflowStatementToMDL(ctx, elseStmt, indent+2)...) } } lines = append(lines, indentStr+"end case;") @@ -450,15 +453,16 @@ func microflowStatementToMDL(ctx *ExecContext, stmt ast.MicroflowStatement, inde case *ast.InheritanceSplitStmt: lines = append(lines, fmt.Sprintf("%ssplit type $%s", indentStr, s.Variable)) for _, c := range s.Cases { - lines = append(lines, fmt.Sprintf("%scase %s", indentStr, c.Entity.String())) + lines = append(lines, fmt.Sprintf("%s when %s then", indentStr, c.Entity.String())) for _, caseStmt := range c.Body { - lines = append(lines, microflowStatementToMDL(ctx, caseStmt, indent+1)...) + lines = append(lines, microflowStatementToMDL(ctx, caseStmt, indent+2)...) } } if len(s.ElseBody) > 0 { - lines = append(lines, indentStr+"else") + // Mendix's `(empty)` flow (null object), not a default branch. + lines = append(lines, indentStr+" when (empty) then") for _, elseStmt := range s.ElseBody { - lines = append(lines, microflowStatementToMDL(ctx, elseStmt, indent+1)...) + lines = append(lines, microflowStatementToMDL(ctx, elseStmt, indent+2)...) } } lines = append(lines, indentStr+"end split;") diff --git a/mdl/executor/cmd_enumerations.go b/mdl/executor/cmd_enumerations.go index c2e353396..d02a7cfea 100644 --- a/mdl/executor/cmd_enumerations.go +++ b/mdl/executor/cmd_enumerations.go @@ -81,9 +81,20 @@ func execCreateEnumeration(ctx *ExecContext, s *ast.CreateEnumerationStmt) error // Excluded is model state, not script state: MDL cannot express it for // an enumeration, so the stored value is the one that survives (#914). enum.Excluded = existingEnum.Excluded + // Placement is model state too when the statement is silent about it. + if s.Folder == "" { + enum.ContainerID = existingEnum.ContainerID + } if err := ctx.Backend.UpdateEnumeration(enum); err != nil { return mdlerrors.NewBackend("update enumeration", err) } + target, terr := resolveRequestedFolder(ctx, module.ID, s.Folder) + if terr != nil { + return terr + } + if _, err := applyDocumentFolder(ctx, enum.ID, existingEnum.ContainerID, target); err != nil { + return err + } invalidateHierarchy(ctx) ctx.ReportMutation("Modified", "enumeration: %s", s.Name) return nil diff --git a/mdl/executor/cmd_export_mappings.go b/mdl/executor/cmd_export_mappings.go index 9c612ee76..7bb0f0cda 100644 --- a/mdl/executor/cmd_export_mappings.go +++ b/mdl/executor/cmd_export_mappings.go @@ -104,6 +104,11 @@ func describeExportMapping(ctx *ExecContext, name ast.QualifiedName) error { moduleName := h.GetModuleName(modID) fmt.Fprintf(ctx.Output, "create or modify export mapping %s.%s\n", moduleName, em.Name) + // Without this the description round-trips to the module root: replaying + // it in a fresh project would recreate the mapping unfiled (#932). + if folderPath := h.BuildFolderPath(em.ContainerID); folderPath != "" { + fmt.Fprintf(ctx.Output, " folder '%s'\n", folderPath) + } if em.JsonStructure != "" { fmt.Fprintf(ctx.Output, " with json structure %s\n", em.JsonStructure) @@ -191,7 +196,18 @@ func execCreateExportMapping(ctx *ExecContext, s *ast.CreateExportMappingStmt) e if err != nil { return mdlerrors.NewNotFound("module", s.Name.Module) } - containerID := module.ID + // A folder clause places the mapping; without one a new mapping goes to the + // module root and an existing one stays where it is (#932). + containerID, err := resolveRequestedFolder(ctx, module.ID, s.Folder) + if err != nil { + return err + } + if containerID == "" { + containerID = module.ID + if existing != nil { + containerID = existing.ContainerID + } + } em := &model.ExportMapping{ ContainerID: containerID, @@ -237,6 +253,9 @@ func execCreateExportMapping(ctx *ExecContext, s *ast.CreateExportMappingStmt) e if err := ctx.Backend.UpdateExportMapping(em); err != nil { return mdlerrors.NewBackend("update export mapping", err) } + if _, err := applyDocumentFolder(ctx, em.ID, existing.ContainerID, containerID); err != nil { + return err + } if !ctx.Quiet { ctx.ReportMutation("Modified", "export mapping %s.%s", s.Name.Module, s.Name.Name) } @@ -246,6 +265,7 @@ func execCreateExportMapping(ctx *ExecContext, s *ast.CreateExportMappingStmt) e if err := ctx.Backend.CreateExportMapping(em); err != nil { return mdlerrors.NewBackend("create export mapping", err) } + invalidateHierarchy(ctx) if !ctx.Quiet { fmt.Fprintf(ctx.Output, "Created export mapping %s.%s\n", s.Name.Module, s.Name.Name) diff --git a/mdl/executor/cmd_imagecollections.go b/mdl/executor/cmd_imagecollections.go index 3d1095966..e39fed927 100644 --- a/mdl/executor/cmd_imagecollections.go +++ b/mdl/executor/cmd_imagecollections.go @@ -12,6 +12,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" ) // execCreateImageCollection handles CREATE IMAGE COLLECTION statements. @@ -32,9 +33,13 @@ func execCreateImageCollection(ctx *ExecContext, s *ast.CreateImageCollectionStm return mdlerrors.NewAlreadyExists("image collection", s.Name.Module+"."+s.Name.Name) } - containerID := module.ID + var existingContainer model.ID if existing != nil { - containerID = existing.ContainerID + existingContainer = existing.ContainerID + } + containerID, err := containerForDocument(ctx, module.ID, s.Folder, existingContainer) + if err != nil { + return err } // Build ImageCollection @@ -76,6 +81,9 @@ func execCreateImageCollection(ctx *ExecContext, s *ast.CreateImageCollectionStm if err := ctx.Backend.UpdateImageCollection(ic); err != nil { return mdlerrors.NewBackend("update image collection", err) } + if _, err := applyDocumentFolder(ctx, ic.ID, existingContainer, containerID); err != nil { + return err + } ctx.ReportMutation("Modified", "image collection: %s", s.Name) } else { if err := ctx.Backend.CreateImageCollection(ic); err != nil { @@ -134,7 +142,7 @@ func describeImageCollection(ctx *ExecContext, name ast.QualifiedName) error { qualifiedName := fmt.Sprintf("%s.%s", modName, ic.Name) if len(ic.Images) == 0 { - fmt.Fprintf(ctx.Output, "create or modify image collection %s", qualifiedName) + fmt.Fprintf(ctx.Output, "create or modify image collection %s%s", qualifiedName, describeFolderClause(ctx, ic.ContainerID)) if exportLevel != "Hidden" { fmt.Fprintf(ctx.Output, " export level '%s'", exportLevel) } @@ -149,7 +157,7 @@ func describeImageCollection(ctx *ExecContext, name ast.QualifiedName) error { return mdlerrors.NewBackend("create preview directory", err) } - fmt.Fprintf(ctx.Output, "create or modify image collection %s", qualifiedName) + fmt.Fprintf(ctx.Output, "create or modify image collection %s%s", qualifiedName, describeFolderClause(ctx, ic.ContainerID)) if exportLevel != "Hidden" { fmt.Fprintf(ctx.Output, " export level '%s'", exportLevel) } diff --git a/mdl/executor/cmd_import_mappings.go b/mdl/executor/cmd_import_mappings.go index faa4d2735..dd68f88fe 100644 --- a/mdl/executor/cmd_import_mappings.go +++ b/mdl/executor/cmd_import_mappings.go @@ -104,6 +104,11 @@ func describeImportMapping(ctx *ExecContext, name ast.QualifiedName) error { moduleName := h.GetModuleName(modID) fmt.Fprintf(ctx.Output, "create or modify import mapping %s.%s\n", moduleName, im.Name) + // Without this the description round-trips to the module root: replaying + // it in a fresh project would recreate the mapping unfiled (#932). + if folderPath := h.BuildFolderPath(im.ContainerID); folderPath != "" { + fmt.Fprintf(ctx.Output, " folder '%s'\n", folderPath) + } if im.JsonStructure != "" { fmt.Fprintf(ctx.Output, " with json structure %s\n", im.JsonStructure) @@ -256,7 +261,18 @@ func execCreateImportMapping(ctx *ExecContext, s *ast.CreateImportMappingStmt) e if err != nil { return mdlerrors.NewNotFound("module", s.Name.Module) } - containerID := module.ID + // A folder clause places the mapping; without one a new mapping goes to the + // module root and an existing one stays where it is (#932). + containerID, err := resolveRequestedFolder(ctx, module.ID, s.Folder) + if err != nil { + return err + } + if containerID == "" { + containerID = module.ID + if existing != nil { + containerID = existing.ContainerID + } + } im := &model.ImportMapping{ ContainerID: containerID, @@ -299,6 +315,9 @@ func execCreateImportMapping(ctx *ExecContext, s *ast.CreateImportMappingStmt) e if err := ctx.Backend.UpdateImportMapping(im); err != nil { return mdlerrors.NewBackend("update import mapping", err) } + if _, err := applyDocumentFolder(ctx, im.ID, existing.ContainerID, containerID); err != nil { + return err + } if !ctx.Quiet { ctx.ReportMutation("Modified", "import mapping %s.%s", s.Name.Module, s.Name.Name) } @@ -308,6 +327,7 @@ func execCreateImportMapping(ctx *ExecContext, s *ast.CreateImportMappingStmt) e if err := ctx.Backend.CreateImportMapping(im); err != nil { return mdlerrors.NewBackend("create import mapping", err) } + invalidateHierarchy(ctx) if !ctx.Quiet { fmt.Fprintf(ctx.Output, "Created import mapping %s.%s\n", s.Name.Module, s.Name.Name) diff --git a/mdl/executor/cmd_javaactions.go b/mdl/executor/cmd_javaactions.go index 13eae1083..7a2a5b1bf 100644 --- a/mdl/executor/cmd_javaactions.go +++ b/mdl/executor/cmd_javaactions.go @@ -92,6 +92,7 @@ func describeJavaAction(ctx *ExecContext, name ast.QualifiedName) error { // Build CREATE JAVA ACTION statement sb.WriteString("create java action ") sb.WriteString(qualifiedName) + sb.WriteString(describeFolderClause(ctx, ja.ContainerID)) sb.WriteString("(") // Parameters — one per line when descriptions are present @@ -319,6 +320,7 @@ func execCreateJavaAction(ctx *ExecContext, s *ast.CreateJavaActionStmt) error { return mdlerrors.NewBackend("list java actions", err) } var existingJAID model.ID + var existingContainer model.ID // Target the live action and carry its exclusion forward (#914). existingExcluded := false if existing, ok := pickLive(jas, @@ -332,6 +334,13 @@ func execCreateJavaAction(ctx *ExecContext, s *ast.CreateJavaActionStmt) error { } existingJAID = existing.ID existingExcluded = existing.Excluded + existingContainer = existing.ContainerID + } + + moduleID := containerID + containerID, err = containerForDocument(ctx, moduleID, s.Folder, existingContainer) + if err != nil { + return err } newID := model.ID(types.GenerateID()) @@ -432,6 +441,9 @@ func execCreateJavaAction(ctx *ExecContext, s *ast.CreateJavaActionStmt) error { if err := ctx.Backend.UpdateJavaAction(ja); err != nil { return mdlerrors.NewBackend("update java action", err) } + if _, err := applyDocumentFolder(ctx, ja.ID, existingContainer, containerID); err != nil { + return err + } } else { if err := ctx.Backend.CreateJavaAction(ja); err != nil { return mdlerrors.NewBackend("create java action", err) diff --git a/mdl/executor/cmd_javascript_actions.go b/mdl/executor/cmd_javascript_actions.go index 7606c1557..198611fee 100644 --- a/mdl/executor/cmd_javascript_actions.go +++ b/mdl/executor/cmd_javascript_actions.go @@ -100,6 +100,7 @@ func describeJavaScriptAction(ctx *ExecContext, name ast.QualifiedName) error { } sb.WriteString(">") } + sb.WriteString(describeFolderClause(ctx, jsa.ContainerID)) sb.WriteString("(") // Parameters diff --git a/mdl/executor/cmd_javascript_actions_write.go b/mdl/executor/cmd_javascript_actions_write.go index acba95818..9409a8be1 100644 --- a/mdl/executor/cmd_javascript_actions_write.go +++ b/mdl/executor/cmd_javascript_actions_write.go @@ -48,6 +48,7 @@ func execCreateJavaScriptAction(ctx *ExecContext, s *ast.CreateJavaScriptActionS return mdlerrors.NewBackend("list javascript actions", err) } var existingID model.ID + var existingContainer model.ID // Target the live action and carry its exclusion forward (#914). existingExcluded := false if ex, ok := pickLive(existing, @@ -61,6 +62,13 @@ func execCreateJavaScriptAction(ctx *ExecContext, s *ast.CreateJavaScriptActionS } existingID = ex.ID existingExcluded = ex.Excluded + existingContainer = ex.ContainerID + } + + moduleID := containerID + containerID, err = containerForDocument(ctx, moduleID, s.Folder, existingContainer) + if err != nil { + return err } newID := model.ID(types.GenerateID()) @@ -145,6 +153,9 @@ func execCreateJavaScriptAction(ctx *ExecContext, s *ast.CreateJavaScriptActionS if err := ctx.Backend.UpdateJavaScriptAction(jsa); err != nil { return mdlerrors.NewBackend("update javascript action", err) } + if _, err := applyDocumentFolder(ctx, jsa.ID, existingContainer, containerID); err != nil { + return err + } } else { if err := ctx.Backend.CreateJavaScriptAction(jsa); err != nil { return mdlerrors.NewBackend("create javascript action", err) diff --git a/mdl/executor/cmd_jsonstructures.go b/mdl/executor/cmd_jsonstructures.go index 3aa3e323f..a97cbf235 100644 --- a/mdl/executor/cmd_jsonstructures.go +++ b/mdl/executor/cmd_jsonstructures.go @@ -227,6 +227,9 @@ func execCreateJsonStructure(ctx *ExecContext, s *ast.CreateJsonStructureStmt) e if err := ctx.Backend.UpdateJsonStructure(js); err != nil { return mdlerrors.NewBackend("update json structure", err) } + if _, err := applyDocumentFolder(ctx, js.ID, existing.ContainerID, containerID); err != nil { + return err + } ctx.ReportMutation("Modified", "json structure: %s", s.Name) } else { if err := ctx.Backend.CreateJsonStructure(js); err != nil { diff --git a/mdl/executor/cmd_list_folders.go b/mdl/executor/cmd_list_folders.go index 5aa0568a4..7bc2ab08a 100644 --- a/mdl/executor/cmd_list_folders.go +++ b/mdl/executor/cmd_list_folders.go @@ -257,9 +257,54 @@ func documentsByContainer(ctx *ExecContext, h *ContainerHierarchy) map[model.ID] put("ImageCollection", x.Name, x.ContainerID) } } + + // Everything above is a hand-maintained list of document kinds, and #892 is + // what that costs: five kinds were missing, so a folder holding four + // documents rendered as `[0]` and dropping it looked safe. Adding the + // missing five did not fix the shape of the problem — layouts, menus, + // JavaScript actions and building blocks were still absent, and #932 made + // all of them movable, so a move you could not see was a move you could not + // check. + // + // So the list is now a source of good LABELS, not the source of truth for + // what exists. This pass walks the unit table, which cannot omit a kind + // because it never asks what kind anything is, and fills in every document + // the typed calls did not already account for. A kind added to Mendix + // tomorrow appears here without anyone editing this function. + if units, err := ctx.Backend.ListDocumentUnits(); err == nil { + seen := make(map[model.ID]map[string]bool, len(out)) + for container, docs := range out { + names := make(map[string]bool, len(docs)) + for _, d := range docs { + names[d.Name] = true + } + seen[container] = names + } + for _, u := range units { + if seen[u.ContainerID][u.Name] { + continue + } + put(documentKindLabel(u.Kind), u.Name, u.ContainerID) + } + } return out } +// documentKindLabel renders a derived kind ("json structure") in the CamelCase +// style the typed entries above use ("JsonStructure"), so one listing does not +// mix two spellings of the same idea. +func documentKindLabel(kind string) string { + var b strings.Builder + for _, word := range strings.Fields(kind) { + b.WriteString(strings.ToUpper(word[:1])) + b.WriteString(word[1:]) + } + if b.Len() == 0 { + return "Document" + } + return b.String() +} + // modulesInScope returns the modules the listing covers. func modulesInScope(ctx *ExecContext, inModule string) []*model.Module { modules, err := getModulesFromCache(ctx) diff --git a/mdl/executor/cmd_menus.go b/mdl/executor/cmd_menus.go index a0cd06721..2d66119b8 100644 --- a/mdl/executor/cmd_menus.go +++ b/mdl/executor/cmd_menus.go @@ -9,6 +9,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" ) // describeMenu renders a standalone Menus$MenuDocument — the reusable menu a @@ -42,7 +43,7 @@ func describeMenu(ctx *ExecContext, name ast.QualifiedName) error { // Output is re-executable: the item syntax is the same one CREATE MENU // accepts, so describe → exec → describe is a fixed point. - fmt.Fprintf(ctx.Output, "create or modify menu %s.%s (\n", name.Module, md.Name) + fmt.Fprintf(ctx.Output, "create or modify menu %s.%s%s (\n", name.Module, md.Name, describeFolderClause(ctx, md.ContainerID)) printMenuMDL(ctx.Output, md.Items, 1, "CREATE MENU") fmt.Fprintln(ctx.Output, ");") return nil @@ -69,9 +70,18 @@ func execCreateMenu(ctx *ExecContext, s *ast.CreateMenuStmt) error { return mdlerrors.NewAlreadyExists("menu", s.Name.String()) } + var existingContainer model.ID + if existing != nil { + existingContainer = existing.ContainerID + } + containerID, err := containerForDocument(ctx, mod.ID, s.Folder, existingContainer) + if err != nil { + return err + } + md := &types.MenuDocument{ Name: s.Name.Name, - ContainerID: mod.ID, + ContainerID: containerID, Documentation: s.Documentation, Items: menuItemsFromAST(s.Items), } @@ -80,7 +90,6 @@ func execCreateMenu(ctx *ExecContext, s *ast.CreateMenuStmt) error { // Preserve the document's identity and the properties MDL does not // author, so a modify does not silently reset them. md.ID = existing.ID - md.ContainerID = existing.ContainerID md.ExportLevel = existing.ExportLevel md.Excluded = existing.Excluded if md.Documentation == "" { @@ -89,6 +98,9 @@ func execCreateMenu(ctx *ExecContext, s *ast.CreateMenuStmt) error { if err := ctx.Backend.UpdateMenuDocument(md); err != nil { return mdlerrors.NewBackend("update menu", err) } + if _, err := applyDocumentFolder(ctx, md.ID, existingContainer, containerID); err != nil { + return err + } ctx.ReportMutation("Modified", "menu %s", s.Name.String()) return nil } diff --git a/mdl/executor/cmd_microflows_create.go b/mdl/executor/cmd_microflows_create.go index 524fcfda1..c389f0ad2 100644 --- a/mdl/executor/cmd_microflows_create.go +++ b/mdl/executor/cmd_microflows_create.go @@ -310,6 +310,9 @@ func execCreateMicroflow(ctx *ExecContext, s *ast.CreateMicroflowStmt) error { if err := ctx.Backend.UpdateMicroflow(mf); err != nil { return mdlerrors.NewBackend("update microflow", err) } + if _, err := applyDocumentFolder(ctx, mf.ID, existingContainerID, containerID); err != nil { + return err + } ctx.ReportMutation("Replaced", "microflow: %s.%s", s.Name.Module, s.Name.Name) } else { if err := ctx.Backend.CreateMicroflow(mf); err != nil { diff --git a/mdl/executor/cmd_microflows_inheritance_test.go b/mdl/executor/cmd_microflows_inheritance_test.go index 069d7d576..a6b9dfc8d 100644 --- a/mdl/executor/cmd_microflows_inheritance_test.go +++ b/mdl/executor/cmd_microflows_inheritance_test.go @@ -111,9 +111,11 @@ func TestTraverseFlow_InheritanceSplit(t *testing.T) { e.traverseFlow(mkID("split"), activityMap, flowsByOrigin, splitMergeMap, visited, entityNames, nil, &lines, 1, nil, 0, nil) assertLineContains(t, lines, "split type $Input") - assertLineContains(t, lines, "case Sample.SpecializedInput") + assertLineContains(t, lines, "when Sample.SpecializedInput then") assertLineContains(t, lines, "cast $SpecificInput;") - assertLineContains(t, lines, "else") + // The empty branch is spelled for what it is — Mendix's `(empty)` flow, a + // null object — not `else`, which read as a default branch (#913). + assertLineContains(t, lines, "when (empty) then") assertLineContains(t, lines, "end split;") } @@ -140,8 +142,8 @@ func TestTraverseFlow_InheritanceSplitPreservesExplicitCaseOrder(t *testing.T) { e.traverseFlow(mkID("split"), activityMap, flowsByOrigin, splitMergeMap, visited, nil, nil, &lines, 1, nil, 0, nil) out := strings.Join(lines, "\n") - accountIdx := strings.Index(out, "case Sample.Account") - userIdx := strings.Index(out, "case Sample.User") + accountIdx := strings.Index(out, "when Sample.Account then") + userIdx := strings.Index(out, "when Sample.User then") if accountIdx == -1 || userIdx == -1 { t.Fatalf("missing expected cases:\n%s", out) } diff --git a/mdl/executor/cmd_microflows_show_helpers.go b/mdl/executor/cmd_microflows_show_helpers.go index 21c890be1..58d1dd895 100644 --- a/mdl/executor/cmd_microflows_show_helpers.go +++ b/mdl/executor/cmd_microflows_show_helpers.go @@ -1571,13 +1571,17 @@ func emitEnumSplitStatement( branches = append(branches, enumBranch{values: []string{caseValue}, flow: flow}) } + // Bodies traverse at indent+2, one level in from their `when` at indent+1. + // They used to traverse at indent+1 — the SAME column as the `when` — which + // made a nested `if`'s `else` land exactly where a reader expects a branch + // keyword, in output where an `else` on a `case` is an MDL008 error (#913). for _, branch := range branches { *lines = append(*lines, indentStr+" when "+formatEnumSplitCaseValues(branch.values)+" then") - traverseFlowUntilMerge(ctx, branch.flow.DestinationID, mergeID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+1, sourceMap, headerLineCount, annotationsByTarget) + traverseFlowUntilMerge(ctx, branch.flow.DestinationID, mergeID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+2, sourceMap, headerLineCount, annotationsByTarget) } if elseFlow != nil { *lines = append(*lines, indentStr+" else") - traverseFlowUntilMerge(ctx, elseFlow.DestinationID, mergeID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+1, sourceMap, headerLineCount, annotationsByTarget) + traverseFlowUntilMerge(ctx, elseFlow.DestinationID, mergeID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+2, sourceMap, headerLineCount, annotationsByTarget) } *lines = append(*lines, indentStr+"end case;") @@ -1623,13 +1627,13 @@ func emitInheritanceSplitStatement( elseFlow = flow continue } - *lines = append(*lines, indentStr+"case "+caseName) - traverseFlowUntilMerge(ctx, flow.DestinationID, branchStopID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+1, sourceMap, headerLineCount, annotationsByTarget) + *lines = append(*lines, indentStr+" when "+caseName+" then") + traverseFlowUntilMerge(ctx, flow.DestinationID, branchStopID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+2, sourceMap, headerLineCount, annotationsByTarget) } if elseFlow != nil { elseLineIdx := len(*lines) - *lines = append(*lines, indentStr+"else") - traverseFlowUntilMerge(ctx, elseFlow.DestinationID, branchStopID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+1, sourceMap, headerLineCount, annotationsByTarget) + *lines = append(*lines, indentStr+" when (empty) then") + traverseFlowUntilMerge(ctx, elseFlow.DestinationID, branchStopID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+2, sourceMap, headerLineCount, annotationsByTarget) // Remove an empty else block, as the if/else emitters above do. On an // object-type decision this flow is the `(empty)` case (for a null // object), which the builder emits unconditionally — CE0089 without it — diff --git a/mdl/executor/cmd_microflows_traverse_test.go b/mdl/executor/cmd_microflows_traverse_test.go index 4264a1930..68c0e5cc8 100644 --- a/mdl/executor/cmd_microflows_traverse_test.go +++ b/mdl/executor/cmd_microflows_traverse_test.go @@ -1579,10 +1579,10 @@ func TestTraverseFlow_InheritanceSplitOmitsEmptyElse(t *testing.T) { return strings.Join(lines, "\n") } - if out := run(true); strings.Contains(out, "else") { - t.Errorf("DESCRIBE invented an `else` for the (empty) flow with no body:\n%s", out) + if out := run(true); strings.Contains(out, "(empty)") { + t.Errorf("DESCRIBE invented an empty branch for the (empty) flow with no body:\n%s", out) } - if out := run(false); !strings.Contains(out, "else") { - t.Errorf("an else WITH a body must still render:\n%s", out) + if out := run(false); !strings.Contains(out, "when (empty) then") { + t.Errorf("an (empty) branch WITH a body must still render:\n%s", out) } } diff --git a/mdl/executor/cmd_move.go b/mdl/executor/cmd_move.go index ff5034c0c..c783845e9 100644 --- a/mdl/executor/cmd_move.go +++ b/mdl/executor/cmd_move.go @@ -90,7 +90,14 @@ func execMove(ctx *ExecContext, s *ast.MoveStmt) error { return err } default: - return mdlerrors.NewUnsupported("unsupported document type: " + string(s.DocumentType)) + // Every remaining doctype is a plain top-level document, and a move is + // one containment row — so they share a handler instead of each needing + // a list call and a finder written for it. The typed cases above are the + // ones whose move does extra work (access-role remapping, reference + // rewriting); this is everything else, including doctypes added later. + if err := moveDocumentUnit(ctx, s.DocumentType, s.Name, targetContainerID); err != nil { + return err + } } // For cross-module moves, update all BY_NAME references throughout the project diff --git a/mdl/executor/cmd_move_document.go b/mdl/executor/cmd_move_document.go new file mode 100644 index 000000000..e81883092 --- /dev/null +++ b/mdl/executor/cmd_move_document.go @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: Apache-2.0 + +// cmd_move_document.go — MOVE for document types that have no bespoke handler. +// +// Before #932 MOVE accepted nine doctypes. The other twenty-odd were not +// unimplemented, merely unlisted: a top-level document's move is one +// containment row whatever the document is, so the work per doctype was a +// grammar entry and a lookup, not new machinery. Rather than write that lookup +// twenty more times — each one a `List()` scan that can only find the +// kinds someone remembered to add — this resolves the name through the unit +// table, which is type-agnostic by construction. +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/model" +) + +// moveDocumentUnit reparents a document located by name rather than by type, +// and reports it by the kind it turned out to be. +func moveDocumentUnit(ctx *ExecContext, docType ast.DocumentType, name ast.QualifiedName, targetContainerID model.ID) error { + asked := strings.ToLower(string(docType)) + // The parser cannot produce a doctype outside the registry, so reaching + // here with one means a DocumentType was added to the AST without being + // added to the registry the grammar and this handler share. Refuse rather + // than fall through: the lookup below is by name only, so an unregistered + // doctype would happily move whatever that name resolved to. + if !ast.IsMoveDocumentType(asked) { + return mdlerrors.NewUnsupported("unsupported document type: " + string(docType)) + } + doc, err := ctx.Backend.FindDocumentUnit(name.Module, name.Name) + if err != nil { + return mdlerrors.NewBackend("find "+asked, err) + } + if doc == nil { + return mdlerrors.NewNotFound(asked, name.String()) + } + if err := checkMovedDocumentType(asked, doc.Kind, name); err != nil { + return err + } + if err := ctx.Backend.MoveDocument(doc.ID, targetContainerID); err != nil { + return mdlerrors.NewBackend("move "+asked, err) + } + invalidateHierarchy(ctx) + // Report the kind read off the document, not the kind the statement named, + // so the line cannot describe something other than what actually moved. + fmt.Fprintf(ctx.Output, "Moved %s %s to new location\n", doc.Kind, name.String()) + return nil +} + +// checkMovedDocumentType refuses a document whose stored type is not the one +// the statement named, and says what it actually is. +// +// The comparison is against the kind DERIVED from the stored $Type, not against +// a hand-written table of storage names. That matters: the two such tables this +// repo already has disagree with each other and with a real project — +// mdl/types/unit_types.go says JavaScript actions live under "JavaActions$", +// while the writer inserts and Studio Pro stores "JavaScriptActions$" — so a +// third would be a third thing to get wrong, and getting it wrong here blocks +// legitimate moves. +// +// Hence the deliberate asymmetry: refuse only when the derived kind is itself a +// doctype MOVE knows how to spell. A kind mxcli has no MDL word for — a page +// template, a document type a future Mendix version adds — is a derivation this +// code cannot vouch for, so it defers to the name lookup. Document names are +// unique within a module, so that lookup is the authority; this check is here to +// catch a mistyped doctype, not to second-guess the model. +func checkMovedDocumentType(asked, found string, name ast.QualifiedName) error { + if found == "" || strings.EqualFold(found, asked) || !ast.IsMoveDocumentType(found) { + return nil + } + return mdlerrors.NewValidation(fmt.Sprintf( + "%s is a %s, not a %s — use 'move %s %s to ...'", + name.String(), found, asked, found, name.String())) +} diff --git a/mdl/executor/cmd_move_document_test.go b/mdl/executor/cmd_move_document_test.go new file mode 100644 index 000000000..4a04c813d --- /dev/null +++ b/mdl/executor/cmd_move_document_test.go @@ -0,0 +1,177 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" +) + +// moveMockBackend wires a project with one document of the given type, found by +// name through the type-agnostic lookup. +func moveMockBackend(t *testing.T, mod *model.Module, doc *types.DocumentUnit, probe *placementProbe) (*mock.MockBackend, *ContainerHierarchy) { + t.Helper() + mb, h := folderMockBackend(t, mod, probe) + mb.FindDocumentUnitFunc = func(moduleName, name string) (*types.DocumentUnit, error) { + if doc != nil && moduleName == mod.Name && name == doc.Name { + return doc, nil + } + return nil, nil + } + return mb, h +} + +// TestMoveImportMappingReachesStorage is the issue's second report: MOVE IMPORT +// MAPPING was a parse error, and the backend method that would have performed it +// existed on the interface, both engines and the mock — with nothing calling it. +func TestMoveImportMappingReachesStorage(t *testing.T) { + mod := mkModule("Module") + doc := &types.DocumentUnit{ + ID: nextID("unit"), + ContainerID: mod.ID, + Name: "IMM_Example", + Type: "ImportMappings$ImportMapping", + Kind: "import mapping", + } + var probe placementProbe + mb, h := moveMockBackend(t, mod, doc, &probe) + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execMove(ctx, &ast.MoveStmt{ + DocumentType: ast.DocumentTypeImportMapping, + Name: ast.QualifiedName{Module: "Module", Name: "IMM_Example"}, + Folder: "Private/Import mappings", + }) + assertNoError(t, err) + + if !probe.moved { + t.Fatal("the mapping was not reparented") + } + if probe.unitID != doc.ID { + t.Errorf("reparented %q, want %q", probe.unitID, doc.ID) + } + if probe.containerID == mod.ID { + t.Error("reparented to the module root rather than the resolved folder") + } + assertContainsStr(t, buf.String(), "Moved import mapping Module.IMM_Example") +} + +// TestMoveReportsTheKindItFound pins that the output names what actually moved +// rather than echoing the statement. The lookup is by name, so the two can +// differ — and a line that repeats the user's word for it would be a +// confident-sounding lie. +func TestMoveReportsTheKindItFound(t *testing.T) { + mod := mkModule("Module") + // A kind mxcli has no MOVE spelling for: the statement is accepted (the + // name resolved to exactly one document) but must be reported truthfully. + doc := &types.DocumentUnit{ + ID: nextID("unit"), + ContainerID: mod.ID, + Name: "Thing", + Type: "Forms$PageTemplate", + Kind: "page template", + } + var probe placementProbe + mb, h := moveMockBackend(t, mod, doc, &probe) + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execMove(ctx, &ast.MoveStmt{ + DocumentType: ast.DocumentTypeJsonStructure, + Name: ast.QualifiedName{Module: "Module", Name: "Thing"}, + Folder: "Private", + }) + assertNoError(t, err) + assertContainsStr(t, buf.String(), "Moved page template Module.Thing") + assertNotContainsStr(t, buf.String(), "json structure") +} + +// TestMoveRefusesAMistypedDoctype is the other half of that: when the derived +// kind IS a doctype MOVE can spell, the mismatch is a typo worth refusing, and +// the error must name the right statement to run instead. +func TestMoveRefusesAMistypedDoctype(t *testing.T) { + mod := mkModule("Module") + doc := &types.DocumentUnit{ + ID: nextID("unit"), + ContainerID: mod.ID, + Name: "JSON_Example", + Type: "JsonStructures$JsonStructure", + Kind: "json structure", + } + var probe placementProbe + mb, h := moveMockBackend(t, mod, doc, &probe) + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execMove(ctx, &ast.MoveStmt{ + DocumentType: ast.DocumentTypeQueue, + Name: ast.QualifiedName{Module: "Module", Name: "JSON_Example"}, + Folder: "Private", + }) + assertError(t, err) + assertContainsStr(t, err.Error(), "is a json structure, not a queue") + if probe.moved { + t.Error("a refused statement still reparented the document") + } +} + +// TestMoveUnknownDocumentIsNotFound pins that an absent name is reported as +// absent. The lookup returns nil for "no such document", and treating that as +// anything other than not-found would make MOVE silently succeed. +func TestMoveUnknownDocumentIsNotFound(t *testing.T) { + mod := mkModule("Module") + var probe placementProbe + mb, h := moveMockBackend(t, mod, nil, &probe) + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execMove(ctx, &ast.MoveStmt{ + DocumentType: ast.DocumentTypeJsonStructure, + Name: ast.QualifiedName{Module: "Module", Name: "Nope"}, + Folder: "Private", + }) + assertError(t, err) + assertContainsStr(t, err.Error(), "not found") + if probe.moved { + t.Error("a document that was not found was still reparented") + } +} + +// TestMoveEveryRegisteredDoctypeHasAPath walks the whole registry through +// execMove, so a doctype that parses but has no working handler fails here +// rather than in a user's project. +// +// Entity is excluded: it is not a unit and has its own handler, which needs a +// domain model rather than a document. +func TestMoveEveryRegisteredDoctypeHasAPath(t *testing.T) { + for _, docType := range ast.MoveDocumentTypeByKeyword { + t.Run(string(docType), func(t *testing.T) { + mod := mkModule("Module") + // Give the document the kind the statement names, so this test is + // about routing rather than about the mistyped-doctype check. + doc := &types.DocumentUnit{ + ID: nextID("unit"), + ContainerID: mod.ID, + Name: "Thing", + Type: "Some$Type", + Kind: strings.ToLower(string(docType)), + } + var probe placementProbe + mb, h := moveMockBackend(t, mod, doc, &probe) + // The typed handlers reach for their own list calls; give each an + // empty project so they report "not found" rather than panicking, + // and assert only that no doctype produces "unsupported". + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execMove(ctx, &ast.MoveStmt{ + DocumentType: docType, + Name: ast.QualifiedName{Module: "Module", Name: "Thing"}, + Folder: "Private", + }) + if err != nil && strings.Contains(err.Error(), "unsupported document type") { + t.Errorf("%s parses but has no handler: %v", docType, err) + } + }) + } +} diff --git a/mdl/executor/cmd_nanoflows_create.go b/mdl/executor/cmd_nanoflows_create.go index 55d4a8ed7..661aa71f5 100644 --- a/mdl/executor/cmd_nanoflows_create.go +++ b/mdl/executor/cmd_nanoflows_create.go @@ -262,6 +262,9 @@ func execCreateNanoflow(ctx *ExecContext, s *ast.CreateNanoflowStmt) error { if err := ctx.Backend.UpdateNanoflow(nf); err != nil { return mdlerrors.NewBackend("update nanoflow", err) } + if _, err := applyDocumentFolder(ctx, nf.ID, existingContainerID, containerID); err != nil { + return err + } ctx.ReportMutation("Replaced", "nanoflow: %s.%s", s.Name.Module, s.Name.Name) } else { if err := ctx.Backend.CreateNanoflow(nf); err != nil { diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index ce4c6107b..1a31d0eb2 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -1051,6 +1051,13 @@ func createODataClient(ctx *ExecContext, stmt *ast.CreateODataClientStmt) error if err := ctx.Backend.UpdateConsumedODataService(svc); err != nil { return mdlerrors.NewBackend("update OData client", err) } + target, terr := resolveRequestedFolder(ctx, module.ID, stmt.Folder) + if terr != nil { + return terr + } + if _, err := applyDocumentFolder(ctx, svc.ID, svc.ContainerID, target); err != nil { + return err + } invalidateHierarchy(ctx) ctx.ReportMutation("Modified", "OData client: %s.%s", modName, svc.Name) return nil @@ -1465,6 +1472,13 @@ func createODataService(ctx *ExecContext, stmt *ast.CreateODataServiceStmt) erro if err := ctx.Backend.UpdatePublishedODataService(svc); err != nil { return mdlerrors.NewBackend("update OData service", err) } + target, terr := resolveRequestedFolder(ctx, module.ID, stmt.Folder) + if terr != nil { + return terr + } + if _, err := applyDocumentFolder(ctx, svc.ID, svc.ContainerID, target); err != nil { + return err + } invalidateHierarchy(ctx) ctx.ReportMutation("Modified", "OData service: %s.%s", modName, svc.Name) return nil diff --git a/mdl/executor/cmd_pages_builder_bb_rebind_test.go b/mdl/executor/cmd_pages_builder_bb_rebind_test.go new file mode 100644 index 000000000..09fdcf880 --- /dev/null +++ b/mdl/executor/cmd_pages_builder_bb_rebind_test.go @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// TestBuildingBlockDataSourceRebindTargetsUnboundWidgets guards the regression +// #941's emitter fix exposed. +// +// `use building block X (datasource: …)` finds its binding point by re-rendering +// the block to MDL and re-parsing it. It used to look for a widget that already +// carried a DataSource property, on the stated assumption that "a block's +// outermost list/grid/dataview always emits one". That was only true because of +// the bug #941 fixed: an unbound datasource — which is what a reusable Atlas +// block has, since it is a template — rendered as the malformed +// `DataSource: database from ,`. Emitting nothing for it, which is correct, left +// the rebind with nothing to match. +// +// So the target is matched by widget TYPE, exactly as the action override +// already does. That case learned this lesson first: its comment notes Atlas +// blocks "ship placeholder buttons with no action", which is the same fact about +// the same blocks. +func TestBuildingBlockDataSourceRebindTargetsUnboundWidgets(t *testing.T) { + for _, widgetType := range []string{"gallery", "listview", "datagrid", "dataview", "DATAGRID2"} { + t.Run(widgetType, func(t *testing.T) { + if !isDataSourceWidget(&ast.WidgetV3{Type: widgetType}) { + t.Errorf("%s is not recognised as a datasource-carrying widget", widgetType) + } + }) + } + for _, widgetType := range []string{"container", "text", "actionbutton", "layoutgrid"} { + t.Run("not/"+widgetType, func(t *testing.T) { + if isDataSourceWidget(&ast.WidgetV3{Type: widgetType}) { + t.Errorf("%s must not be treated as a datasource target", widgetType) + } + }) + } +} + +// TestBuildingBlockRebindPrefersTheOutermost pins that the override still lands +// on the block's outermost datasource widget when the tree nests several — the +// binding-point rule the feature documents. +func TestBuildingBlockRebindPrefersTheOutermost(t *testing.T) { + inner := &ast.WidgetV3{Type: "listview", Name: "inner"} + outer := &ast.WidgetV3{Type: "gallery", Name: "outer", Children: []*ast.WidgetV3{inner}} + var got string + rebindFirst([]*ast.WidgetV3{outer}, isDataSourceWidget, func(w *ast.WidgetV3) { got = w.Name }) + if got != "outer" { + t.Errorf("rebound %q, want the outermost datasource widget", got) + } +} diff --git a/mdl/executor/cmd_pages_builder_listview_templates_test.go b/mdl/executor/cmd_pages_builder_listview_templates_test.go new file mode 100644 index 000000000..0f5730927 --- /dev/null +++ b/mdl/executor/cmd_pages_builder_listview_templates_test.go @@ -0,0 +1,159 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// newVehiclePB mirrors ako/TestApp: Pages.Vehicle with four specializations, plus +// an unrelated entity to test the specialization check against. +func newVehiclePB() *pageBuilder { + const modID = model.ID("mod") + ent := func(name, gen string) *domainmodel.Entity { + return &domainmodel.Entity{ + BaseElement: model.BaseElement{ID: model.ID("e-" + name)}, + Name: name, + GeneralizationRef: gen, + } + } + return &pageBuilder{ + paramEntityNames: map[string]string{}, + widgetScope: map[string]model.ID{}, + execCache: &executorCache{ + hierarchy: &ContainerHierarchy{moduleNames: map[model.ID]string{modID: "Pages"}}, + domainModels: []*domainmodel.DomainModel{{ + ContainerID: modID, + Entities: []*domainmodel.Entity{ + ent("Vehicle", ""), + ent("Bus", "Pages.Vehicle"), + ent("Truck", "Pages.Vehicle"), + ent("Car", "Pages.Vehicle"), + ent("SUV", "Pages.Vehicle"), + ent("Unrelated", ""), + }, + }}, + }, + } +} + +func templateWidget(specialization, childName string) *ast.WidgetV3 { + return &ast.WidgetV3{ + Type: "template", + Specialization: specialization, + Properties: map[string]any{}, + Children: []*ast.WidgetV3{{ + Type: "dynamictext", + Name: childName, + Properties: map[string]any{"Content": "x"}, + }}, + } +} + +func listViewWidget(children ...*ast.WidgetV3) *ast.WidgetV3 { + return &ast.WidgetV3{ + Type: "listview", + Name: "vehicleListView", + Properties: map[string]any{"DataSource": &ast.DataSourceV3{Type: "database", Reference: "Pages.Vehicle"}}, + Children: children, + } +} + +// TestBuildListViewTemplates pins the split: `template for` blocks become +// Templates, everything else stays the list view's own body, and source order is +// preserved because Mendix's order is authored rather than derived. +func TestBuildListViewTemplates(t *testing.T) { + pb := newVehiclePB() + lv, err := pb.buildListViewV3(listViewWidget( + &ast.WidgetV3{Type: "dynamictext", Name: "defaultVehicle", Properties: map[string]any{"Content": "v"}}, + templateWidget("Pages.Bus", "busLabel"), + templateWidget("Pages.Truck", "truckLabel"), + templateWidget("Pages.Car", "carLabel"), + templateWidget("Pages.SUV", "suvLabel"), + )) + if err != nil { + t.Fatalf("build failed: %v", err) + } + + if len(lv.Widgets) != 1 { + t.Errorf("list view body has %d widget(s), want 1 — templates must not land in Widgets", len(lv.Widgets)) + } + want := []string{"Pages.Bus", "Pages.Truck", "Pages.Car", "Pages.SUV"} + if len(lv.Templates) != len(want) { + t.Fatalf("got %d template(s), want %d", len(lv.Templates), len(want)) + } + for i, w := range want { + if got := lv.Templates[i].Specialization; got != w { + t.Errorf("template %d = %q, want %q (source order must be preserved)", i, got, w) + } + if lv.Templates[i].TypeName != "Forms$ListViewTemplate" { + t.Errorf("template %d TypeName = %q", i, lv.Templates[i].TypeName) + } + if len(lv.Templates[i].Widgets) != 1 { + t.Errorf("template %d has %d widget(s), want 1", i, len(lv.Templates[i].Widgets)) + } + } +} + +// TestBuildListViewTemplateRejections covers the three ways a template can be +// wrong. Each refuses rather than writing a document that cannot render: Mendix +// matches a template against the object's type, so a template for an unrelated +// entity is unreachable by construction. +func TestBuildListViewTemplateRejections(t *testing.T) { + cases := []struct { + name string + children []*ast.WidgetV3 + wantErr string + }{ + { + "entity is not a specialization of the list view's entity", + []*ast.WidgetV3{templateWidget("Pages.Unrelated", "x")}, + "is not Pages.Vehicle or a specialization of it", + }, + { + "two templates for one specialization", + []*ast.WidgetV3{templateWidget("Pages.Bus", "a"), templateWidget("Pages.Bus", "b")}, + "more than one template for Pages.Bus", + }, + { + "nested template", + func() []*ast.WidgetV3 { + outer := templateWidget("Pages.Bus", "a") + outer.Children = append(outer.Children, templateWidget("Pages.Truck", "b")) + return []*ast.WidgetV3{outer} + }(), + "cannot nest", + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + pb := newVehiclePB() + _, err := pb.buildListViewV3(listViewWidget(c.children...)) + if err == nil { + t.Fatalf("build succeeded; want an error containing %q", c.wantErr) + } + if !strings.Contains(err.Error(), c.wantErr) { + t.Errorf("error = %q, want it to contain %q", err.Error(), c.wantErr) + } + }) + } +} + +// TestBuildListViewTemplateOnTheListEntityItself is allowed: a template for the +// list view's own entity is the base case Mendix permits, and +// entityIsOrDescendsFrom returns true for the entity itself. +func TestBuildListViewTemplateOnTheListEntityItself(t *testing.T) { + pb := newVehiclePB() + lv, err := pb.buildListViewV3(listViewWidget(templateWidget("Pages.Vehicle", "base"))) + if err != nil { + t.Fatalf("a template for the list view's own entity was refused: %v", err) + } + if len(lv.Templates) != 1 { + t.Fatalf("got %d template(s), want 1", len(lv.Templates)) + } +} diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index 0d3e48584..6cf46fa4d 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -2158,13 +2158,19 @@ func (pb *pageBuilder) expandBuildingBlockRef(w *ast.WidgetV3) ([]*ast.WidgetV3, } // Apply optional rebind overrides. Binding-point rule (prototype): the - // override rewrites the FIRST widget in pre-order that carries a datasource / - // an action — the block's outermost datasource and primary action. + // override rewrites the FIRST widget in pre-order that CAN carry a datasource + // / an action — the block's outermost datasource and primary action. + // + // Can, not does. A reusable block is a template: its datasource is unbound + // and its buttons have no action, so neither property is present in the + // rendered MDL. The action override already matched by widget type for that + // reason; the datasource override matched on an existing DataSource property + // and only worked because an unbound datasource used to render as the + // malformed `DataSource: database from ,` — the very output #941 fixed. if ds, ok := w.Properties["DataSourceOverride"].(*ast.DataSourceV3); ok && ds != nil { // Datasource target: the first widget that already carries a datasource // (a block's outermost list/grid/dataview always emits one). - hit := rebindFirst(widgets, - func(t *ast.WidgetV3) bool { _, has := t.Properties["DataSource"]; return has }, + hit := rebindFirst(widgets, isDataSourceWidget, func(t *ast.WidgetV3) { t.Properties["DataSource"] = ds }) if !hit { return nil, mdlerrors.NewValidation(fmt.Sprintf( @@ -2186,6 +2192,18 @@ func (pb *pageBuilder) expandBuildingBlockRef(w *ast.WidgetV3) ([]*ast.WidgetV3, } // isButtonWidget reports whether a widget is an action-capable button. +// isDataSourceWidget reports whether a widget is one that takes a datasource, +// whether or not it currently carries one. The list is the data containers a +// building block can be built around; anything else in a block is layout or a +// leaf. +func isDataSourceWidget(w *ast.WidgetV3) bool { + switch strings.ToLower(w.Type) { + case "gallery", "listview", "datagrid", "datagrid2", "dataview", "templategrid", "referenceselector": + return true + } + return false +} + func isButtonWidget(w *ast.WidgetV3) bool { switch strings.ToLower(w.Type) { case "actionbutton", "linkbutton", "button": diff --git a/mdl/executor/cmd_pages_builder_v3_widgets.go b/mdl/executor/cmd_pages_builder_v3_widgets.go index 9a3d98e62..80eced6e4 100644 --- a/mdl/executor/cmd_pages_builder_v3_widgets.go +++ b/mdl/executor/cmd_pages_builder_v3_widgets.go @@ -274,12 +274,14 @@ func (pb *pageBuilder) buildListViewV3(w *ast.WidgetV3) (*pages.ListView, error) } // Handle DataSource + var listEntity string if ds := w.GetDataSource(); ds != nil { dataSource, entityName, err := pb.buildDataSourceV3(ds) if err != nil { return nil, mdlerrors.NewBackend("build datasource", err) } lv.DataSource = dataSource + listEntity = entityName // Save and restore entity context so nested containers work correctly oldContext := pb.entityContext @@ -297,18 +299,88 @@ func (pb *pageBuilder) buildListViewV3(w *ast.WidgetV3) (*pages.ListView, error) return nil, err } - // Build template widgets + // Split the body: `template for Module.Entity { ... }` blocks become + // specialization templates, everything else is the list view's own body (the + // default rendering, which Mendix uses for an object no template matches). + // The BSON keeps the two in separate arrays and so do we. + seen := make(map[string]bool, len(w.Children)) for _, child := range w.Children { - widget, err := pb.buildWidgetV3(child) + if child.Specialization == "" { + widget, err := pb.buildWidgetV3(child) + if err != nil { + return nil, err + } + lv.Widgets = append(lv.Widgets, widget) + continue + } + tpl, err := pb.buildListViewTemplateV3(child, w.Name, listEntity, seen) if err != nil { return nil, err } - lv.Widgets = append(lv.Widgets, widget) + lv.Templates = append(lv.Templates, tpl) } return lv, nil } +// buildListViewTemplateV3 builds one `template for Module.Entity { ... }` block. +// +// Source order is preserved: Mendix stores templates in an ordered array and the +// order is authored, not derived — the four templates on ako/TestApp's +// Vehicle_Overview are Bus, Truck, Car, SUV, which is neither alphabetical nor +// domain-model order. +func (pb *pageBuilder) buildListViewTemplateV3(w *ast.WidgetV3, listViewName, listEntity string, seen map[string]bool) (*pages.ListViewTemplate, error) { + spec := w.Specialization + + if seen[spec] { + return nil, mdlerrors.NewValidation(fmt.Sprintf( + "list view %s has more than one template for %s — a list view renders at most "+ + "one template per specialization", listViewName, spec)) + } + seen[spec] = true + + // The specialization must actually be one: Mendix matches a template against + // the object's type, so a template for an unrelated entity can never render. + // listEntity is empty when the datasource could not be resolved to an entity, + // and an unresolvable datasource is already reported elsewhere — do not + // report it a second time as a bogus specialization error. + if listEntity != "" && !pb.entityIsOrDescendsFrom(spec, listEntity) { + return nil, mdlerrors.NewValidation(fmt.Sprintf( + "template for %s in list view %s: %s is not %s or a specialization of it, "+ + "so the template can never match an object the list view shows", + spec, listViewName, spec, listEntity)) + } + + tpl := &pages.ListViewTemplate{ + BaseElement: model.BaseElement{ + ID: model.ID(types.GenerateID()), + TypeName: "Forms$ListViewTemplate", + }, + Specialization: spec, + } + + // Inside the template the context object IS the specialization, so an + // attribute the specialization adds resolves here even though it does not + // exist on the list view's own entity. + oldContext := pb.entityContext + pb.entityContext = spec + defer func() { pb.entityContext = oldContext }() + + for _, child := range w.Children { + if child.Specialization != "" { + return nil, mdlerrors.NewValidation(fmt.Sprintf( + "template for %s in list view %s contains a nested `template for %s` — "+ + "list view templates cannot nest", spec, listViewName, child.Specialization)) + } + widget, err := pb.buildWidgetV3(child) + if err != nil { + return nil, err + } + tpl.Widgets = append(tpl.Widgets, widget) + } + return tpl, nil +} + // applyOnChangeV3 resolves a widget's `OnChange:` client action into dst. // // Every input widget carrying an OnChangeAction must call this. Before ledger diff --git a/mdl/executor/cmd_pages_create_v3.go b/mdl/executor/cmd_pages_create_v3.go index bb3d2fb18..7a28e0c3d 100644 --- a/mdl/executor/cmd_pages_create_v3.go +++ b/mdl/executor/cmd_pages_create_v3.go @@ -48,6 +48,9 @@ func execCreatePageV3(ctx *ExecContext, s *ast.CreatePageStmtV3) error { // neither rewritten nor deleted here, and its exclusion is carried forward // when it is the only match (#914). existingExcluded := false + // Where the page being rewritten currently sits, so a statement that says + // nothing about folders leaves it there (#932). + var existingContainerID model.ID var excludedMatches []*pages.Page matches := 0 for _, p := range existingPages { @@ -67,6 +70,7 @@ func execCreatePageV3(ctx *ExecContext, s *ast.CreatePageStmtV3) error { if len(pagesToDelete) == 0 { existingAllowedRoles = cloneRoleIDs(p.AllowedRoles) preserveAllowedRoles = true + existingContainerID = p.ContainerID } pagesToDelete = append(pagesToDelete, p.ID) } @@ -78,6 +82,7 @@ func execCreatePageV3(ctx *ExecContext, s *ast.CreatePageStmtV3) error { existingAllowedRoles = cloneRoleIDs(p.AllowedRoles) preserveAllowedRoles = true existingExcluded = true + existingContainerID = p.ContainerID pagesToDelete = append(pagesToDelete, p.ID) } @@ -111,9 +116,15 @@ func execCreatePageV3(ctx *ExecContext, s *ast.CreatePageStmtV3) error { if len(pagesToDelete) > 0 { // Reuse first existing page's UUID to avoid git delete+add (which crashes Studio Pro RevStatusCache) page.ID = pagesToDelete[0] + if s.Folder == "" { + page.ContainerID = existingContainerID + } if err := ctx.Backend.UpdatePage(page); err != nil { return mdlerrors.NewBackend("update page", err) } + if _, err := applyDocumentFolder(ctx, page.ID, existingContainerID, page.ContainerID); err != nil { + return err + } // Delete any additional duplicates for _, id := range pagesToDelete[1:] { if err := ctx.Backend.DeletePage(id); err != nil { @@ -168,6 +179,12 @@ func execCreateSnippetV3(ctx *ExecContext, s *ast.CreateSnippetStmtV3) error { // Mendix allows, so it is neither replaced nor deleted, and an exclusion is // carried forward when every match is excluded (#914). existingExcluded := false + // Where the snippet being replaced currently sits. A snippet is rewritten + // as delete+create, so unlike the Update* doctypes its container really is + // re-applied on every statement — leaving this unset filed a foldered + // snippet back into the module root whenever a script that says nothing + // about folders was re-run (#932). + var existingContainerID model.ID var excludedSnippets []*pages.Snippet matches := 0 for _, snip := range existingSnippets { @@ -184,10 +201,14 @@ func execCreateSnippetV3(ctx *ExecContext, s *ast.CreateSnippetStmtV3) error { excludedSnippets = append(excludedSnippets, snip) continue } + if len(snippetsToDelete) == 0 { + existingContainerID = snip.ContainerID + } snippetsToDelete = append(snippetsToDelete, snip.ID) } if len(snippetsToDelete) == 0 && len(excludedSnippets) > 0 { existingExcluded = true + existingContainerID = excludedSnippets[0].ContainerID snippetsToDelete = append(snippetsToDelete, excludedSnippets[0].ID) } @@ -211,6 +232,9 @@ func execCreateSnippetV3(ctx *ExecContext, s *ast.CreateSnippetStmtV3) error { return mdlerrors.NewBackend("build snippet", err) } snippet.Excluded = snippet.Excluded || existingExcluded + if s.Folder == "" && existingContainerID != "" { + snippet.ContainerID = existingContainerID + } // Delete old snippets only after successful build for _, id := range snippetsToDelete { diff --git a/mdl/executor/cmd_pages_describe.go b/mdl/executor/cmd_pages_describe.go index 2ee9faf63..7132f06f6 100644 --- a/mdl/executor/cmd_pages_describe.go +++ b/mdl/executor/cmd_pages_describe.go @@ -506,6 +506,24 @@ type rawDataSource struct { XPathConstraint string // XPath constraint (WHERE clause) SortColumns []rawSortColumn // Multiple sort columns ContextVariable string // association source: context variable name (empty → $currentObject) + // Args carries a flow datasource's argument bindings, in stored order. A + // microflow used as a datasource needs an argument for every parameter, + // exactly as a call action does (#835) — describing it without them yields + // MDL that rebuilds into CE1571 (mxcli-formula1 §57.2). + Args []rawDataSourceArg + // Unsupported carries the stored $Type of a datasource that has no MDL + // spelling. Set instead of guessing a Type: describe reports it as a + // comment, so the binding is visible in the output without producing a + // statement that cannot be re-executed (#941). + Unsupported string +} + +// rawDataSourceArg is one argument bound to a flow datasource's parameter. +// Name is the bare parameter name; storage qualifies it with the flow it +// belongs to, which MDL does not repeat. +type rawDataSourceArg struct { + Name string + Value string } // associationSourcePath reconstructs the association navigation of a @@ -615,6 +633,11 @@ type rawWidget struct { // TabControl parsing — preserves the original tab page name/caption so // DESCRIBE output shows which tab each nested widget belongs to). TabCaption string + // Specialization is the entity a List View template renders. Set only on the + // synthetic wrappers parseListViewContent emits for Forms$ListViewTemplate, + // which is the same shape as TabCaption above: a container with no name, whose + // identity is a single field DESCRIBE has to put back. + Specialization string // Conditional visibility/editability VisibleIf string // Expression from ConditionalVisibilitySettings EditableIf string // Expression from ConditionalEditabilitySettings diff --git a/mdl/executor/cmd_pages_describe_datasource.go b/mdl/executor/cmd_pages_describe_datasource.go new file mode 100644 index 000000000..2eb5c0c8e --- /dev/null +++ b/mdl/executor/cmd_pages_describe_datasource.go @@ -0,0 +1,384 @@ +// SPDX-License-Identifier: Apache-2.0 + +// cmd_pages_describe_datasource.go — reading a widget's datasource out of page +// BSON, and rendering it back as MDL. +// +// There used to be five copies of the read switch and six of the write switch, +// one per widget family, and they disagreed about which datasource `$Type`s +// exist. The pluggable copy — the one every Gallery, DataGrid2 and chart goes +// through — knew about microflows and nanoflows but not associations, selection +// targets or context sources, so a Gallery bound over an association described +// back with no datasource at all while a ListView bound the same way kept it +// (#941). The write side had the mirror-image defect: the object-list item +// emitter had no type switch, so a chart series bound to a microflow described +// as `database from Module.TheMicroflow` and re-executing it reported +// "entity not found". +// +// A datasource is the same thing whatever contains it, so it is read and +// rendered in one place. Adding a widget family does not mean copying a switch, +// and adding a `$Type` means one case rather than five. +package executor + +import ( + "fmt" + "github.com/mendixlabs/mxcli/mdl/visitor" + "strings" +) + +// Datasource storage names. The set is the metamodel's `DataSource` interface +// (generated/metamodel/types.go): AssociationSource, CustomWidgetXPathSource, +// DataViewSource, GridXPathSource, ImageViewerSource, ListViewXPathSource, +// ListenTargetSource, MicroflowSource, NanoflowSource, ReferenceSetSource — +// plus the CustomWidgets-namespaced nanoflow source seen in stored documents. +const ( + dsTypeMicroflow = "Forms$MicroflowSource" + dsTypeNanoflow = "Forms$NanoflowSource" + dsTypeCustomNanoflow = "CustomWidgets$CustomWidgetNanoflowSource" + dsTypeDatabase = "Forms$DatabaseSource" + dsTypeCustomWidgetXPath = "CustomWidgets$CustomWidgetXPathSource" + dsTypeListViewXPath = "Forms$ListViewXPathSource" + dsTypeGridXPath = "Forms$GridXPathSource" + dsTypeReferenceSet = "Forms$ReferenceSetSource" + dsTypeImageViewer = "Forms$ImageViewerSource" + dsTypeAssociation = "Forms$AssociationSource" + dsTypeDataView = "Forms$DataViewSource" + dsTypeEntityPath = "Forms$EntityPathSource" + dsTypeListenTarget = "Forms$ListenTargetSource" +) + +// entityBackedSourceTypes are the datasources that name an entity plus an +// optional XPath constraint and sort. They differ only in which container +// declares them, which is why they share one reader. +var entityBackedSourceTypes = map[string]bool{ + dsTypeDatabase: true, + dsTypeCustomWidgetXPath: true, + dsTypeListViewXPath: true, + dsTypeGridXPath: true, + dsTypeReferenceSet: true, + dsTypeImageViewer: true, +} + +// parseDataSource reads any widget datasource. Returns nil when ds is not a +// datasource at all; a non-nil result with Unsupported set means the datasource +// is real but has no MDL spelling, which the emitter reports as a comment +// rather than dropping (silent loss) or rendering half of it (a parse error). +func parseDataSource(ds map[string]any) *rawDataSource { + if ds == nil { + return nil + } + dsType := extractString(ds["$Type"]) + if dsType == "" { + return nil + } + + // Known-but-empty and unknown are different facts and get different + // answers. A known type whose payload is empty (a listen target with no + // target) describes a datasource the model itself has not finished — there + // is nothing to reproduce and nothing useful to say, so it yields nil, as it + // always has. A type this build has never heard of is mxcli's gap, not the + // model's, and is reported rather than guessed at or silently dropped. + known := true + switch { + case dsType == dsTypeMicroflow: + if mf := microflowSourceRef(ds); mf != "" { + return &rawDataSource{Type: "microflow", Reference: mf, Args: flowSourceArgs(ds, "MicroflowSettings", mf)} + } + case dsType == dsTypeNanoflow: + if nf := nanoflowSourceRef(ds); nf != "" { + return &rawDataSource{Type: "nanoflow", Reference: nf, Args: flowSourceArgs(ds, "NanoflowSettings", nf)} + } + case dsType == dsTypeCustomNanoflow: + if nf := extractString(ds["Nanoflow"]); nf != "" { + return &rawDataSource{Type: "nanoflow", Reference: nf} + } + case dsType == dsTypeListenTarget: + if target := extractString(ds["ListenTarget"]); target != "" { + return &rawDataSource{Type: "selection", Reference: target} + } + case dsType == dsTypeAssociation: + if path, ctxVar := associationSourcePath(ds); path != "" { + return &rawDataSource{Type: "association", Reference: path, ContextVariable: ctxVar} + } + case dsType == dsTypeDataView || dsType == dsTypeEntityPath: + if got := parseContextSource(ds); got != nil { + return got + } + case entityBackedSourceTypes[dsType]: + if got := parseEntitySource(ds); got != nil { + return got + } + // An entity-backed source with no entity is really a context source: + // Studio Pro stores a pluggable list bound over an association or to a + // page parameter as an XPath source whose EntityRef is empty and whose + // navigation lives in Steps / SourceVariable / EntityPath. Reading only + // EntityRef produced `database from ` with an empty slot (#941). + if got := parseContextSource(ds); got != nil { + return got + } + default: + known = false + } + + if known { + return nil + } + // A datasource type this build does not know. Reported, not guessed at: an + // invented `database from` is a statement that cannot be re-executed, and + // dropping it loses the binding without saying so. + return &rawDataSource{Unsupported: dsType} +} + +// parseEntitySource reads the entity, XPath and sort of an entity-backed +// datasource, or nil when it names no entity. +func parseEntitySource(ds map[string]any) *rawDataSource { + entityRef, ok := ds["EntityRef"].(map[string]any) + if !ok || entityRef == nil { + return nil + } + entity := extractString(entityRef["Entity"]) + if entity == "" { + return nil + } + result := &rawDataSource{ + Type: "database", + Reference: entity, + XPathConstraint: extractString(ds["XPathConstraint"]), + SortColumns: parseSortColumns(ds), + } + return result +} + +// parseContextSource reads the "data from context" forms: over an association, +// or straight from a page parameter. +func parseContextSource(ds map[string]any) *rawDataSource { + if path, ctxVar := associationSourcePath(ds); path != "" { + return &rawDataSource{Type: "association", Reference: path, ContextVariable: ctxVar} + } + if sv, ok := ds["SourceVariable"].(map[string]any); ok && sv != nil { + if param := extractString(sv["PageParameter"]); param != "" { + return &rawDataSource{Type: "parameter", Reference: param} + } + } + if entityPath := extractString(ds["EntityPath"]); entityPath != "" { + return &rawDataSource{Type: "parameter", Reference: entityPath} + } + return nil +} + +// parseSortColumns reads a datasource's sort, accepting both stored shapes: +// a GridSortBar (`SortBar.SortItems`, used by grid-like widgets) and a +// ListViewSort (`Sort.Paths`). Which one a datasource carries depends on its +// type, so reading whichever is present keeps this reader container-agnostic. +func parseSortColumns(ds map[string]any) []rawSortColumn { + items := []any(nil) + if sortBar, ok := ds["SortBar"].(map[string]any); ok && sortBar != nil { + items = getBsonArrayElements(sortBar["SortItems"]) + } + if len(items) == 0 { + if sortObj, ok := ds["Sort"].(map[string]any); ok && sortObj != nil { + items = getBsonArrayElements(sortObj["Paths"]) + } + } + var cols []rawSortColumn + for _, item := range items { + sortItem, ok := item.(map[string]any) + if !ok { + continue + } + col := rawSortColumn{Order: "asc"} + if attrRef, ok := sortItem["AttributeRef"].(map[string]any); ok { + col.Attribute = shortAttributeName(extractString(attrRef["Attribute"])) + } + if gridSortDirection(sortItem) == "Descending" { + col.Order = "desc" + } + if col.Attribute != "" { + cols = append(cols, col) + } + } + return cols +} + +// dataSourceExpr renders a datasource as the MDL that reproduces it — the part +// after `DataSource: `. Returns "" when there is nothing to emit. +// +// One renderer for every widget, because a datasource means the same thing +// wherever it appears. The six hand-written switches this replaces disagreed: +// the object-list one had no switch at all and labelled everything `database`, +// and two others omitted the WHERE and SORT they had read (#941). +func dataSourceExpr(ds *rawDataSource) string { + if ds == nil || ds.Unsupported != "" { + return "" + } + switch ds.Type { + case "database": + // Never render a database source with no entity: `database from ` parses + // `from` as the entity name and reports "entity not found: from". + if ds.Reference == "" { + return "" + } + expr := "database from " + ds.Reference + if clause := xpathConstraintClause(ds.XPathConstraint); clause != "" { + expr += " where " + clause + } + if len(ds.SortColumns) > 0 { + parts := make([]string, 0, len(ds.SortColumns)) + for _, col := range ds.SortColumns { + parts = append(parts, col.Attribute+" "+col.Order) + } + expr += " sort by " + strings.Join(parts, ", ") + } + return expr + case "microflow", "nanoflow": + if ds.Reference == "" { + return "" + } + expr := ds.Type + " " + ds.Reference + if len(ds.Args) > 0 { + parts := make([]string, 0, len(ds.Args)) + for _, arg := range ds.Args { + parts = append(parts, arg.Name+": "+arg.Value) + } + expr += "(" + strings.Join(parts, ", ") + ")" + } + return expr + case "parameter": + if ds.Reference == "" { + return "" + } + // A page parameter is stored bare ("Account") while an entity path + // arrives already written as an expression ("$Account/Mod.Assoc"), and + // both reach here as Type "parameter". Prefixing only the bare form + // keeps each spelling as MDL writes it. + if strings.HasPrefix(ds.Reference, "$") { + return ds.Reference + } + return "$" + ds.Reference + case "selection": + if ds.Reference == "" { + return "" + } + return "selection " + mdlIdent(ds.Reference) + case "association": + if ds.Reference == "" { + return "" + } + return associationDataSourceExpr(ds) + } + return "" +} + +// dataSourceProp renders the whole `DataSource: …` property, or "" when the +// datasource cannot be expressed. +func dataSourceProp(ds *rawDataSource) string { + expr := dataSourceExpr(ds) + if expr == "" { + return "" + } + return "DataSource: " + expr +} + +// dataSourceComment describes a datasource MDL cannot express, so a reader of +// the output learns the binding exists rather than silently losing it. Returns +// "" for a datasource that renders normally. +func dataSourceComment(ds *rawDataSource) string { + if ds == nil { + return "" + } + if ds.Unsupported != "" { + return fmt.Sprintf("-- DataSource (%s) has no MDL spelling and is not reproduced here", ds.Unsupported) + } + if dataSourceExpr(ds) == "" && ds.Type != "" { + return fmt.Sprintf("-- DataSource (%s) is incomplete in the model and is not reproduced here", ds.Type) + } + return "" +} + +// appendDataSourceProp adds a widget's `DataSource: …` property, or — when the +// datasource cannot be spelled in MDL — a comment saying so. +// +// Every widget goes through this, so a datasource cannot be rendered one way in +// a DataView and another in a Gallery, which is the drift #941 was. +func appendDataSourceProp(props []string, ds *rawDataSource) []string { + if prop := dataSourceProp(ds); prop != "" { + return append(props, prop) + } + if comment := dataSourceComment(ds); comment != "" { + return append(props, comment) + } + return props +} + +// xpathConstraintClause renders a stored XPath constraint as the MDL the page +// grammar accepts after WHERE, or "" when there is no constraint. +// +// Always the bracketed `xpathConstraint` production, never a bare expression. +// The grammar takes either, and the bare form reads more nicely — but it only +// parses for simple comparisons, and whether a given XPath happens to also be a +// valid MDL expression is not a question this emitter can answer. Emitting it +// raw is what turned stock Administration.Account_New from describing cleanly +// into describing to MDL that does not parse: its constraint carries an inner +// predicate (`System.grantableRoles[reversed()]/…`), and the `[` ends the +// expression parse (mxcli-formula1 §57.1). A bracketed group is accepted for +// every constraint, so the emitter does not have to be right about which is +// which. +// +// Splitting goes through the same quote- and nesting-aware helper the microflow +// emitter uses (#772) rather than a second copy: the previous code here took +// the outer brackets off by testing the first and last byte, which turns +// `[a][b]` into the mangled `a][b`. +func xpathConstraintClause(constraint string) string { + xpath := strings.TrimSpace(constraint) + if xpath == "" { + return "" + } + if groups := visitor.SplitXPathPredicateGroups(xpath); len(groups) > 0 { + return strings.Join(groups, " ") + } + return "[" + xpath + "]" +} + +// flowSourceArgs reads the argument bindings of a microflow or nanoflow +// datasource from its settings sub-document. +// +// Storage qualifies each parameter with the flow it belongs to +// ("Mod.DS_Filtered.Term"), while MDL names the parameter alone — the flow is +// already on the left of the parentheses. The prefix is stripped by matching +// the flow's own qualified name rather than by cutting at the last dot, so a +// document that stores the name bare is left alone instead of losing its first +// segment. +// +// The bound value lives in Expression for both a variable reference ($Term) and +// a literal (10); Variable is the older spelling and is honoured when present. +// A parameterless flow yields nil, which the renderer emits without parentheses +// — the grammar makes the list optional, and adding empty parens would churn +// every existing description. +func flowSourceArgs(ds map[string]any, settingsKey, flowName string) []rawDataSourceArg { + settings, ok := ds[settingsKey].(map[string]any) + if !ok || settings == nil { + return nil + } + var out []rawDataSourceArg + for _, item := range getBsonArrayElements(settings["ParameterMappings"]) { + mapping, ok := item.(map[string]any) + if !ok || mapping == nil { + continue + } + name := strings.TrimPrefix(extractString(mapping["Parameter"]), flowName+".") + if name == "" { + continue + } + value := extractString(mapping["Expression"]) + if value == "" { + value = extractString(mapping["Variable"]) + } + if value == "" { + // A parameter with no bound value is what CE1571 is about. There is + // nothing to reproduce, and inventing one would be worse than + // leaving the gap the model already has. + continue + } + out = append(out, rawDataSourceArg{Name: name, Value: value}) + } + return out +} diff --git a/mdl/executor/cmd_pages_describe_datasource_test.go b/mdl/executor/cmd_pages_describe_datasource_test.go new file mode 100644 index 000000000..e6ba969d1 --- /dev/null +++ b/mdl/executor/cmd_pages_describe_datasource_test.go @@ -0,0 +1,373 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" +) + +// dsBSON builds a datasource map in the shape page BSON stores. +func dsBSON(dsType string, kv ...any) map[string]any { + m := map[string]any{"$Type": dsType} + for i := 0; i+1 < len(kv); i += 2 { + m[kv[i].(string)] = kv[i+1] + } + return m +} + +// TestParseDataSourceCoversEveryStoredType is the read half of #941. The +// pluggable copy of this switch knew microflows and nanoflows but not +// associations, selection targets or context sources — so a Gallery bound over +// an association described back with no datasource, while a ListView bound the +// same way kept it. +func TestParseDataSourceCoversEveryStoredType(t *testing.T) { + cases := []struct { + name string + ds map[string]any + wantTyp string + wantRef string + }{ + { + name: "microflow", + ds: dsBSON(dsTypeMicroflow, "MicroflowSettings", map[string]any{"Microflow": "Mod.DS_GetItems"}), + wantTyp: "microflow", wantRef: "Mod.DS_GetItems", + }, + { + name: "association — dropped entirely by the pluggable reader", + ds: dsBSON(dsTypeAssociation, "EntityRef", map[string]any{"Steps": []any{map[string]any{"Association": "Mod.Item_Bucket"}}}), + wantTyp: "association", wantRef: "Mod.Item_Bucket", + }, + { + name: "selection — dropped entirely by the pluggable reader", + ds: dsBSON(dsTypeListenTarget, "ListenTarget", "gallery1"), + wantTyp: "selection", wantRef: "gallery1", + }, + { + name: "page parameter", + ds: dsBSON(dsTypeDataView, "SourceVariable", map[string]any{"PageParameter": "Bucket"}), + wantTyp: "parameter", wantRef: "Bucket", + }, + { + name: "database via EntityRef", + ds: dsBSON(dsTypeDatabase, "EntityRef", map[string]any{"Entity": "Mod.Item"}), + wantTyp: "database", wantRef: "Mod.Item", + }, + { + name: "list view xpath source", + ds: dsBSON(dsTypeListViewXPath, "EntityRef", map[string]any{"Entity": "Mod.Item"}), + wantTyp: "database", wantRef: "Mod.Item", + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := parseDataSource(tc.ds) + if got == nil { + t.Fatalf("datasource was dropped") + } + if got.Unsupported != "" { + t.Fatalf("reported unsupported (%s)", got.Unsupported) + } + if got.Type != tc.wantTyp || got.Reference != tc.wantRef { + t.Errorf("got Type=%q Reference=%q, want %q / %q", got.Type, got.Reference, tc.wantTyp, tc.wantRef) + } + }) + } +} + +// TestParseDataSourceReadsAContextScopedXPathSource is symptom 1 of #941. Studio +// Pro stores a pluggable list bound over an association as an XPath source whose +// EntityRef is empty; reading only EntityRef yielded a database source with no +// entity, which the emitter rendered as `database from ` — and `mxcli check` +// then read `from` as the entity name. +func TestParseDataSourceReadsAContextScopedXPathSource(t *testing.T) { + ds := dsBSON(dsTypeCustomWidgetXPath, + "EntityRef", map[string]any{"Steps": []any{map[string]any{"Association": "Mod.Item_Bucket"}}}, + ) + got := parseDataSource(ds) + if got == nil { + t.Fatal("datasource was dropped") + } + if got.Type == "database" && got.Reference == "" { + t.Fatalf("read as a database source with no entity — emits %q", "database from "+got.Reference) + } + if got.Type != "association" || got.Reference != "Mod.Item_Bucket" { + t.Errorf("got Type=%q Reference=%q, want association / Mod.Item_Bucket", got.Type, got.Reference) + } +} + +// TestDataSourceExprNeverEmitsAnEmptyEntitySlot is the guard for the exact +// string in the report. Whatever a datasource turns out to be, describe must not +// produce `database from ` with nothing after it: that is not a near-miss, it is +// a statement the parser reads as an entity called "from". +func TestDataSourceExprNeverEmitsAnEmptyEntitySlot(t *testing.T) { + for _, ds := range []*rawDataSource{ + {Type: "database", Reference: ""}, + {Type: "microflow", Reference: ""}, + {Type: "parameter", Reference: ""}, + {Type: "selection", Reference: ""}, + {Type: "association", Reference: ""}, + {Unsupported: "Forms$SomethingNew"}, + } { + if expr := dataSourceExpr(ds); expr != "" { + t.Errorf("%+v rendered as %q, want nothing", ds, expr) + } + } +} + +// TestDataSourceExprKeepsTheType is symptom 2 of #941: the object-list emitter +// had no type switch at all, so a chart series bound to a microflow described as +// `database from Module.TheMicroflow` and re-executing it reported +// "entity not found". +func TestDataSourceExprKeepsTheType(t *testing.T) { + cases := map[string]struct { + ds *rawDataSource + want string + }{ + "microflow": {&rawDataSource{Type: "microflow", Reference: "Mod.DS_GetItems"}, "microflow Mod.DS_GetItems"}, + "nanoflow": {&rawDataSource{Type: "nanoflow", Reference: "Mod.NF_GetItems"}, "nanoflow Mod.NF_GetItems"}, + "database": {&rawDataSource{Type: "database", Reference: "Mod.Item"}, "database from Mod.Item"}, + "selection": {&rawDataSource{Type: "selection", Reference: "gallery1"}, "selection gallery1"}, + "parameter": {&rawDataSource{Type: "parameter", Reference: "Bucket"}, "$Bucket"}, + "parameter already an expression": { + &rawDataSource{Type: "parameter", Reference: "$Account/Mod.Assoc"}, "$Account/Mod.Assoc", + }, + "association": { + &rawDataSource{Type: "association", Reference: "Mod.Item_Bucket"}, "$currentObject/Mod.Item_Bucket", + }, + // Bracketed, not bare: the emitter always uses the grammar's + // xpathConstraint production, because it cannot tell which constraints + // also happen to be valid MDL expressions (mxcli-formula1 §57.1). + "database with where and sort": { + &rawDataSource{ + Type: "database", Reference: "Mod.Item", + XPathConstraint: "[IsActive = true]", + SortColumns: []rawSortColumn{{Attribute: "Name", Order: "asc"}}, + }, + "database from Mod.Item where [IsActive = true] sort by Name asc", + }, + } + for name, tc := range cases { + t.Run(name, func(t *testing.T) { + if got := dataSourceExpr(tc.ds); got != tc.want { + t.Errorf("got %q, want %q", got, tc.want) + } + }) + } +} + +// TestParseDataSourceReportsWhatItCannotSpell pins the third option. A +// datasource mxcli has no MDL word for is neither dropped (the binding +// disappears with no trace) nor rendered as a guess (a statement that will not +// re-execute) — it comes back as a comment naming the stored type. +func TestParseDataSourceReportsWhatItCannotSpell(t *testing.T) { + got := parseDataSource(dsBSON("Forms$SomeFutureSource", "Whatever", "x")) + if got == nil { + t.Fatal("an unknown datasource was dropped silently") + } + if got.Unsupported != "Forms$SomeFutureSource" { + t.Errorf("Unsupported = %q, want the stored $Type", got.Unsupported) + } + comment := dataSourceComment(got) + if !strings.Contains(comment, "Forms$SomeFutureSource") { + t.Errorf("comment %q does not name the stored type", comment) + } + if dataSourceExpr(got) != "" { + t.Error("an unspellable datasource still rendered an expression") + } +} + +// TestParseSortColumnsAcceptsBothStoredShapes pins that sorting is read +// wherever it lives. Grid-like datasources carry a GridSortBar and list views a +// Sort.Paths; a reader that knew only one silently dropped the other's sort. +func TestParseSortColumnsAcceptsBothStoredShapes(t *testing.T) { + sortItem := func(attr, dir string) map[string]any { + return map[string]any{ + "AttributeRef": map[string]any{"Attribute": attr}, + "SortDirection": dir, + } + } + sortBar := parseSortColumns(map[string]any{ + "SortBar": map[string]any{"SortItems": []any{sortItem("Mod.Item.Name", "Ascending")}}, + }) + paths := parseSortColumns(map[string]any{ + "Sort": map[string]any{"Paths": []any{sortItem("Mod.Item.Name", "Descending")}}, + }) + if len(sortBar) != 1 || sortBar[0].Attribute != "Name" || sortBar[0].Order != "asc" { + t.Errorf("SortBar shape gave %+v", sortBar) + } + if len(paths) != 1 || paths[0].Attribute != "Name" || paths[0].Order != "desc" { + t.Errorf("Sort.Paths shape gave %+v", paths) + } +} + +// TestXPathConstraintClauseParses covers the regression in mxcli-formula1 §57.1. +// +// Restoring the WHERE clause (defect 4 of #941) collided with an older gap: +// the constraint was emitted raw, so a stored XPath containing a predicate — +// `System.grantableRoles[reversed()]/…`, which stock Administration.Account_New +// carries — produced MDL that no longer parsed. The page describing clean +// BEFORE the fix and failing after is the worst shape a fix can have. +// +// The emitted form is always the grammar's own `xpathConstraint` production +// (bracketed), never a bare expression. The bare form parses only for simple +// comparisons, and "does this particular XPath happen to be a valid MDL +// expression" is not a question the emitter can answer — while a bracketed +// group is accepted for every constraint. +func TestXPathConstraintClauseParses(t *testing.T) { + cases := map[string]string{ + "simple comparison": "Qty > 5", + "already bracketed": "[Qty > 5]", + "inner predicate": "[System.grantableRoles[reversed()]/System.UserRole/System.UserRoles = '[%CurrentUser%]']", + "two groups": "[Active = true][Qty > 5]", + "bracket in literal": "[Name = 'a]b']", + } + for name, stored := range cases { + t.Run(name, func(t *testing.T) { + got := xpathConstraintClause(stored) + if got == "" { + t.Fatalf("rendered nothing for %q", stored) + } + if !strings.HasPrefix(got, "[") || !strings.HasSuffix(got, "]") { + t.Errorf("rendered %q — want bracketed groups the page grammar accepts", got) + } + }) + } +} + +// TestXPathConstraintClauseKeepsGroupsIntact pins the trap the old code fell +// into: it stripped the outer brackets by looking at the first and last byte, +// so `[a][b]` became `a][b`. That is the same mangling #772 fixed for the +// microflow emitter, which is why this reuses that fix's splitter rather than +// keeping a second, simpler copy of the logic. +func TestXPathConstraintClauseKeepsGroupsIntact(t *testing.T) { + got := xpathConstraintClause("[Active = true][Qty > 5]") + if strings.Contains(got, "a][b") || strings.Count(got, "[") != 2 { + t.Errorf("rendered %q — want both predicate groups preserved", got) + } +} + +// TestXPathConstraintClauseIgnoresBlank guards mxcli-formula1 §57.3: the old +// guard was `strings.Trim(s, "")`, whose empty cutset trims nothing, so a +// whitespace-only constraint passed the != "" test and emitted a bare `where`. +func TestXPathConstraintClauseIgnoresBlank(t *testing.T) { + for _, blank := range []string{"", " ", "\t\n"} { + if got := xpathConstraintClause(blank); got != "" { + t.Errorf("xpathConstraintClause(%q) = %q, want empty", blank, got) + } + } +} + +// TestDataSourceArgsRoundTrip covers mxcli-formula1 §57.2. +// +// A microflow used as a widget datasource needs an argument for every +// parameter, exactly as a call action does — #835 fixed the write side. DESCRIBE +// never read them back, so `microflow Mod.DS(Race: $Race)` described as +// `microflow Mod.DS`, and replaying that description produced +// +// [error] [CE1571] "No argument has been selected for parameter 'Race' …" +// +// which is the very error #835 fixed. The loss predates #941; the old code +// printed the whole line wrong, so it hid behind a larger bug. +func TestDataSourceArgsRoundTrip(t *testing.T) { + ds := map[string]any{ + "$Type": "Forms$MicroflowSource", + "MicroflowSettings": map[string]any{ + "Microflow": "Mod.DS_Filtered", + // Stored qualified — . — and in + // order. Verified against a real unit rather than inferred. + "ParameterMappings": []any{ + int32(3), + map[string]any{"Parameter": "Mod.DS_Filtered.Term", "Expression": "$Term"}, + map[string]any{"Parameter": "Mod.DS_Filtered.Limit", "Expression": "10"}, + }, + }, + } + got := parseDataSource(ds) + if got == nil { + t.Fatal("datasource not read at all") + } + if len(got.Args) != 2 { + t.Fatalf("read %d arguments, want 2 (%+v)", len(got.Args), got.Args) + } + // The qualified prefix is storage's, not MDL's: the clause names the + // parameter, not the microflow it belongs to. + if got.Args[0].Name != "Term" || got.Args[0].Value != "$Term" { + t.Errorf("first arg = %+v, want {Term $Term}", got.Args[0]) + } + if got.Args[1].Name != "Limit" || got.Args[1].Value != "10" { + t.Errorf("second arg = %+v, want {Limit 10}", got.Args[1]) + } + + want := "microflow Mod.DS_Filtered(Term: $Term, Limit: 10)" + if expr := dataSourceExpr(got); expr != want { + t.Errorf("rendered %q, want %q", expr, want) + } +} + +// TestDataSourceArgsOmittedWhenThereAreNone pins that a parameterless microflow +// keeps its bare form. The grammar makes the argument list optional, and +// emitting `microflow Mod.DS()` for a flow that takes nothing would be noise at +// best — and is not what the previous output looked like, so every existing +// description would churn. +func TestDataSourceArgsOmittedWhenThereAreNone(t *testing.T) { + for name, ds := range map[string]map[string]any{ + "no settings": {"$Type": "Forms$MicroflowSource", "Microflow": "Mod.DS"}, + "empty list": {"$Type": "Forms$MicroflowSource", "MicroflowSettings": map[string]any{"Microflow": "Mod.DS", "ParameterMappings": []any{int32(3)}}}, + "nil mappings": {"$Type": "Forms$MicroflowSource", "MicroflowSettings": map[string]any{"Microflow": "Mod.DS", "ParameterMappings": nil}}, + } { + t.Run(name, func(t *testing.T) { + got := parseDataSource(ds) + if got == nil { + t.Fatal("datasource not read") + } + if expr := dataSourceExpr(got); expr != "microflow Mod.DS" { + t.Errorf("rendered %q, want the bare form", expr) + } + }) + } +} + +// TestDataSourceArgsNanoflow pins the sibling shape: a nanoflow source stores +// its arguments under NanoflowSettings, and losing them there fails the build +// the same way. +func TestDataSourceArgsNanoflow(t *testing.T) { + ds := map[string]any{ + "$Type": "Forms$NanoflowSource", + "NanoflowSettings": map[string]any{ + "Nanoflow": "Mod.NF_Rows", + "ParameterMappings": []any{ + int32(3), + map[string]any{"Parameter": "Mod.NF_Rows.Ctx", "Expression": "$currentObject"}, + }, + }, + } + got := parseDataSource(ds) + if got == nil { + t.Fatal("nanoflow datasource not read") + } + want := "nanoflow Mod.NF_Rows(Ctx: $currentObject)" + if expr := dataSourceExpr(got); expr != want { + t.Errorf("rendered %q, want %q", expr, want) + } +} + +// TestDataSourceArgsUnqualifiedParameter guards the prefix strip. The stored +// name is qualified by the flow it belongs to, but a document that stores it +// bare must not have its first character eaten by a blind prefix trim. +func TestDataSourceArgsUnqualifiedParameter(t *testing.T) { + ds := map[string]any{ + "$Type": "Forms$MicroflowSource", + "MicroflowSettings": map[string]any{ + "Microflow": "Mod.DS", + "ParameterMappings": []any{int32(3), map[string]any{"Parameter": "Term", "Expression": "$Term"}}, + }, + } + got := parseDataSource(ds) + if got == nil || len(got.Args) != 1 { + t.Fatalf("read %+v", got) + } + if got.Args[0].Name != "Term" { + t.Errorf("parameter name = %q, want Term", got.Args[0].Name) + } +} diff --git a/mdl/executor/cmd_pages_describe_listview_templates_test.go b/mdl/executor/cmd_pages_describe_listview_templates_test.go new file mode 100644 index 000000000..d367d626c --- /dev/null +++ b/mdl/executor/cmd_pages_describe_listview_templates_test.go @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" +) + +// rawListViewWithTemplates mirrors ako/TestApp's Vehicle_Overview: a list view +// over Pages.Vehicle with its own default body plus four specialization +// templates, in the order Studio Pro stored them. +func rawListViewWithTemplates() map[string]any { + tmpl := func(entity, name string) map[string]any { + return map[string]any{ + "$Type": "Forms$ListViewTemplate", + "Entity": entity, + "Widgets": []any{map[string]any{"$Type": "Forms$TextBox", "Name": name}}, + } + } + return map[string]any{ + "$Type": "Forms$ListView", + "Name": "vehicleListView", + "DataSource": map[string]any{ + "$Type": "Forms$ListViewXPathSource", + "EntityRef": map[string]any{ + "$Type": "DomainModels$DirectEntityRef", + "Entity": "Pages.Vehicle", + }, + }, + "Widgets": []any{map[string]any{"$Type": "Forms$TextBox", "Name": "defaultVehicle"}}, + "Templates": []any{ + tmpl("Pages.Bus", "busLabel"), + tmpl("Pages.Truck", "truckLabel"), + tmpl("Pages.Car", "carLabel"), + tmpl("Pages.SUV", "suvLabel"), + }, + } +} + +// TestDescribeRendersListViewTemplates is the regression test for #940. +// +// parseListViewContent read only the Widgets array, so a list view's +// specialization templates were dropped from DESCRIBE with no warning — and +// SEARCH inherited the gap, because the catalog's source table is built from +// DESCRIBE output. Worse, DESCRIBE emits `create or modify page`, so +// re-executing its output rebuilt the page without them: measured on +// ako/TestApp, a describe → exec round trip took the page from 4 templates to 0, +// with `mx check` reporting 0 errors either way. +func TestDescribeRendersListViewTemplates(t *testing.T) { + ctx, buf := newMockCtx(t) + + for _, w := range parseRawWidget(ctx, rawListViewWithTemplates(), "Pages.Vehicle") { + outputWidgetMDLV3(ctx, w, 0) + } + out := buf.String() + + // Every template, identified by the entity it renders. + for _, want := range []string{ + "template for Pages.Bus {", + "template for Pages.Truck {", + "template for Pages.Car {", + "template for Pages.SUV {", + } { + if !strings.Contains(out, want) { + t.Errorf("missing %q from DESCRIBE output:\n%s", want, out) + } + } + + // Contents, which is what SHOW REFERENCES and SEARCH ultimately index. + for _, want := range []string{"busLabel", "truckLabel", "carLabel", "suvLabel", "defaultVehicle"} { + if !strings.Contains(out, want) { + t.Errorf("missing widget %q from DESCRIBE output:\n%s", want, out) + } + } + + // Order is authored, not derived — TestApp's is Bus, Truck, Car, SUV, which + // is neither alphabetical nor domain-model order, so a describe that sorted + // them would not round-trip to the same document. + idx := func(s string) int { return strings.Index(out, s) } + if !(idx("Pages.Bus") < idx("Pages.Truck") && idx("Pages.Truck") < idx("Pages.Car") && idx("Pages.Car") < idx("Pages.SUV")) { + t.Errorf("templates are not in stored order:\n%s", out) + } + + // The default body is the list view's own Widgets array, NOT a template, and + // must not be wrapped in one. + if strings.Contains(out, "template for Pages.Vehicle") { + t.Errorf("the list view's own body was rendered as a template:\n%s", out) + } +} + +// TestDescribeListViewTemplateEntityContext pins that a template's body is +// parsed in the specialization's context, not the list view's. An attribute the +// specialization adds does not exist on the list view's entity, so parsing the +// children against the parent entity resolves the wrong thing (or nothing). +func TestDescribeListViewTemplateEntityContext(t *testing.T) { + ctx, _ := newMockCtx(t) + + widgets := parseRawWidget(ctx, rawListViewWithTemplates(), "Pages.Vehicle") + if len(widgets) != 1 { + t.Fatalf("expected 1 list view, got %d", len(widgets)) + } + var templates []rawWidget + for _, c := range widgets[0].Children { + if c.Type == "Forms$ListViewTemplate" { + templates = append(templates, c) + } + } + if len(templates) != 4 { + t.Fatalf("got %d template(s), want 4", len(templates)) + } + for _, tpl := range templates { + if tpl.EntityContext != tpl.Specialization { + t.Errorf("template for %s has EntityContext %q, want the specialization itself", + tpl.Specialization, tpl.EntityContext) + } + } +} diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index f30dca6c3..36d0c8622 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -400,20 +400,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { case "Forms$DataView", "Pages$DataView": header := fmt.Sprintf("dataview %s", mdlIdent(w.Name)) props := []string{} - if w.DataSource != nil { - switch w.DataSource.Type { - case "microflow": - props = append(props, fmt.Sprintf("DataSource: microflow %s", w.DataSource.Reference)) - case "nanoflow": - props = append(props, fmt.Sprintf("DataSource: nanoflow %s", w.DataSource.Reference)) - case "parameter": - props = append(props, fmt.Sprintf("DataSource: $%s", w.DataSource.Reference)) - case "selection": - props = append(props, fmt.Sprintf("DataSource: selection %s", mdlIdent(w.DataSource.Reference))) - case "association": - props = append(props, fmt.Sprintf("DataSource: %s", associationDataSourceExpr(w.DataSource))) - } - } + props = appendDataSourceProp(props, w.DataSource) switch { case w.LabelWidth == 0: props = append(props, "FormOrientation: Vertical") @@ -533,33 +520,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { if widgetType == "datagrid2" && (w.DataSource != nil || len(w.DataGridColumns) > 0) { header := fmt.Sprintf("datagrid %s", mdlIdent(w.Name)) props := []string{} - if w.DataSource != nil { - switch w.DataSource.Type { - case "database": - dsVal := fmt.Sprintf("database from %s", w.DataSource.Reference) - if w.DataSource.XPathConstraint != "" { - xpath := w.DataSource.XPathConstraint - if len(xpath) >= 2 && xpath[0] == '[' && xpath[len(xpath)-1] == ']' { - xpath = xpath[1 : len(xpath)-1] - } - dsVal += fmt.Sprintf(" where %s", xpath) - } - if len(w.DataSource.SortColumns) > 0 { - var sortParts []string - for _, col := range w.DataSource.SortColumns { - sortParts = append(sortParts, col.Attribute+" "+col.Order) - } - dsVal += fmt.Sprintf(" sort by %s", strings.Join(sortParts, ", ")) - } - props = append(props, fmt.Sprintf("DataSource: %s", dsVal)) - case "microflow": - props = append(props, fmt.Sprintf("DataSource: microflow %s", w.DataSource.Reference)) - case "nanoflow": - props = append(props, fmt.Sprintf("DataSource: nanoflow %s", w.DataSource.Reference)) - case "parameter": - props = append(props, fmt.Sprintf("DataSource: %s", w.DataSource.Reference)) - } - } + props = appendDataSourceProp(props, w.DataSource) // Add selection mode if specified if w.Selection != "" { props = append(props, fmt.Sprintf("Selection: %s", w.Selection)) @@ -597,32 +558,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { // Handle Gallery specially with datasource, selection, filter and content widgets header := fmt.Sprintf("gallery %s", mdlIdent(w.Name)) props := []string{} - if w.DataSource != nil { - switch w.DataSource.Type { - case "database": - dsVal := fmt.Sprintf("database from %s", w.DataSource.Reference) - if w.DataSource.XPathConstraint != "" { - xpath := w.DataSource.XPathConstraint - if len(xpath) >= 2 && xpath[0] == '[' && xpath[len(xpath)-1] == ']' { - xpath = xpath[1 : len(xpath)-1] - } - dsVal += fmt.Sprintf(" where %s", xpath) - } - // Add SORT BY if present - if len(w.DataSource.SortColumns) > 0 { - var sortParts []string - for _, col := range w.DataSource.SortColumns { - sortParts = append(sortParts, col.Attribute+" "+col.Order) - } - dsVal += fmt.Sprintf(" sort by %s", strings.Join(sortParts, ", ")) - } - props = append(props, fmt.Sprintf("DataSource: %s", dsVal)) - case "microflow": - props = append(props, fmt.Sprintf("DataSource: microflow %s", w.DataSource.Reference)) - case "nanoflow": - props = append(props, fmt.Sprintf("DataSource: nanoflow %s", w.DataSource.Reference)) - } - } + props = appendDataSourceProp(props, w.DataSource) // Add column counts if non-default if w.DesktopColumns != "" && w.DesktopColumns != "1" { props = append(props, fmt.Sprintf("DesktopColumns: %s", w.DesktopColumns)) @@ -728,13 +664,11 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { for i, item := range ol.Items { itemHeader := fmt.Sprintf("%s %s", ol.Keyword, mdlIdent(fmt.Sprintf("%s%d", ol.Keyword, i+1))) itemProps := []string{} - if item.DataSource != nil && item.DataSource.Reference != "" { - dsExpr := fmt.Sprintf("DataSource: database from %s", item.DataSource.Reference) - if item.DataSource.XPathConstraint != "" { - dsExpr += fmt.Sprintf(" where %s", item.DataSource.XPathConstraint) - } - itemProps = append(itemProps, dsExpr) - } + // An object-list item's datasource goes through the same + // renderer as a widget's. This site used to have no type + // switch at all, so a chart series bound to a microflow + // described as `database from Module.TheMicroflow` (#941). + itemProps = appendDataSourceProp(itemProps, item.DataSource) for _, p := range item.Props { if p.IsRef { itemProps = append(itemProps, fmt.Sprintf("%s: %s", p.Key, p.Value)) @@ -773,12 +707,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { // through the same branch — without it the filter described back as a // bare `dropdownfilter name` and the mode was lost on re-exec (#830). if w.DataSource != nil && (widgetType == "combobox" || widgetType == "dropdownfilter") { - switch w.DataSource.Type { - case "database": - props = append(props, fmt.Sprintf("DataSource: database from %s", w.DataSource.Reference)) - case "microflow": - props = append(props, fmt.Sprintf("DataSource: microflow %s", w.DataSource.Reference)) - } + props = appendDataSourceProp(props, w.DataSource) if w.CaptionAttribute != "" { props = append(props, fmt.Sprintf("CaptionAttribute: %s", w.CaptionAttribute)) } @@ -826,31 +755,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { case "Forms$Gallery", "Pages$Gallery": header := fmt.Sprintf("gallery %s", mdlIdent(w.Name)) props := []string{} - if w.DataSource != nil { - switch w.DataSource.Type { - case "database": - dsVal := fmt.Sprintf("database from %s", w.DataSource.Reference) - if w.DataSource.XPathConstraint != "" { - xpath := w.DataSource.XPathConstraint - if len(xpath) >= 2 && xpath[0] == '[' && xpath[len(xpath)-1] == ']' { - xpath = xpath[1 : len(xpath)-1] - } - dsVal += fmt.Sprintf(" where %s", xpath) - } - if len(w.DataSource.SortColumns) > 0 { - var sortParts []string - for _, col := range w.DataSource.SortColumns { - sortParts = append(sortParts, col.Attribute+" "+col.Order) - } - dsVal += fmt.Sprintf(" sort by %s", strings.Join(sortParts, ", ")) - } - props = append(props, fmt.Sprintf("DataSource: %s", dsVal)) - case "microflow": - props = append(props, fmt.Sprintf("DataSource: microflow %s", w.DataSource.Reference)) - case "parameter": - props = append(props, fmt.Sprintf("DataSource: %s", w.DataSource.Reference)) - } - } + props = appendDataSourceProp(props, w.DataSource) props = appendAppearanceProps(props, w) if len(w.Children) > 0 { formatWidgetProps(ctx.Output, prefix, header, props, " {\n") @@ -872,6 +777,16 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = appendAppearanceProps(props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") + case "Forms$ListViewTemplate": + // A List View specialization template. It has no name — the entity it + // renders is what identifies it — so the header is `template for ` + // rather than the `template ` a Gallery content slot uses. + fmt.Fprintf(ctx.Output, "%stemplate for %s {\n", prefix, w.Specialization) + for _, child := range w.Children { + outputWidgetMDLV3(ctx, child, indent+1) + } + fmt.Fprintf(ctx.Output, "%s}\n", prefix) + case "Footer": fmt.Fprintf(ctx.Output, "%sfooter %s {\n", prefix, mdlIdent(w.Name)) for _, child := range w.Children { @@ -883,28 +798,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { // ListView (also used for Gallery serialization) header := fmt.Sprintf("listview %s", mdlIdent(w.Name)) props := []string{} - if w.DataSource != nil { - switch w.DataSource.Type { - case "database": - dsVal := fmt.Sprintf("database from %s", w.DataSource.Reference) - if w.DataSource.XPathConstraint != "" { - xpath := w.DataSource.XPathConstraint - if len(xpath) >= 2 && xpath[0] == '[' && xpath[len(xpath)-1] == ']' { - xpath = xpath[1 : len(xpath)-1] - } - dsVal += fmt.Sprintf(" where %s", xpath) - } - props = append(props, fmt.Sprintf("DataSource: %s", dsVal)) - case "microflow": - props = append(props, fmt.Sprintf("DataSource: microflow %s", w.DataSource.Reference)) - case "nanoflow": - props = append(props, fmt.Sprintf("DataSource: nanoflow %s", w.DataSource.Reference)) - case "parameter": - props = append(props, fmt.Sprintf("DataSource: %s", w.DataSource.Reference)) - case "association": - props = append(props, fmt.Sprintf("DataSource: %s", associationDataSourceExpr(w.DataSource))) - } - } + props = appendDataSourceProp(props, w.DataSource) // Emit a non-default PageSize so it round-trips (Studio Pro's default is 20). if w.PageSize != "" && w.PageSize != "20" { props = append(props, fmt.Sprintf("PageSize: %s", w.PageSize)) diff --git a/mdl/executor/cmd_pages_describe_parse.go b/mdl/executor/cmd_pages_describe_parse.go index 3f24c728b..81f3817dd 100644 --- a/mdl/executor/cmd_pages_describe_parse.go +++ b/mdl/executor/cmd_pages_describe_parse.go @@ -600,50 +600,9 @@ func extractDataViewDataSource(ctx *ExecContext, w map[string]any) *rawDataSourc if !ok { return nil } - - dsType, _ := ds["$Type"].(string) - - switch dsType { - case "Forms$MicroflowSource": - if mf := microflowSourceRef(ds); mf != "" { - return &rawDataSource{Type: "microflow", Reference: mf} - } - case "Forms$NanoflowSource": - if nf := nanoflowSourceRef(ds); nf != "" { - return &rawDataSource{Type: "nanoflow", Reference: nf} - } - case "Forms$DataViewSource": - // "Data from context over an association" — the DataViewSource carries an - // IndirectEntityRef navigating the association. Reconstruct $ctx/Assoc. - if path, ctx := associationSourcePath(ds); path != "" { - return &rawDataSource{Type: "association", Reference: path, ContextVariable: ctx} - } - // Plain context source — extract the page parameter from SourceVariable. - if srcVar, ok := ds["SourceVariable"].(map[string]any); ok { - if paramName, ok := srcVar["PageParameter"].(string); ok && paramName != "" { - return &rawDataSource{Type: "parameter", Reference: paramName} - } - } - case "Forms$DatabaseSource": - // Database/XPath source - for now just note it's a database source - return &rawDataSource{Type: "database", Reference: ""} - case "Forms$ListenTargetSource": - // Master-detail binding: DataView listens to a selection-aware container - // (Gallery/ListView/DataGrid) by widget name. - if target, ok := ds["ListenTarget"].(string); ok && target != "" { - return &rawDataSource{Type: "selection", Reference: target} - } - case "Forms$AssociationSource": - if path, ctx := associationSourcePath(ds); path != "" { - return &rawDataSource{Type: "association", Reference: path, ContextVariable: ctx} - } - } - - return nil + return parseDataSource(ds) } -// extractDataViewLabelWidth reads the DataView LabelWidth as an int. Returns -// -1 when absent so callers can omit the property from output. func extractDataViewLabelWidth(w map[string]any) int { v, ok := w["LabelWidth"] if !ok { @@ -773,18 +732,51 @@ func parseListViewContent(ctx *ExecContext, w map[string]any, entityContext ...s if len(entityContext) > 0 { entCtx = entityContext[0] } - widgets := getBsonArrayElements(w["Widgets"]) - if widgets == nil { - return nil - } var result []rawWidget - for _, wgt := range widgets { + for _, wgt := range getBsonArrayElements(w["Widgets"]) { wgtMap, ok := wgt.(map[string]any) if !ok { continue } result = append(result, parseRawWidget(ctx, wgtMap, entCtx)...) } + result = append(result, parseListViewTemplates(ctx, w)...) + return result +} + +// parseListViewTemplates reads a List View's specialization templates — the +// per-specialization bodies Studio Pro stores in a Templates array, alongside +// (not inside) the list view's own Widgets. +// +// Nothing read this array before, so a template's entire contents were absent +// from DESCRIBE with no warning, and SEARCH inherited the blind spot because the +// catalog's source table is built from DESCRIBE output. Order is preserved: it is +// authored, not derived. Issue #940. +func parseListViewTemplates(ctx *ExecContext, w map[string]any) []rawWidget { + var result []rawWidget + for _, tpl := range getBsonArrayElements(w["Templates"]) { + tplMap, ok := tpl.(map[string]any) + if !ok { + continue + } + // Inside a template the context object is the specialization, so an + // attribute it adds resolves even though the list view's own entity + // does not have it. + spec := extractString(tplMap["Entity"]) + wrapper := rawWidget{ + Type: "Forms$ListViewTemplate", + Specialization: spec, + EntityContext: spec, + } + for _, wgt := range getBsonArrayElements(tplMap["Widgets"]) { + wgtMap, ok := wgt.(map[string]any) + if !ok { + continue + } + wrapper.Children = append(wrapper.Children, parseRawWidget(ctx, wgtMap, spec)...) + } + result = append(result, wrapper) + } return result } @@ -794,58 +786,9 @@ func extractListViewDataSource(ctx *ExecContext, w map[string]any) *rawDataSourc if !ok || ds == nil { return nil } - - dsType := extractString(ds["$Type"]) - switch dsType { - case "Forms$ListViewXPathSource": - result := &rawDataSource{Type: "database"} - entityRef, ok := ds["EntityRef"].(map[string]any) - if ok && entityRef != nil { - result.Reference = extractString(entityRef["Entity"]) - } - result.XPathConstraint = extractString(ds["XPathConstraint"]) - // Extract sorting from Sort field - if sortObj, ok := ds["Sort"].(map[string]any); ok { - sortPaths := getBsonArrayElements(sortObj["Paths"]) - for _, item := range sortPaths { - sortItem, ok := item.(map[string]any) - if !ok { - continue - } - col := rawSortColumn{Order: "asc"} - if attrRef, ok := sortItem["AttributeRef"].(map[string]any); ok { - col.Attribute = shortAttributeName(extractString(attrRef["Attribute"])) - } - sortOrder := gridSortDirection(sortItem) - if sortOrder == "Descending" { - col.Order = "desc" - } - if col.Attribute != "" { - result.SortColumns = append(result.SortColumns, col) - } - } - } - if result.Reference != "" { - return result - } - case "Forms$MicroflowSource": - if mf := microflowSourceRef(ds); mf != "" { - return &rawDataSource{Type: "microflow", Reference: mf} - } - case "Forms$NanoflowSource": - nanoflow := nanoflowSourceRef(ds) - if nanoflow != "" { - return &rawDataSource{Type: "nanoflow", Reference: nanoflow} - } - case "Forms$AssociationSource": - if path, ctx := associationSourcePath(ds); path != "" { - return &rawDataSource{Type: "association", Reference: path, ContextVariable: ctx} - } - } - return nil + return parseDataSource(ds) } -// extractSnippetRef extracts the snippet reference from a SnippetCallWidget. func extractSnippetRef(ctx *ExecContext, w map[string]any) string { // First try the FormCall.Form path (used for BY_NAME_REFERENCE) if formCall, ok := w["FormCall"].(map[string]any); ok { diff --git a/mdl/executor/cmd_pages_describe_pluggable.go b/mdl/executor/cmd_pages_describe_pluggable.go index 219896abf..010bdada8 100644 --- a/mdl/executor/cmd_pages_describe_pluggable.go +++ b/mdl/executor/cmd_pages_describe_pluggable.go @@ -194,95 +194,12 @@ func gridSortDirection(sortItem map[string]any) string { return extractString(sortItem["SortOrder"]) } -// extractDataGrid2DataSource extracts the datasource from a DataGrid2 CustomWidget. +// extractDataGrid2DataSource extracts the datasource from a DataGrid2 +// CustomWidget: it lives on one of the widget object's properties. func extractDataGrid2DataSource(ctx *ExecContext, w map[string]any) *rawDataSource { - obj, ok := w["Object"].(map[string]any) - if !ok { - return nil - } - - // Search through properties for datasource - props := getBsonArrayElements(obj["Properties"]) - for _, prop := range props { - propMap, ok := prop.(map[string]any) - if !ok { - continue - } - value, ok := propMap["Value"].(map[string]any) - if !ok { - continue - } - // Check for DataSource - ds, ok := value["DataSource"].(map[string]any) - if !ok || ds == nil { - continue - } - - dsType := extractString(ds["$Type"]) - switch dsType { - case "Forms$DatabaseSource": - entityRef, ok := ds["EntityRef"].(map[string]any) - if ok && entityRef != nil { - entity := extractString(entityRef["Entity"]) - if entity != "" { - return &rawDataSource{Type: "database", Reference: entity} - } - } - case "CustomWidgets$CustomWidgetXPathSource": - // CustomWidget datasource format - EntityRef contains Entity as qualified name - result := &rawDataSource{Type: "database"} - entityRef, ok := ds["EntityRef"].(map[string]any) - if ok && entityRef != nil { - result.Reference = extractString(entityRef["Entity"]) - } - // Extract XPathConstraint - result.XPathConstraint = extractString(ds["XPathConstraint"]) - // Extract sorting from SortBar - support multiple sort columns - if sortBar, ok := ds["SortBar"].(map[string]any); ok { - sortItems := getBsonArrayElements(sortBar["SortItems"]) - for _, item := range sortItems { - sortItem, ok := item.(map[string]any) - if !ok { - continue - } - col := rawSortColumn{Order: "asc"} - // Extract attribute from AttributeRef - if attrRef, ok := sortItem["AttributeRef"].(map[string]any); ok { - col.Attribute = shortAttributeName(extractString(attrRef["Attribute"])) - } - // Extract sort order - sortOrder := gridSortDirection(sortItem) - if sortOrder == "Descending" { - col.Order = "desc" - } - if col.Attribute != "" { - result.SortColumns = append(result.SortColumns, col) - } - } - } - if result.Reference != "" { - return result - } - case "Forms$MicroflowSource": - if mf := microflowSourceRef(ds); mf != "" { - return &rawDataSource{Type: "microflow", Reference: mf} - } - case "Forms$NanoflowSource": - if nf := nanoflowSourceRef(ds); nf != "" { - return &rawDataSource{Type: "nanoflow", Reference: nf} - } - case "Forms$EntityPathSource", "Forms$DataViewSource": - entityPath := extractString(ds["EntityPath"]) - if entityPath != "" { - return &rawDataSource{Type: "parameter", Reference: entityPath} - } - } - } - return nil + return firstObjectPropertyDataSource(w) } -// extractDataGrid2Columns extracts the columns from a DataGrid2 CustomWidget. -// entityContext is the resolved entity context from the DataGrid2's datasource. func extractDataGrid2Columns(ctx *ExecContext, w map[string]any, entityContext ...string) []rawDataGridColumn { obj, ok := w["Object"].(map[string]any) if !ok { @@ -710,98 +627,53 @@ func extractTextTemplateParameters(ctx *ExecContext, textTemplate map[string]any return result } -// extractGalleryDataSource extracts the datasource from a Gallery widget. -// Handles both Forms$Gallery and CustomWidgets$CustomWidget Gallery formats. +// extractGalleryDataSource extracts the datasource from a Gallery widget, +// which may be stored either as a pluggable widget (datasource on one of the +// object's properties) or as the older Forms$Gallery (datasource at the top +// level). func extractGalleryDataSource(ctx *ExecContext, w map[string]any) *rawDataSource { - // First check for CustomWidget Gallery format (datasource in Object.Properties) - if obj, ok := w["Object"].(map[string]any); ok { - props := getBsonArrayElements(obj["Properties"]) - for _, prop := range props { - propMap, ok := prop.(map[string]any) - if !ok { - continue - } - value, ok := propMap["Value"].(map[string]any) - if !ok { - continue - } - // Check for DataSource field in Value - only process if not nil - dsVal, hasDS := value["DataSource"] - if !hasDS { - continue - } - if ds, ok := dsVal.(map[string]any); ok && ds != nil { - result := parseCustomWidgetDataSource(ctx, ds) - if result != nil { - return result - } - } - } + if ds := firstObjectPropertyDataSource(w); ds != nil { + return ds } - - // Fall back to Forms$Gallery format (DataSource at top level) - ds, ok := w["DataSource"].(map[string]any) - if !ok || ds == nil { + top, ok := w["DataSource"].(map[string]any) + if !ok || top == nil { return nil } + return parseDataSource(top) +} - dsType := extractString(ds["$Type"]) - switch dsType { - case "Forms$DatabaseSource": - result := &rawDataSource{Type: "database"} - entityRef, ok := ds["EntityRef"].(map[string]any) - if ok && entityRef != nil { - result.Reference = extractString(entityRef["Entity"]) - } - result.XPathConstraint = extractString(ds["XPathConstraint"]) - // Extract sorting - if sortBar, ok := ds["SortBar"].(map[string]any); ok { - sortItems := getBsonArrayElements(sortBar["SortItems"]) - for _, item := range sortItems { - sortItem, ok := item.(map[string]any) - if !ok { - continue - } - col := rawSortColumn{Order: "asc"} - if attrRef, ok := sortItem["AttributeRef"].(map[string]any); ok { - col.Attribute = shortAttributeName(extractString(attrRef["Attribute"])) - } - sortOrder := gridSortDirection(sortItem) - if sortOrder == "Descending" { - col.Order = "desc" - } - if col.Attribute != "" { - result.SortColumns = append(result.SortColumns, col) - } - } - } - if result.Reference != "" { - return result +// firstObjectPropertyDataSource returns the datasource of the first property of +// a pluggable widget's object that carries one. +// +// Shared by every pluggable container rather than copied per widget: where the +// datasource sits is a property of the storage format, not of the widget, and +// the per-widget copies of this walk were what let their datasource switches +// drift apart (#941). +func firstObjectPropertyDataSource(w map[string]any) *rawDataSource { + obj, ok := w["Object"].(map[string]any) + if !ok { + return nil + } + for _, prop := range getBsonArrayElements(obj["Properties"]) { + propMap, ok := prop.(map[string]any) + if !ok { + continue } - case "Forms$MicroflowSource": - if mf := microflowSourceRef(ds); mf != "" { - return &rawDataSource{Type: "microflow", Reference: mf} + value, ok := propMap["Value"].(map[string]any) + if !ok { + continue } - case "Forms$NanoflowSource": - if nf := nanoflowSourceRef(ds); nf != "" { - return &rawDataSource{Type: "nanoflow", Reference: nf} + ds, ok := value["DataSource"].(map[string]any) + if !ok || ds == nil { + continue } - case "Forms$EntityPathSource", "Forms$DataViewSource": - entityPath := extractString(ds["EntityPath"]) - if entityPath != "" { - return &rawDataSource{Type: "parameter", Reference: entityPath} + if result := parseDataSource(ds); result != nil { + return result } } return nil } -// microflowSourceRef returns the microflow a Forms$MicroflowSource points at. -// -// Studio Pro and the codec engine store the name in the nested Forms$MicroflowSettings; -// a top-level "Microflow" key is the legacy shape, still honoured so older files -// round-trip. Reading only the top-level key made DESCRIBE PAGE drop a datagrid's -// microflow datasource entirely (mendixlabs/mxcli#795), so every reader goes through -// this helper rather than keeping its own copy of the lookup. func microflowSourceRef(ds map[string]any) string { if mf := extractString(ds["Microflow"]); mf != "" { return mf @@ -823,58 +695,13 @@ func nanoflowSourceRef(ds map[string]any) string { return "" } -// parseCustomWidgetDataSource parses datasource from CustomWidget property format. +// parseCustomWidgetDataSource reads a pluggable widget property's datasource. +// Kept as a named seam over the shared reader because callers read better for +// it; the switch itself lives in one place (see cmd_pages_describe_datasource.go). func parseCustomWidgetDataSource(ctx *ExecContext, ds map[string]any) *rawDataSource { - dsType := extractString(ds["$Type"]) - switch dsType { - case "CustomWidgets$CustomWidgetXPathSource": - result := &rawDataSource{Type: "database"} - entityRef, ok := ds["EntityRef"].(map[string]any) - if ok && entityRef != nil { - result.Reference = extractString(entityRef["Entity"]) - } - result.XPathConstraint = extractString(ds["XPathConstraint"]) - // Extract sorting if present - if sortBar, ok := ds["SortBar"].(map[string]any); ok { - sortItems := getBsonArrayElements(sortBar["SortItems"]) - for _, item := range sortItems { - sortItem, ok := item.(map[string]any) - if !ok { - continue - } - col := rawSortColumn{Order: "asc"} - if attrRef, ok := sortItem["AttributeRef"].(map[string]any); ok { - col.Attribute = shortAttributeName(extractString(attrRef["Attribute"])) - } - sortOrder := gridSortDirection(sortItem) - if sortOrder == "Descending" { - col.Order = "desc" - } - if col.Attribute != "" { - result.SortColumns = append(result.SortColumns, col) - } - } - } - return result - case "Forms$MicroflowSource": - if mf := microflowSourceRef(ds); mf != "" { - return &rawDataSource{Type: "microflow", Reference: mf} - } - case "Forms$NanoflowSource": - if nf := nanoflowSourceRef(ds); nf != "" { - return &rawDataSource{Type: "nanoflow", Reference: nf} - } - case "CustomWidgets$CustomWidgetNanoflowSource": - nanoflow := extractString(ds["Nanoflow"]) - if nanoflow != "" { - return &rawDataSource{Type: "nanoflow", Reference: nanoflow} - } - } - return nil + return parseDataSource(ds) } -// extractGalleryContent extracts the content widgets from a CustomWidget Gallery. -// entityContext is the resolved entity context from the Gallery's datasource. func extractGalleryContent(ctx *ExecContext, w map[string]any, entityContext ...string) []rawWidget { entCtx := "" if len(entityContext) > 0 { diff --git a/mdl/executor/cmd_published_rest.go b/mdl/executor/cmd_published_rest.go index c69396b2b..e9f6a3074 100644 --- a/mdl/executor/cmd_published_rest.go +++ b/mdl/executor/cmd_published_rest.go @@ -249,9 +249,15 @@ func execCreatePublishedRestService(ctx *ExecContext, s *ast.CreatePublishedRest } if existing != nil { + if s.Folder == "" { + svc.ContainerID = existing.ContainerID + } if err := ctx.Backend.UpdatePublishedRestService(svc); err != nil { return mdlerrors.NewBackend("update published rest service", err) } + if _, err := applyDocumentFolder(ctx, svc.ID, existing.ContainerID, svc.ContainerID); err != nil { + return err + } if !ctx.Quiet { ctx.ReportMutation("Modified", "published rest service %s.%s", s.Name.Module, s.Name.Name) } diff --git a/mdl/executor/cmd_queues.go b/mdl/executor/cmd_queues.go index 23abcf4b7..16a7934b2 100644 --- a/mdl/executor/cmd_queues.go +++ b/mdl/executor/cmd_queues.go @@ -15,6 +15,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" ) // findQueue returns the queue with the given module-qualified name, or nil. @@ -60,9 +61,13 @@ func execCreateQueue(ctx *ExecContext, s *ast.CreateQueueStmt) error { return mdlerrors.NewAlreadyExists("queue", s.Name.String()) } - containerID := module.ID + var existingContainer model.ID if existing != nil { - containerID = existing.ContainerID + existingContainer = existing.ContainerID + } + containerID, err := containerForDocument(ctx, module.ID, s.Folder, existingContainer) + if err != nil { + return err } q := &types.Queue{ @@ -83,6 +88,9 @@ func execCreateQueue(ctx *ExecContext, s *ast.CreateQueueStmt) error { if err := ctx.Backend.UpdateQueue(q); err != nil { return mdlerrors.NewBackend("update queue", err) } + if _, err := applyDocumentFolder(ctx, q.ID, existingContainer, containerID); err != nil { + return err + } ctx.ReportMutation("Modified", "queue: %s", s.Name.String()) return nil } @@ -165,7 +173,7 @@ func execDescribeQueue(ctx *ExecContext, s *ast.DescribeQueueStmt) error { if q.Documentation != "" { fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", q.Documentation) } - fmt.Fprintf(ctx.Output, "create or modify queue %s (\n", s.Name.String()) + fmt.Fprintf(ctx.Output, "create or modify queue %s%s (\n", s.Name.String(), describeFolderClause(ctx, q.ContainerID)) // Parallelism is an expression string; quote it unless it is a plain integer, // so an expression survives the round-trip. fmt.Fprintf(ctx.Output, " Parallelism: %s,\n", formatParallelism(q.Parallelism)) diff --git a/mdl/executor/cmd_regularexpressions.go b/mdl/executor/cmd_regularexpressions.go index d82a6b9be..0035089e3 100644 --- a/mdl/executor/cmd_regularexpressions.go +++ b/mdl/executor/cmd_regularexpressions.go @@ -64,8 +64,17 @@ func execCreateRegularExpression(ctx *ExecContext, s *ast.CreateRegularExpressio return mdlerrors.NewAlreadyExists("regular expression", s.Name.String()) } + var existingContainer model.ID + if existing != nil { + existingContainer = existing.ContainerID + } + containerID, err := containerForDocument(ctx, module.ID, s.Folder, existingContainer) + if err != nil { + return err + } + re := &model.RegularExpression{ - ContainerID: module.ID, + ContainerID: containerID, Name: s.Name.Name, Documentation: s.Documentation, Expression: s.Expression, @@ -73,11 +82,13 @@ func execCreateRegularExpression(ctx *ExecContext, s *ast.CreateRegularExpressio } if existing != nil { re.ID = existing.ID - re.ContainerID = existing.ContainerID re.Excluded = existing.Excluded if err := ctx.Backend.UpdateRegularExpression(re); err != nil { return mdlerrors.NewBackend("update regular expression", err) } + if _, err := applyDocumentFolder(ctx, re.ID, existingContainer, containerID); err != nil { + return err + } ctx.ReportMutation("Modified", "regular expression: %s", s.Name.String()) return nil } @@ -183,7 +194,7 @@ func execDescribeRegularExpression(ctx *ExecContext, s *ast.DescribeRegularExpre if re.Documentation != "" { fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", re.Documentation) } - fmt.Fprintf(ctx.Output, "create or modify regular expression %s (\n", s.Name.String()) + fmt.Fprintf(ctx.Output, "create or modify regular expression %s%s (\n", s.Name.String(), describeFolderClause(ctx, re.ContainerID)) fmt.Fprintf(ctx.Output, " Expression: '%s',\n", strings.ReplaceAll(re.Expression, "'", "''")) if re.ExportLevel != "" && re.ExportLevel != "Hidden" { fmt.Fprintf(ctx.Output, " ExportLevel: %s,\n", re.ExportLevel) diff --git a/mdl/executor/cmd_rest_clients.go b/mdl/executor/cmd_rest_clients.go index 2d7024e7b..8ee18aced 100644 --- a/mdl/executor/cmd_rest_clients.go +++ b/mdl/executor/cmd_rest_clients.go @@ -335,6 +335,10 @@ func createRestClient(ctx *ExecContext, stmt *ast.CreateRestClientStmt) error { } var preservedID model.ID + // Where the service currently sits. A rest client is rewritten as + // delete+create, so its container is re-applied on every statement and an + // unset value files a foldered service back into the module root (#932). + var preservedContainerID model.ID wasModified := false for _, existing := range existingServices { existModID := h.FindModuleID(existing.ContainerID) @@ -343,6 +347,7 @@ func createRestClient(ctx *ExecContext, stmt *ast.CreateRestClientStmt) error { if stmt.CreateOrModify { // Preserve the existing ID so SEND REST REQUEST references stay valid after replace. preservedID = existing.ID + preservedContainerID = existing.ContainerID wasModified = true if err := ctx.Backend.DeleteConsumedRestService(existing.ID); err != nil { return mdlerrors.NewBackend("delete existing rest client", err) @@ -353,7 +358,7 @@ func createRestClient(ctx *ExecContext, stmt *ast.CreateRestClientStmt) error { } } - // Resolve folder if specified + // Resolve folder if specified, else keep where the service already was. containerID := module.ID if stmt.Folder != "" { folderID, err := resolveFolder(ctx, module.ID, stmt.Folder) @@ -361,6 +366,8 @@ func createRestClient(ctx *ExecContext, stmt *ast.CreateRestClientStmt) error { return mdlerrors.NewBackend(fmt.Sprintf("resolve folder '%s'", stmt.Folder), err) } containerID = folderID + } else if preservedContainerID != "" { + containerID = preservedContainerID } // Build the model from AST @@ -799,8 +806,13 @@ func createRestClientFromSpec(ctx *ExecContext, stmt *ast.CreateRestClientStmt) existModName := h.GetModuleName(existModID) if strings.EqualFold(existModName, moduleName) && strings.EqualFold(existing.Name, stmt.Name.Name) { if stmt.CreateOrModify { - // Reuse the existing ID so microflow references stay valid. + // Reuse the existing ID so microflow references stay valid, and + // its container so a spec re-import does not file the service + // back into the module root (#932). svc.ID = existing.ID + if stmt.Folder == "" { + svc.ContainerID = existing.ContainerID + } openAPIWasModified = true if err := ctx.Backend.DeleteConsumedRestService(existing.ID); err != nil { return mdlerrors.NewBackend("delete existing rest client", err) diff --git a/mdl/executor/cmd_scheduledevents.go b/mdl/executor/cmd_scheduledevents.go index 8895aedb9..4294144f0 100644 --- a/mdl/executor/cmd_scheduledevents.go +++ b/mdl/executor/cmd_scheduledevents.go @@ -110,10 +110,17 @@ func execCreateScheduledEvent(ctx *ExecContext, s *ast.CreateScheduledEventStmt) if err != nil { return err } - ev.ContainerID = module.ID + var existingContainer model.ID + if existing != nil { + existingContainer = existing.ContainerID + } + containerID, err := containerForDocument(ctx, module.ID, s.Folder, existingContainer) + if err != nil { + return err + } + ev.ContainerID = containerID if existing != nil { ev.ID = existing.ID - ev.ContainerID = existing.ContainerID // Interval/IntervalType are legacy siblings of Schedule that Studio Pro // writes but does not keep in sync, and MDL has no syntax for them. // Carry the stored values so a modify does not invent new ones. @@ -122,6 +129,9 @@ func execCreateScheduledEvent(ctx *ExecContext, s *ast.CreateScheduledEventStmt) if err := ctx.Backend.UpdateScheduledEvent(ev); err != nil { return mdlerrors.NewBackend("update scheduled event", err) } + if _, err := applyDocumentFolder(ctx, ev.ID, existingContainer, containerID); err != nil { + return err + } ctx.ReportMutation("Modified", "scheduled event: %s", s.Name.String()) return nil } @@ -517,7 +527,7 @@ func execDescribeScheduledEvent(ctx *ExecContext, s *ast.DescribeScheduledEventS if ev.Documentation != "" { fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", ev.Documentation) } - fmt.Fprintf(ctx.Output, "create or modify scheduled event %s (\n", s.Name.String()) + fmt.Fprintf(ctx.Output, "create or modify scheduled event %s%s (\n", s.Name.String(), describeFolderClause(ctx, ev.ContainerID)) fmt.Fprintf(ctx.Output, " Microflow: %s,\n", ev.MicroflowID) for _, line := range describeScheduleProperties(ev.Schedule) { fmt.Fprintf(ctx.Output, " %s,\n", line) diff --git a/mdl/executor/cmd_workflows.go b/mdl/executor/cmd_workflows.go index 7d622e643..24b14c50c 100644 --- a/mdl/executor/cmd_workflows.go +++ b/mdl/executor/cmd_workflows.go @@ -186,6 +186,9 @@ func describeWorkflowToString(ctx *ExecContext, name ast.QualifiedName) (string, lines = append(lines, "") lines = append(lines, fmt.Sprintf("create workflow %s", qualifiedName)) + if clause := describeFolderClause(ctx, targetWf.ContainerID); clause != "" { + lines = append(lines, " "+strings.TrimSpace(clause)) + } // Context parameter if targetWf.Parameter != nil && targetWf.Parameter.EntityRef != "" { diff --git a/mdl/executor/cmd_workflows_write.go b/mdl/executor/cmd_workflows_write.go index 6f6d24959..7da427519 100644 --- a/mdl/executor/cmd_workflows_write.go +++ b/mdl/executor/cmd_workflows_write.go @@ -53,6 +53,7 @@ func execCreateWorkflow(ctx *ExecContext, s *ast.CreateWorkflowStmt) error { } var existingID model.ID + var existingContainer model.ID // Excluded is model state, not script state, and a module may hold an // excluded twin of this name — target the live workflow and carry its // exclusion forward (#914). @@ -68,11 +69,17 @@ func execCreateWorkflow(ctx *ExecContext, s *ast.CreateWorkflowStmt) error { } existingID = existing.ID existingExcluded = existing.Excluded + existingContainer = existing.ContainerID + } + + containerID, err := containerForDocument(ctx, module.ID, s.Folder, existingContainer) + if err != nil { + return err } wf := &workflows.Workflow{} wf.Excluded = existingExcluded - wf.ContainerID = module.ID + wf.ContainerID = containerID wf.Name = s.Name.Name wf.Documentation = s.Documentation @@ -138,6 +145,9 @@ func execCreateWorkflow(ctx *ExecContext, s *ast.CreateWorkflowStmt) error { if err := ctx.Backend.UpdateWorkflow(wf); err != nil { return mdlerrors.NewBackend("update workflow", err) } + if _, err := applyDocumentFolder(ctx, wf.ID, existingContainer, containerID); err != nil { + return err + } } else { if err := ctx.Backend.CreateWorkflow(wf); err != nil { return mdlerrors.NewBackend("create workflow", err) diff --git a/mdl/executor/document_placement.go b/mdl/executor/document_placement.go new file mode 100644 index 000000000..443943de5 --- /dev/null +++ b/mdl/executor/document_placement.go @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: Apache-2.0 + +// document_placement.go — applying a FOLDER clause to a document that already +// exists. +// +// A document's folder is its unit's container, which lives in the Unit row and +// not in the document's contents. Every Update* on both engines rewrites +// contents only, so a handler that resolved a folder, stored it on the model +// object and called Update had its placement silently dropped: the statement +// reported success, `resolveFolder` had already created the folder as a side +// effect, and the document stayed where it was. That was true of every doctype +// with a FOLDER clause, not of any one of them (#932). +// +// The fix is one call per CREATE OR MODIFY handler, not a per-doctype Move* +// method: placement is the same row update whatever the document is. +package executor + +import ( + "strings" + + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/model" +) + +// resolveRequestedFolder resolves a statement's FOLDER clause to a container, +// returning "" when the statement did not name one. +// +// The empty return is the point. resolveFolder answers an empty path with the +// module root, which is the right answer for a CREATE and exactly the wrong one +// for a MODIFY: it would file every foldered document back into the module root +// the moment anyone re-ran a script that says nothing about folders. Pairing +// this with applyDocumentFolder — which does nothing for an empty target — +// makes "no folder clause" mean "leave it alone" at every call site, instead of +// leaving each handler to remember the distinction. +func resolveRequestedFolder(ctx *ExecContext, moduleID model.ID, folder string) (model.ID, error) { + if folder == "" { + return "", nil + } + id, err := resolveFolder(ctx, moduleID, folder) + if err != nil { + return "", mdlerrors.NewBackend("resolve folder "+folder, err) + } + return id, nil +} + +// describeFolderClause renders a document's placement as the MDL clause that +// would reproduce it, or "" when the document sits at the module root. +// +// DESCRIBE output is advertised as re-executable, so a description that omits +// the folder does not round-trip: replaying it in a fresh project recreates the +// document unfiled. Returned with a leading space so a caller can append it to +// the name without deciding whether one is needed. +// +// Best-effort in the same way the rest of DESCRIBE is: a hierarchy that cannot +// resolve the container yields no clause rather than a wrong one. +func describeFolderClause(ctx *ExecContext, containerID model.ID) string { + h, err := getHierarchy(ctx) + if err != nil || h == nil { + return "" + } + path := h.BuildFolderPath(containerID) + if path == "" { + return "" + } + return " folder '" + strings.ReplaceAll(path, "'", "''") + "'" +} + +// containerForDocument picks the container a CREATE OR MODIFY should use, in +// the order that makes both a create and a modify behave sensibly: the folder +// the statement named, else where the document already sits, else the module +// root. +// +// The middle term is the one worth stating out loud. It means a statement that +// says nothing about folders is silent about placement rather than asserting +// the module root — which matters for the doctypes rewritten as delete+create, +// where the container really is re-applied on every statement. +func containerForDocument(ctx *ExecContext, moduleID model.ID, folder string, existing model.ID) (model.ID, error) { + target, err := resolveRequestedFolder(ctx, moduleID, folder) + if err != nil { + return "", err + } + if target != "" { + return target, nil + } + if existing != "" { + return existing, nil + } + return moduleID, nil +} + +// applyDocumentFolder moves an existing document to the container a CREATE OR +// MODIFY resolved for it, and reports whether anything moved. +// +// Called only when the placement actually differs. That is not just an +// optimisation: it keeps a statement that says nothing about folders from +// touching the containment row at all, so the common path stays exactly as it +// was and a backend that cannot move documents only ever hears about it when a +// user really asked for a folder. +func applyDocumentFolder(ctx *ExecContext, unitID, from, to model.ID) (bool, error) { + if unitID == "" || to == "" || from == to { + return false, nil + } + if err := ctx.Backend.MoveDocument(unitID, to); err != nil { + return false, mdlerrors.NewBackend("apply folder", err) + } + // The hierarchy cache indexes documents by container, so it is stale the + // moment one moves — a DESCRIBE later in the same script would otherwise + // still render the old folder and look like the bug this fixes. + invalidateHierarchy(ctx) + return true, nil +} diff --git a/mdl/executor/document_placement_test.go b/mdl/executor/document_placement_test.go new file mode 100644 index 000000000..550f124b0 --- /dev/null +++ b/mdl/executor/document_placement_test.go @@ -0,0 +1,184 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// placementProbe captures what a handler asked storage to reparent, so a test +// can assert on the move itself rather than on a rendering of it. +type placementProbe struct { + moved bool + unitID model.ID + containerID model.ID +} + +// folderMockBackend wires the minimum a CREATE OR MODIFY handler needs to +// resolve a folder: an existing module, a folder tree it can extend, and a +// recording MoveDocument. +func folderMockBackend(t *testing.T, mod *model.Module, probe *placementProbe) (*mock.MockBackend, *ContainerHierarchy) { + t.Helper() + folders := []*types.FolderInfo{} + h := mkHierarchy(mod) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + GetModuleByNameFunc: func(name string) (*model.Module, error) { + if name == mod.Name { + return mod, nil + } + return nil, nil + }, + ListFoldersFunc: func() ([]*types.FolderInfo, error) { return folders, nil }, + CreateFolderFunc: func(f *model.Folder) error { + folders = append(folders, &types.FolderInfo{ID: f.ID, ContainerID: f.ContainerID, Name: f.Name}) + // Register the new folder so BuildFolderPath can walk up from it. + withContainer(h, f.ID, f.ContainerID) + h.folderNames[f.ID] = f.Name + return nil + }, + MoveDocumentFunc: func(unitID, containerID model.ID) error { + probe.moved = true + probe.unitID = unitID + probe.containerID = containerID + return nil + }, + } + return mb, h +} + +// TestCreateOrModifyJsonStructureAppliesTheFolder is the reported case (#932): +// a FOLDER clause on a JSON structure that already exists was accepted, +// reported as a success, and dropped. +func TestCreateOrModifyJsonStructureAppliesTheFolder(t *testing.T) { + mod := mkModule("RT") + existing := &types.JsonStructure{ + BaseElement: model.BaseElement{ID: nextID("js")}, + ContainerID: mod.ID, // module root + Name: "JSON_Example", + } + + var probe placementProbe + mb, h := folderMockBackend(t, mod, &probe) + mb.ListJsonStructuresFunc = func() ([]*types.JsonStructure, error) { + return []*types.JsonStructure{existing}, nil + } + mb.UpdateJsonStructureFunc = func(*types.JsonStructure) error { return nil } + withContainer(h, existing.ContainerID, mod.ID) + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execCreateJsonStructure(ctx, &ast.CreateJsonStructureStmt{ + Name: ast.QualifiedName{Module: "RT", Name: "JSON_Example"}, + Folder: "Private/JSON Structures", + JsonSnippet: `{"result": 1}`, + CreateOrModify: true, + }) + assertNoError(t, err) + + if !probe.moved { + t.Fatal("the folder clause was accepted but nothing was reparented") + } + if probe.unitID != existing.ID { + t.Errorf("reparented %q, want the existing json structure %q", probe.unitID, existing.ID) + } + if probe.containerID == mod.ID || probe.containerID == "" { + t.Errorf("reparented to %q, want the resolved folder rather than the module root", probe.containerID) + } +} + +// TestCreateOrModifyMicroflowAppliesTheFolder pins that this is a property of +// the class and not of JSON structures. The issue reported one doctype; every +// doctype with a FOLDER clause dropped it the same way, because the placement +// lives in the unit row that no Update* touches. +func TestCreateOrModifyMicroflowAppliesTheFolder(t *testing.T) { + mod := mkModule("RT") + existing := mkMicroflow(mod.ID, "ACT_Existing") + + var probe placementProbe + mb, h := folderMockBackend(t, mod, &probe) + mb.ListMicroflowsFunc = func() ([]*microflows.Microflow, error) { return []*microflows.Microflow{existing}, nil } + mb.UpdateMicroflowFunc = func(*microflows.Microflow) error { return nil } + withContainer(h, existing.ContainerID, mod.ID) + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execCreateMicroflow(ctx, &ast.CreateMicroflowStmt{ + Name: ast.QualifiedName{Module: "RT", Name: "ACT_Existing"}, + Folder: "Private", + CreateOrModify: true, + }) + assertNoError(t, err) + + if !probe.moved { + t.Fatal("the folder clause was accepted but nothing was reparented") + } + if probe.unitID != existing.ID { + t.Errorf("reparented %q, want the existing microflow %q", probe.unitID, existing.ID) + } +} + +// TestCreateOrModifyWithoutFolderLeavesPlacementAlone is the control, and the +// one that decides whether the fix is a fix or a new bug. An omitted FOLDER +// must keep the document where it is — a handler that reparented to the module +// root by default would yank every foldered document out of its folder on the +// next CREATE OR MODIFY, which is far worse than the silent no-op being fixed. +func TestCreateOrModifyWithoutFolderLeavesPlacementAlone(t *testing.T) { + mod := mkModule("RT") + folderID := nextID("fld") + existing := &types.JsonStructure{ + BaseElement: model.BaseElement{ID: nextID("js")}, + ContainerID: folderID, // already filed away + Name: "JSON_Example", + } + + var probe placementProbe + mb, h := folderMockBackend(t, mod, &probe) + mb.ListJsonStructuresFunc = func() ([]*types.JsonStructure, error) { + return []*types.JsonStructure{existing}, nil + } + mb.UpdateJsonStructureFunc = func(*types.JsonStructure) error { return nil } + withContainer(h, folderID, mod.ID) + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execCreateJsonStructure(ctx, &ast.CreateJsonStructureStmt{ + Name: ast.QualifiedName{Module: "RT", Name: "JSON_Example"}, + JsonSnippet: `{"result": 2}`, + CreateOrModify: true, + }) + assertNoError(t, err) + + if probe.moved { + t.Errorf("a statement with no folder clause reparented the document to %q", probe.containerID) + } +} + +// TestApplyDocumentFolderSkipsAMoveToTheSamePlace pins that re-running a script +// that already placed a document does not touch the containment row. Storage +// elides it too, but the handler must not depend on that: a backend is allowed +// to be a plain writer, and ADR-0008's observable promise is that an in-sync +// project is left byte-identical. +func TestApplyDocumentFolderSkipsAMoveToTheSamePlace(t *testing.T) { + folderID := nextID("fld") + var probe placementProbe + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + MoveDocumentFunc: func(unitID, containerID model.ID) error { + probe.moved = true + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb)) + + moved, err := applyDocumentFolder(ctx, nextID("unit"), folderID, folderID) + assertNoError(t, err) + if moved || probe.moved { + t.Error("moving a document to the container it already occupies reached the backend") + } +} diff --git a/mdl/executor/editorconfig_extract.go b/mdl/executor/editorconfig_extract.go index 1e98bf11f..8aa71af44 100644 --- a/mdl/executor/editorconfig_extract.go +++ b/mdl/executor/editorconfig_extract.go @@ -19,14 +19,15 @@ import ( type editorConfigExtractStats struct { TotalHideCalls int // hidePropertyIn + hidePropertiesIn call sites seen Recognized int // lifted into a top-level WidgetVisibilityRule - SkippedNested int // object-list-nested (e.g. hidePropertyIn(...,"columns",n,"key")) + SkippedNested int // object-list-nested hide whose guard could not be lifted SkippedComplex int // ternary/compound guard, or an alias we couldn't resolve } -// hideCallRE locates a hidePropertyIn / hidePropertiesIn call and captures its -// (balanced-enough) argument list. hideNestedPropertiesIn is deliberately not -// matched — it only ever targets object-list items (Phase 2). -var hideCallRE = regexp.MustCompile(`hidePropert(?:y|ies)In\(`) +// hideCallRE locates a hidePropertyIn / hidePropertiesIn / hideNestedPropertiesIn +// call and captures its (balanced-enough) argument list. All three are matched: +// the nested forms target object-list items, which is where the Accordion hides +// the group properties that make an authored widget fail CE0463 (upstream #931). +var hideCallRE = regexp.MustCompile(`hide(?:Property|Properties|NestedProperties)In\(`) // aliasAssignRE finds `IDENT=OBJ.PROP` (a `var x=e.selection`-style alias). // Resolution is scoped to the enclosing function body (see enclosingAliases), @@ -140,40 +141,55 @@ func extractVisibilityRulesFromJS(js string) ([]types.WidgetVisibilityRule, edit stats.SkippedComplex++ continue } - keys, nested := hideTargetKeys(args) - if nested { - stats.SkippedNested++ - continue - } - if len(keys) == 0 { + listKey, keys, ok := hideTargetKeys(args) + if !ok || len(keys) == 0 { stats.SkippedComplex++ continue } - cond, ok := parseGuard(js, callStart) + // A nested hide sits inside the list's `forEach(function(item, i){…})`, so + // a guard reading `item.` is about the ITEM, not the widget. + itemIdent := "" + if listKey != "" { + itemIdent = enclosingForEachParam(js, callStart) + } + cond, ok := parseGuard(js, callStart, itemIdent) if !ok { - stats.SkippedComplex++ + if listKey != "" { + stats.SkippedNested++ + } else { + stats.SkippedComplex++ + } continue } stats.Recognized++ for _, key := range keys { - sig := key + "\x00" + cond.PropertyKey + cond.Operator + cond.Value + sig := listKey + "\x00" + key + "\x00" + cond.PropertyKey + cond.Operator + cond.Value + cond.Scope if seen[sig] { continue } seen[sig] = true c := cond // copy per rule - rules = append(rules, types.WidgetVisibilityRule{PropertyKey: key, HiddenWhen: &c}) + rules = append(rules, types.WidgetVisibilityRule{ + PropertyKey: key, + ListPropertyKey: listKey, + HiddenWhen: &c, + }) } } return rules, stats } -// hideTargetKeys returns the property key(s) a hide call targets and whether the -// call is object-list-nested (which we skip). A top-level hidePropertyIn has a -// single string arg (the key); a top-level hidePropertiesIn has one array of -// string keys. A `"columns"`/`"..."`-prefixed string arg followed by more args -// marks the nested form. -func hideTargetKeys(args string) (keys []string, nested bool) { +// hideTargetKeys returns the object-list property a hide call targets (empty for +// a top-level hide) and the property key(s) it hides. +// +// hidePropertyIn(t, e, "key") → "", ["key"] +// hidePropertiesIn(t, e, ["a","b"]) → "", ["a","b"] +// hidePropertyIn(t, e, "groups", i, "key") → "groups", ["key"] +// hideNestedPropertiesIn(t, e, "groups", i, ["a"]) → "groups", ["a"] +// +// ok is false for a shape it does not recognize; the caller counts it as skipped +// and emits no rule, which degrades to "not hidden". +func hideTargetKeys(args string) (listKey string, keys []string, ok bool) { parts := splitTopLevelCommas(args) // Collect string-literal positional args and any array literal. var stringArgs []string @@ -191,21 +207,80 @@ func hideTargetKeys(args string) (keys []string, nested bool) { } } if len(arrayKeys) > 0 { - // hidePropertiesIn(obj, obj, [keys]) — nested if a leading string arg - // (e.g. "columns") also appears. - if len(stringArgs) > 0 { - return nil, true + switch len(stringArgs) { + case 0: + return "", arrayKeys, true // hidePropertiesIn(obj, obj, [keys]) + case 1: + return stringArgs[0], arrayKeys, true // hideNestedPropertiesIn(…, "groups", i, [keys]) + default: + return "", nil, false } - return arrayKeys, false } switch len(stringArgs) { case 1: - return stringArgs, false // hidePropertyIn(obj, obj, "key") - case 0: - return nil, false + return "", stringArgs, true // hidePropertyIn(obj, obj, "key") + case 2: + return stringArgs[0], stringArgs[1:], true // hidePropertyIn(…, "groups", i, "key") default: - return nil, true // "columns","key" etc. — nested + return "", nil, false + } +} + +// forEachParamRE matches the callback parameter list of a `.forEach(function(a,b){` +// immediately before an enclosing block brace. The extra `\(*` is load-bearing: +// the Accordion Mendix ships is compiled as `forEach((function(n,o){…}))`, and +// without it the regex reads `function` itself as the parameter name — which +// scoped every nested condition to the widget and dropped the group's own +// `initialCollapsedState` guard on the floor. +var forEachParamRE = regexp.MustCompile(`forEach\(\s*\(*\s*(?:function\b\s*)?\(\s*([A-Za-z_$][\w$]*)`) + +// enclosingForEachParam returns the name of the first parameter of the +// `.forEach(function(item, index){…})` callback whose body encloses pos, or "" +// when pos is not inside one. +// +// It matters because a nested hide's guard can read either object: the Accordion +// writes `"dynamic" !== item.initialCollapsedState && hide(…, "initiallyCollapsed")` +// (an ITEM property) in the same callback as +// `(…) && widget.collapsible || hideNested(…)` (a WIDGET property). Without the +// distinction the two conditions are indistinguishable once resolveRef has +// dropped the base identifier. +func enclosingForEachParam(js string, pos int) string { + for i := 0; i < 8; i++ { // bounded: a hide is never nested this deep + open := enclosingOpener(js, pos) + if open < 0 { + return "" + } + if js[open] == '{' { + // Look just before the brace for the callback's parameter list. + start := open - 80 + if start < 0 { + start = 0 + } + if m := forEachParamRE.FindStringSubmatch(js[start:open]); m != nil && m[1] != "function" { + return m[1] + } + } + pos = open } + return "" +} + +// enclosingOpener returns the index of the nearest unbalanced opening bracket +// before pos, or -1 when there is none. +func enclosingOpener(js string, pos int) int { + depth := 0 + for i := pos - 1; i >= 0; i-- { + switch js[i] { + case '}', ')', ']': + depth++ + case '{', '(', '[': + if depth == 0 { + return i + } + depth-- + } + } + return -1 } // nsPrefixRE matches the widget-editor namespace prefix before a hide call — the @@ -216,7 +291,7 @@ var nsPrefixRE = regexp.MustCompile(`[A-Za-z_$][\w$]*\.$`) // parseGuard reads the guard expression immediately preceding a hide call and // converts it to a WidgetVisibilityCondition. callStart points at the hide // function name; the connector just before it is `&&`, `||`, or `?`. -func parseGuard(js string, callStart int) (types.WidgetVisibilityCondition, bool) { +func parseGuard(js string, callStart int, itemIdent string) (types.WidgetVisibilityCondition, bool) { pre := strings.TrimRight(js[:callStart], " ") // Strip the widget-editor namespace prefix (any `.`, not just `_.`). if loc := nsPrefixRE.FindStringIndex(pre); loc != nil { @@ -268,18 +343,35 @@ func parseGuard(js string, callStart int) (types.WidgetVisibilityCondition, bool } // Skip guards nested inside a larger expression. A clean statement-level guard // is bounded by a statement separator (`,`, `;`, `{`, or start-of-input); a - // boundary of `(`/`?`/`:`/`&`/`|` means the guard is one operand of a compound + // boundary of `(`/`?`/`:`/`|` means the guard is one operand of a compound // or ternary condition (e.g. `"web"===r ? (e.advanced || hide(...))`), of which // we'd capture only a fragment — producing a WRONG rule that over-fires. Better // to emit no rule (→ "not hidden" → template default), which is safe. + // + // `&` is the one compound boundary that is safe, and only for the `||` + // connector: in `X && Y || hide`, the hide fires when `X && Y` is falsy, and + // `Y` falsy is sufficient for that WHATEVER X is. So "hide when Y falsy" is + // implied by the code rather than guessed — it may miss the case where X + // alone is falsy, never fire where the widget would not hide. This is the + // Accordion's shape, and the reason its group properties went unflagged: + // + // (e.advancedMode || "web" !== platform) && e.collapsible + // || hideNestedPropertiesIn(t, e, "groups", i, ["initialCollapsedState", …]) + // + // The `&&` connector gets no such rule: there, hiding needs BOTH operands + // truthy, which a single condition cannot express. switch boundary { case 0, ',', ';', '{': // clean + case '&': + if !falsy { + return types.WidgetVisibilityCondition{}, false + } default: return types.WidgetVisibilityCondition{}, false } aliases := enclosingAliases(js, callStart) - return guardToCondition(guard, falsy, aliases) + return guardToCondition(guard, falsy, aliases, itemIdent) } // ternaryCondition returns the text preceding the `?` that matches a trailing @@ -409,29 +501,45 @@ var ( // guardToCondition parses a single guard expression into a visibility // condition, resolving a bare identifier through the scope's alias map. -func guardToCondition(guard string, falsy bool, aliases map[string]string) (types.WidgetVisibilityCondition, bool) { +func guardToCondition(guard string, falsy bool, aliases map[string]string, itemIdent string) (types.WidgetVisibilityCondition, bool) { + // A comparison guard obeys the connector's polarity just as a bare reference + // does: with `||`, or in a ternary's ELSE branch, the hide fires when the + // comparison is FALSE, so `===` becomes "not equal" and `!==` becomes + // "equal". The Accordion is where this shows: + // + // "text" === item.headerRenderMode + // ? (hide(…, "headerContent"), …) + // : (hide(…, "headerText"), hide(…, "headerHeading")) + // + // headerContent is hidden when the mode IS "text"; headerText when it is NOT. + // Reading both as `eq` marks headerText hidden in exactly the configuration + // where it is the property being used. + eqOp, neOp := "eq", "ne" + if falsy { + eqOp, neOp = "ne", "eq" + } // "V" === ref / ref === "V" if m := eqCmpRE.FindStringSubmatch(guard); m != nil { - if key, ok := resolveRef(m[2], aliases); ok { - return types.WidgetVisibilityCondition{PropertyKey: key, Operator: "eq", Value: m[1]}, true + if key, scope, ok := resolveRef(m[2], aliases, itemIdent); ok { + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: eqOp, Value: m[1], Scope: scope}, true } return types.WidgetVisibilityCondition{}, false } if m := eqCmpRE2.FindStringSubmatch(guard); m != nil { - if key, ok := resolveRef(m[1], aliases); ok { - return types.WidgetVisibilityCondition{PropertyKey: key, Operator: "eq", Value: m[2]}, true + if key, scope, ok := resolveRef(m[1], aliases, itemIdent); ok { + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: eqOp, Value: m[2], Scope: scope}, true } return types.WidgetVisibilityCondition{}, false } if m := neCmpRE.FindStringSubmatch(guard); m != nil { - if key, ok := resolveRef(m[2], aliases); ok { - return types.WidgetVisibilityCondition{PropertyKey: key, Operator: "ne", Value: m[1]}, true + if key, scope, ok := resolveRef(m[2], aliases, itemIdent); ok { + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: neOp, Value: m[1], Scope: scope}, true } return types.WidgetVisibilityCondition{}, false } if m := neCmpRE2.FindStringSubmatch(guard); m != nil { - if key, ok := resolveRef(m[1], aliases); ok { - return types.WidgetVisibilityCondition{PropertyKey: key, Operator: "ne", Value: m[2]}, true + if key, scope, ok := resolveRef(m[1], aliases, itemIdent); ok { + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: neOp, Value: m[2], Scope: scope}, true } return types.WidgetVisibilityCondition{}, false } @@ -441,7 +549,7 @@ func guardToCondition(guard string, falsy bool, aliases map[string]string) (type // !ref && hide → hide when ref falsy // ref ? hide:… → hide when ref truthy if m := refRE.FindStringSubmatch(guard); m != nil { - key, ok := resolveRef(m[2], aliases) + key, scope, ok := resolveRef(m[2], aliases, itemIdent) if !ok { return types.WidgetVisibilityCondition{}, false } @@ -451,22 +559,30 @@ func guardToCondition(guard string, falsy bool, aliases map[string]string) (type if wantFalsy { op = "falsy" } - return types.WidgetVisibilityCondition{PropertyKey: key, Operator: op}, true + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: op, Scope: scope}, true } return types.WidgetVisibilityCondition{}, false } -// resolveRef turns a guard reference into a widget property key: `obj.prop` -// yields `prop`; a bare identifier is looked up in the scope alias map. A bare -// identifier with no alias (e.g. a computed local) is unresolvable. -func resolveRef(ref string, aliases map[string]string) (string, bool) { +// resolveRef turns a guard reference into a property key and the scope that key +// belongs to: `obj.prop` yields `prop`; a bare identifier is looked up in the +// scope alias map. A bare identifier with no alias (e.g. a computed local) is +// unresolvable. +// +// When itemIdent is non-empty (the hide is inside that object list's forEach +// callback) a reference based on it is scoped to the list ITEM; everything else +// is a property of the widget. +func resolveRef(ref string, aliases map[string]string, itemIdent string) (key, scope string, ok bool) { if i := strings.LastIndexByte(ref, '.'); i >= 0 { - return ref[i+1:], true + if itemIdent != "" && ref[:i] == itemIdent { + return ref[i+1:], types.ConditionScopeItem, true + } + return ref[i+1:], "", true } if key, ok := aliases[ref]; ok { - return key, true + return key, "", true } - return "", false + return "", "", false } // enclosingAliases returns the `ident → property` aliases declared in the diff --git a/mdl/executor/editorconfig_extract_test.go b/mdl/executor/editorconfig_extract_test.go index f0f4f0294..33e59a81e 100644 --- a/mdl/executor/editorconfig_extract_test.go +++ b/mdl/executor/editorconfig_extract_test.go @@ -102,16 +102,33 @@ func TestExtractVisibility_ScopedAlias(t *testing.T) { } } -// TestExtractVisibility_SkipsNested confirms object-list-nested hides -// (hidePropertyIn(...,"columns",n,"key")) are not lifted as top-level rules. -func TestExtractVisibility_SkipsNested(t *testing.T) { +// TestExtractVisibility_NestedIsScopedToItsList confirms an object-list-nested +// hide (hidePropertyIn(...,"columns",n,"key")) is lifted as a NESTED rule and +// never as a top-level one: `sortable` is a property of a column, not of the +// grid, so a consumer evaluating it against the grid would read an absent key. +// +// The rules used to be skipped entirely (#574 Phase 1), which is what let an +// Accordion group carry a value its widget hides all the way to a CE0463 that +// only `mx create-module-package` reported (upstream #931). +func TestExtractVisibility_NestedIsScopedToItsList(t *testing.T) { js := `g=function(e,t){e.columns.forEach(function(r,n){e.columnsSortable||_.hidePropertyIn(t,e,"columns",n,"sortable")})}` - rules, stats := extractVisibilityRulesFromJS(js) - if findRule(rules, "sortable") != nil { + rules, _ := extractVisibilityRulesFromJS(js) + if findRule(rules, "sortable") != nil && findRule(rules, "sortable").ListPropertyKey == "" { t.Error("nested column hide must not produce a top-level rule") } - if stats.SkippedNested == 0 { - t.Error("expected the nested hide to be counted as skipped") + r := findRule(rules, "sortable") + if r == nil { + t.Fatalf("nested column hide not lifted at all; got %+v", rules) + } + if r.ListPropertyKey != "columns" { + t.Errorf("listPropertyKey = %q, want columns", r.ListPropertyKey) + } + if r.HiddenWhen == nil || r.HiddenWhen.PropertyKey != "columnsSortable" || r.HiddenWhen.Operator != "falsy" { + t.Errorf("condition = %+v, want columnsSortable falsy", r.HiddenWhen) + } + // `columnsSortable` is read off the GRID, so the condition stays widget-scoped. + if r.HiddenWhen.Scope != "" { + t.Errorf("scope = %q, want widget scope", r.HiddenWhen.Scope) } } diff --git a/mdl/executor/editorconfig_nested_test.go b/mdl/executor/editorconfig_nested_test.go new file mode 100644 index 000000000..da291fe6e --- /dev/null +++ b/mdl/executor/editorconfig_nested_test.go @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/types" +) + +// accordionGetProperties is the getProperties body of the stock Accordion's +// compiled editorConfig.js (com.mendix.widget.web.Accordion 2.3.4, shipped with +// every Mendix 11 app), trimmed to the hide calls and kept minified — the +// extractor reads minified JS, so a prettified fixture would not exercise it. +// +// It is the case behind upstream #931: the widget hides its whole State group +// when `collapsible` is off, and hides `initiallyCollapsed` unless the GROUP's +// own `initialCollapsedState` is "dynamic". +const accordionGetProperties = `exports.getProperties=function(e,r,t){` + + `return e.groups.forEach((function(n,o){` + + `"text"===n.headerRenderMode?(p.hidePropertyIn(r,e,"groups",o,"headerContent"),` + + `e.advancedMode||"web"!==t||p.hidePropertyIn(r,e,"groups",o,"headerHeading")):` + + `(p.hidePropertyIn(r,e,"groups",o,"headerText"),p.hidePropertyIn(r,e,"groups",o,"headerHeading")),` + + `"dynamic"!==n.initialCollapsedState&&p.hidePropertyIn(r,e,"groups",o,"initiallyCollapsed"),` + + `(e.advancedMode||"web"!==t)&&e.collapsible||p.hideNestedPropertiesIn(r,e,"groups",o,` + + `["collapsed","onToggleCollapsed","initialCollapsedState","initiallyCollapsed"])})),` + + `e.collapsible||p.hidePropertiesIn(r,e,["expandBehavior","animate"]),r}` + +func findNestedRule(rules []types.WidgetVisibilityRule, listKey, propertyKey, condKey string) *types.WidgetVisibilityRule { + for i := range rules { + r := rules[i] + if r.ListPropertyKey == listKey && r.PropertyKey == propertyKey && + r.HiddenWhen != nil && r.HiddenWhen.PropertyKey == condKey { + return &rules[i] + } + } + return nil +} + +// The nested hides are the ones #931 turns on: without them mxcli reported four +// harmless warnings (all on properties left at their defaults) and said nothing +// about the two values that actually fail the build. +func TestExtractNestedHideRules(t *testing.T) { + rules, stats := extractVisibilityRulesFromJS(accordionGetProperties) + + for _, key := range []string{"initialCollapsedState", "initiallyCollapsed", "collapsed", "onToggleCollapsed"} { + r := findNestedRule(rules, "groups", key, "collapsible") + if r == nil { + t.Fatalf("no rule hiding groups/%s when collapsible is falsy; got %+v", key, rules) + } + if r.HiddenWhen.Operator != "falsy" { + t.Errorf("groups/%s: operator = %q, want falsy", key, r.HiddenWhen.Operator) + } + if r.HiddenWhen.Scope != "" { + t.Errorf("groups/%s: scope = %q, want widget scope (collapsible is a widget property)", key, r.HiddenWhen.Scope) + } + } + + // The top-level hides must still be lifted — this is the pre-#931 behaviour. + if r := findNestedRule(rules, "", "expandBehavior", "collapsible"); r == nil { + t.Errorf("top-level expandBehavior rule lost; got %+v", rules) + } + + if stats.Recognized == 0 { + t.Error("no hide call recognized at all") + } +} + +// A nested condition can name the ITEM rather than the widget. Reading +// `item.initialCollapsedState` as a widget property looks up an absent key and +// reports nothing — which is exactly how the CE0463 for a non-default +// `initiallyCollapsed` stayed silent even once the nested rules existed. +func TestExtractNestedRuleItemScope(t *testing.T) { + rules, _ := extractVisibilityRulesFromJS(accordionGetProperties) + + r := findNestedRule(rules, "groups", "initiallyCollapsed", "initialCollapsedState") + if r == nil { + t.Fatalf("no rule hiding groups/initiallyCollapsed on initialCollapsedState; got %+v", rules) + } + if r.HiddenWhen.Scope != types.ConditionScopeItem { + t.Errorf("scope = %q, want %q — the condition reads the group, not the widget", + r.HiddenWhen.Scope, types.ConditionScopeItem) + } + if r.HiddenWhen.Operator != "ne" || r.HiddenWhen.Value != "dynamic" { + t.Errorf("condition = %s %q, want ne \"dynamic\"", r.HiddenWhen.Operator, r.HiddenWhen.Value) + } +} + +// A comparison guard obeys the connector's polarity. `"text" === mode ? hide(A) : +// hide(B)` hides A when the mode IS "text" and B when it is NOT; recording both +// as `eq` marks headerText hidden in the one configuration where it is used. +func TestExtractComparisonGuardPolarity(t *testing.T) { + rules, _ := extractVisibilityRulesFromJS(accordionGetProperties) + + then := findNestedRule(rules, "groups", "headerContent", "headerRenderMode") + if then == nil { + t.Fatalf("no rule for groups/headerContent; got %+v", rules) + } + if then.HiddenWhen.Operator != "eq" { + t.Errorf("headerContent: operator = %q, want eq (hidden in the THEN branch)", then.HiddenWhen.Operator) + } + + els := findNestedRule(rules, "groups", "headerText", "headerRenderMode") + if els == nil { + t.Fatalf("no rule for groups/headerText; got %+v", rules) + } + if els.HiddenWhen.Operator != "ne" { + t.Errorf("headerText: operator = %q, want ne (hidden in the ELSE branch)", els.HiddenWhen.Operator) + } +} + +// `X && Y || hide(...)` hides when `X && Y` is falsy, so "hide when Y falsy" is +// implied whatever X is. `X && Y && hide(...)` needs BOTH truthy, which one +// condition cannot express — emitting a rule there would over-fire. +func TestConjunctGuardOnlyLiftedForFalsyConnector(t *testing.T) { + falsyForm := `f=function(e,r,t){return (e.a||"web"!==t)&&e.b||p.hidePropertiesIn(r,e,["k"]),r}` + rules, _ := extractVisibilityRulesFromJS(falsyForm) + r := findNestedRule(rules, "", "k", "b") + if r == nil { + t.Fatalf("`X && Y || hide` should yield 'hide when Y falsy'; got %+v", rules) + } + if r.HiddenWhen.Operator != "falsy" { + t.Errorf("operator = %q, want falsy", r.HiddenWhen.Operator) + } + + truthyForm := `f=function(e,r,t){return (e.a||"web"!==t)&&e.b&&p.hidePropertiesIn(r,e,["k"]),r}` + rules, _ = extractVisibilityRulesFromJS(truthyForm) + if r := findNestedRule(rules, "", "k", "b"); r != nil { + t.Errorf("`X && Y && hide` must yield no rule (needs both operands); got %+v", *r.HiddenWhen) + } +} diff --git a/mdl/executor/split_indentation_test.go b/mdl/executor/split_indentation_test.go new file mode 100644 index 000000000..4b0d06042 --- /dev/null +++ b/mdl/executor/split_indentation_test.go @@ -0,0 +1,140 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// indentOf returns the number of leading spaces on the first line containing want. +func indentOf(t *testing.T, lines []string, want string) int { + t.Helper() + for _, line := range lines { + if strings.Contains(line, want) { + return len(line) - len(strings.TrimLeft(line, " ")) + } + } + t.Fatalf("no line containing %q in:\n%s", want, strings.Join(lines, "\n")) + return -1 +} + +func renderMicroflowBody(t *testing.T, src string) []string { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + mf, ok := prog.Statements[0].(*ast.CreateMicroflowStmt) + if !ok { + t.Fatalf("statement 0: got %T, want *ast.CreateMicroflowStmt", prog.Statements[0]) + } + var lines []string + for _, stmt := range mf.Body { + lines = append(lines, microflowStatementToMDL(nil, stmt, 1)...) + } + return lines +} + +// Both splits used to render a branch body at the SAME column as its branch +// keyword, while `if` indented correctly. The result was unreadable once +// anything nested: a nested `if`'s `else` landed exactly where a reader expects +// a case branch, in output where `else` on a `case` is an MDL008 error. (#913) +// +// Reverting either `indent+2` in cmd_diff_mdl.go fails this test. +func TestSplitBranchBodiesIndentFromTheirBranchKeyword(t *testing.T) { + t.Run("enum split", func(t *testing.T) { + lines := renderMicroflowBody(t, `CREATE MICROFLOW Sample.F ($Status: Enumeration(Sample.Status)) +RETURNS String +BEGIN + case $Status + when Open then + return 'open'; + when (empty) then + return 'unset'; + end case; +END;`) + assertBranchIndent(t, lines, "case $Status", "when Open then", "return 'open';", "end case;") + }) + + t.Run("type split", func(t *testing.T) { + lines := renderMicroflowBody(t, `CREATE MICROFLOW Sample.F ($Animal: Sample.Animal) +RETURNS String +BEGIN + split type $Animal + when Sample.Dog then + return 'woof'; + when (empty) then + return 'none'; + end split; +END;`) + assertBranchIndent(t, lines, "split type $Animal", "when Sample.Dog then", "return 'woof';", "end split;") + }) + + // The reference the two splits must match. If `if` ever changes, the splits + // should follow it rather than this test being relaxed. + t.Run("if is the reference", func(t *testing.T) { + lines := renderMicroflowBody(t, `CREATE MICROFLOW Sample.F ($Flag: Boolean) +RETURNS String +BEGIN + if $Flag then + return 'yes'; + else + return 'no'; + end if; +END;`) + if got, want := indentOf(t, lines, "return 'yes';"), indentOf(t, lines, "if $Flag then")+2; got != want { + t.Errorf("if body indent = %d, want %d (one level in from `if`)", got, want) + } + }) +} + +func assertBranchIndent(t *testing.T, lines []string, opener, branch, body, terminator string) { + t.Helper() + openIndent := indentOf(t, lines, opener) + branchIndent := indentOf(t, lines, branch) + bodyIndent := indentOf(t, lines, body) + endIndent := indentOf(t, lines, terminator) + + if branchIndent != openIndent+2 { + t.Errorf("branch indent = %d, want %d (one level in from %q)", branchIndent, openIndent+2, opener) + } + if bodyIndent != branchIndent+2 { + t.Errorf("branch body indent = %d, want %d (one level in from %q).\n"+ + "A body at the same column as its branch keyword is the #913 defect:\n%s", + bodyIndent, branchIndent+2, branch, strings.Join(lines, "\n")) + } + if endIndent != openIndent { + t.Errorf("terminator indent = %d, want %d (flush with %q)", endIndent, openIndent, opener) + } +} + +// The type split renders the branch keyword and the empty branch in the +// unified spelling. A regression here would reintroduce `case` as a branch +// introducer, which is the overloading #913 reported. +func TestTypeSplitRendersUnifiedSpelling(t *testing.T) { + out := strings.Join(renderMicroflowBody(t, `CREATE MICROFLOW Sample.F ($Animal: Sample.Animal) +RETURNS String +BEGIN + split type $Animal + when Sample.Dog then + return 'woof'; + when (empty) then + return 'none'; + end split; +END;`), "\n") + + for _, want := range []string{"when Sample.Dog then", "when (empty) then"} { + if !strings.Contains(out, want) { + t.Errorf("output is missing %q:\n%s", want, out) + } + } + for _, unwanted := range []string{"case Sample.Dog", "\nelse", " else"} { + if strings.Contains(out, unwanted) { + t.Errorf("output still uses the legacy spelling %q:\n%s", unwanted, out) + } + } +} diff --git a/mdl/executor/validate_microflow.go b/mdl/executor/validate_microflow.go index d124d842e..c1c84092d 100644 --- a/mdl/executor/validate_microflow.go +++ b/mdl/executor/validate_microflow.go @@ -207,6 +207,7 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { } v.walkBody(stmt.ElseBody) case *ast.InheritanceSplitStmt: + v.checkInheritanceSplitSpelling(stmt) for _, c := range stmt.Cases { v.walkBody(c.Body) } @@ -1330,6 +1331,40 @@ func (v *microflowValidator) checkEnumSplitEmptyBranch(stmt *ast.EnumSplitStmt) "A branch may list several values (`when Open, (empty) then …`) if they share a path.") } +// checkInheritanceSplitSpelling warns on the pre-#913 spelling of a type split. +// +// `case X ` used `case` to introduce a BRANCH, while the enumeration +// split (`case $x when V then`) and the caseExpression in MDLSettings.g4 use it +// to introduce the SUBJECT. Two of the three agreed; the type split was the +// outlier, so the word meant two things depending on the statement. +// +// `else` is the worse half. It is not a default branch: it is Mendix's +// `(empty)` outgoing flow, taken when the object is NULL. Measured on mxbuild +// 11.13.0, a split with one `case` and an `else` still fails CE0090 demanding a +// flow for every other subtype AND for the base entity — so the `else` +// contributes nothing to type coverage, which is exactly what its name promises. +// +// A warning, not an error: both spellings build the identical flow, and scripts +// in the wild use the old one. Nothing downstream of this function may branch on +// the spelling flags. +func (v *microflowValidator) checkInheritanceSplitSpelling(stmt *ast.InheritanceSplitStmt) { + if stmt.LegacyCaseKeyword { + v.addViolation("MDL065", linter.SeverityWarning, + fmt.Sprintf("split type '$%s' uses the legacy `case ` branch spelling; "+ + "`case` introduces the subject in every other MDL statement (`case $x when V then`), "+ + "not a branch", stmt.Variable), + "Write `when Module.Entity then …` instead. Both build the identical flow.") + } + if stmt.LegacyElseKeyword { + v.addViolation("MDL065", linter.SeverityWarning, + fmt.Sprintf("split type '$%s' uses `else`, which reads as a default branch and is not one: "+ + "it is Mendix's `(empty)` flow, taken when the object is null. Mendix still requires an "+ + "outgoing flow for every subtype and for the base entity (CE0090)", stmt.Variable), + "Write `when (empty) then …` instead — same flow, accurate name. "+ + "To handle unmatched types, add a `when then …` branch.") + } +} + // knownActivityAnnotations is the set the visitor implements. It is the visitor's // own switch arms, restated: the two are pinned together by // TestKnownAnnotationsMatchTheVisitor, because a name added to one and not the diff --git a/mdl/executor/validate_split_spelling_test.go b/mdl/executor/validate_split_spelling_test.go new file mode 100644 index 000000000..c15a976e3 --- /dev/null +++ b/mdl/executor/validate_split_spelling_test.go @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/linter" +) + +func splitSource(branch, empty string) string { + return `CREATE MICROFLOW Sample.Route ($Animal: Sample.Animal) +RETURNS String +BEGIN + split type $Animal + ` + branch + ` + return 'woof'; + ` + empty + ` + return 'none'; + end split; +END;` +} + +// MDL065 warns on the pre-#913 spelling and stays quiet on the current one. +// Both build the identical flow, so it is a warning: an error would reject +// working scripts. +func TestMDL065_WarnsOnLegacySplitSpelling(t *testing.T) { + tests := []struct { + name string + branch, empty string + wantWarnings int + }{ + {"current spelling is silent", "when Sample.Dog then", "when (empty) then", 0}, + {"legacy branch keyword", "case Sample.Dog", "when (empty) then", 1}, + {"legacy empty branch", "when Sample.Dog then", "else", 1}, + {"both legacy", "case Sample.Dog", "else", 2}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + var got int + for _, v := range checkMicroflowSource(t, splitSource(tc.branch, tc.empty)) { + if v.RuleID != "MDL065" { + continue + } + got++ + if v.Severity != linter.SeverityWarning { + t.Errorf("MDL065 severity = %v, want warning — an error would reject working scripts", + v.Severity) + } + } + if got != tc.wantWarnings { + t.Errorf("MDL065 count = %d, want %d", got, tc.wantWarnings) + } + }) + } +} + +// The `else` diagnostic has to say what `else` actually IS, not just that it is +// deprecated. An author who reads it as a default branch is wrong, and mxbuild +// tells them so only indirectly, via CE0090 on the types they did not cover. +func TestMDL065_ElseWarningExplainsTheEmptySemantics(t *testing.T) { + var msg, hint string + for _, v := range checkMicroflowSource(t, splitSource("when Sample.Dog then", "else")) { + if v.RuleID == "MDL065" { + msg, hint = v.Message, v.Suggestion + } + } + if msg == "" { + t.Fatal("no MDL065 violation for the `else` spelling") + } + for _, want := range []string{"(empty)", "null", "CE0090"} { + if !strings.Contains(msg, want) { + t.Errorf("message does not mention %q — it reads as a rename, not a meaning change:\n%s", want, msg) + } + } + if !strings.Contains(hint, "when (empty) then") { + t.Errorf("suggestion does not name the replacement:\n%s", hint) + } +} + +// A microflow carrying the legacy spelling must still EXECUTE. MDL065 is a +// warning, and `exec`'s pre-flight gate halts only on errors — if this rule +// were ever promoted to an error it would break every script in the wild. +func TestMDL065_DoesNotBlockExecution(t *testing.T) { + violations := checkMicroflowSource(t, splitSource("case Sample.Dog", "else")) + if summary := linter.Summarize(violations); summary.Errors > 0 { + t.Fatalf("legacy split spelling produced %d error(s); exec would refuse the script:\n%+v", + summary.Errors, violations) + } +} diff --git a/mdl/executor/validate_widget_hidden.go b/mdl/executor/validate_widget_hidden.go new file mode 100644 index 000000000..8f49b288d --- /dev/null +++ b/mdl/executor/validate_widget_hidden.go @@ -0,0 +1,311 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "path/filepath" + "strings" + "sync" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/modelsdk/widgets/mpk" +) + +// A pluggable widget's editorConfig.js can hide a property under some +// configurations of the same widget. Studio Pro and mxbuild both evaluate that +// logic, and a HIDDEN property is required to hold its DEFAULT value: a +// non-default one is CE0463 "the definition of this widget has changed", which +// fails `mx check` and makes `mx create-module-package` refuse the module. +// +// Measured on mxbuild 11.13 with the stock Accordion (upstream #931). The widget +// hides its whole State group when `collapsible` is off, so: +// +// collapsible: false + InitialCollapsedState: 'expanded' → CE0463 +// collapsible: false + InitiallyCollapsed: 'false' → CE0463 +// collapsible: false + expandBehavior: multipleExpanded → CE0463 +// collapsible: false + animate: false → CE0463 +// collapsible: TRUE + the same values → 0 errors +// collapsible: false + every value left at its default → 0 errors +// +// So the diagnostic splits on the value, not on the property: an explicitly set +// DEFAULT is merely redundant (warning — Studio Pro ignores it), a non-default +// one does not build (error). + +// hiddenPropertySeverity returns the severity for setting value on a property the +// widget hides, plus the tail of the message explaining the consequence. +// +// def is the property's declared default; when it is unknown ("") the rule stays +// a warning — mxcli will not fail a script on a comparison it could not make. +func hiddenPropertySeverity(value, def string) (linter.Severity, string) { + if def == "" || strings.EqualFold(value, def) { + return linter.SeverityWarning, "the value will be ignored" + } + return linter.SeverityError, fmt.Sprintf( + "a non-default value there fails the build with CE0463 (the default is %q)", def) +} + +// hiddenPropertyViolation builds the MDL-WIDGET10 diagnostic. itemLabel names the +// object-list item a nested rule fired on (e.g. "group `g1`"), empty for a +// top-level property. +func hiddenPropertyViolation(locationPrefix, widgetName, mdlName, itemLabel string, + rule types.WidgetVisibilityRule, value, def string) linter.Violation { + severity, consequence := hiddenPropertySeverity(value, def) + where := "" + if itemLabel != "" { + where = " " + itemLabel + } + scope := "" + if rule.HiddenWhen.Scope == types.ConditionScopeItem && itemLabel != "" { + scope = "its own " + } + return linter.Violation{ + RuleID: "MDL-WIDGET10", + Severity: severity, + Message: fmt.Sprintf( + "%s: widget `%s` (%s)%s property `%s` is hidden when %s`%s` %s — %s", + locationPrefix, widgetName, mdlName, where, rule.PropertyKey, + scope, rule.HiddenWhen.PropertyKey, visibilityCondWord(rule.HiddenWhen), consequence, + ), + } +} + +// validateWidgetItemVisibility applies the nested (object-list) visibility rules +// of parent's widget definition to one of its items — an Accordion GROUP, a +// DataGrid COLUMN, a PopupMenu ITEM. +// +// A nested rule's condition can name either object: `collapsible` belongs to the +// widget, `initialCollapsedState` to the group itself, and the Accordion uses +// both in the same callback. The condition's Scope says which, and each is looked +// up in its own value map — evaluating an item condition against the widget (or +// the reverse) silently reads an absent key and reports nothing. +func validateWidgetItemVisibility(parent *ast.WidgetV3, item *ast.WidgetV3, + mapping *ObjectListMapping, registry *WidgetRegistry, locationPrefix string) []linter.Violation { + if parent == nil || item == nil || mapping == nil { + return nil + } + def := lookupWidgetDef(parent, registry) + if def == nil { + return nil + } + rules := visibilityRulesFor(def, registry) + if len(rules) == 0 { + return nil + } + widgetValues, _ := widgetValueMap(parent, def) + itemValues, itemExplicit := itemValueMap(item, mapping) + defaults := widgetPropertyDefaults(registryProjectPath(registry), def.WidgetID) + + var out []linter.Violation + for _, rule := range rules { + if !rule.Nested() || !strings.EqualFold(rule.ListPropertyKey, mapping.PropertyKey) { + continue + } + if rule.HiddenWhen == nil { + continue + } + if !itemExplicit[strings.ToLower(rule.PropertyKey)] { + continue // the author did not set this sub-property + } + values := widgetValues + if rule.HiddenWhen.Scope == types.ConditionScopeItem { + values = itemValues + } + condVal, known := values[strings.ToLower(rule.HiddenWhen.PropertyKey)] + if !known { + continue // condition value indeterminable — don't guess + } + if !rule.HiddenWhen.Hidden(map[string]string{rule.HiddenWhen.PropertyKey: condVal}) { + continue + } + value := itemValues[strings.ToLower(rule.PropertyKey)] + declared := declaredDefault(defaults, nil, mapping.ItemProperties, mapping.PropertyKey, rule.PropertyKey) + label := fmt.Sprintf("%s `%s`", strings.ToLower(item.Type), item.Name) + out = append(out, hiddenPropertyViolation(locationPrefix, parent.Name, def.MDLName, label, rule, value, declared)) + } + return out +} + +// visibilityRulesFor returns a widget definition's visibility rules, falling back +// to lifting them from the installed .mpk when the def carries none. Mirrors what +// validateWidgetVisibility does for top-level rules. +func visibilityRulesFor(def *WidgetDefinition, registry *WidgetRegistry) []types.WidgetVisibilityRule { + if len(def.PropertyVisibility) > 0 { + return def.PropertyVisibility + } + if path := registryProjectPath(registry); path != "" { + return resolveWidgetVisibilityRules(path, def.WidgetID) + } + return nil +} + +func registryProjectPath(registry *WidgetRegistry) string { + if registry == nil { + return "" + } + return registry.projectPath +} + +// itemValueMap resolves an object-list item's sub-property values (keyed by +// lowercased schema key) and reports which the MDL set explicitly. The item form +// of widgetValueMap. +func itemValueMap(item *ast.WidgetV3, mapping *ObjectListMapping) (values map[string]string, explicit map[string]bool) { + values = map[string]string{} + explicit = map[string]bool{} + + for _, m := range mapping.ItemProperties { + key := strings.ToLower(m.PropertyKey) + val, set := "", false + if m.Source != "" { + if v, ok := lookupWidgetProp(item, m.Source); ok { + val, set = v, true + } + } + if !set { + for _, a := range m.MdlAliases { + if v, ok := lookupWidgetProp(item, a); ok { + val, set = v, true + break + } + } + } + if !set { + if v, ok := lookupWidgetProp(item, m.PropertyKey); ok { + val, set = v, true + } + } + if set { + explicit[key] = true + } else if m.Default != "" { + val = m.Default + } else if m.Value != "" { + val = m.Value + } + if val != "" { + if m.Operation == "selection" { + val = canonicalSelection(val) + } + values[key] = val + } + } + // Sub-properties named after a schema key with no mapping entry. + for k, raw := range item.Properties { + lk := strings.ToLower(k) + if _, ok := values[lk]; ok { + continue + } + if s := stringifyPropValue(raw); s != "" { + values[lk] = s + explicit[lk] = true + } + } + return values, explicit +} + +// declaredDefault returns a property's default: the .mpk's declared value when +// the project is reachable, otherwise the definition's own mapping default. +// +// The mapping is the weaker source — it carries a default only for the +// primitive-typed properties (an expression property like the Accordion group's +// `initiallyCollapsed` has none) — but it is the one available when there is no +// project to read, which keeps the severity split working for `mxcli check` +// without `-p` and for in-memory tests. An unknown default keeps the diagnostic a +// warning; it never invents one. +func declaredDefault(mpkDefaults map[string]string, mappings []PropertyMapping, + itemMappings []ItemPropertyMapping, listKey, propertyKey string) string { + if v := mpkDefaults[defaultsKey(listKey, propertyKey)]; v != "" { + return v + } + if listKey == "" { + for _, m := range mappings { + if strings.EqualFold(m.PropertyKey, propertyKey) { + return firstNonEmpty(m.Default, m.Value) + } + } + return "" + } + for _, m := range itemMappings { + if strings.EqualFold(m.PropertyKey, propertyKey) { + return firstNonEmpty(m.Default, m.Value) + } + } + return "" +} + +func firstNonEmpty(vals ...string) string { + for _, v := range vals { + if v != "" { + return v + } + } + return "" +} + +// defaultsKey builds the lookup key used by widgetPropertyDefaults: a bare +// lowercased property key for a top-level property, "list/key" for an item +// sub-property. +func defaultsKey(listKey, propertyKey string) string { + if listKey == "" { + return strings.ToLower(propertyKey) + } + return strings.ToLower(listKey) + "/" + strings.ToLower(propertyKey) +} + +var ( + widgetDefaultsCache = map[string]map[string]string{} + widgetDefaultsCacheMu sync.Mutex +) + +// widgetPropertyDefaults returns the DECLARED defaults of a widget's properties, +// read from the installed .mpk — top-level keys plus "list/key" entries for +// object-list item sub-properties. +// +// The .mpk is the only complete source: the executor's PropertyMapping carries a +// default only for the primitive-typed properties, so an expression property like +// the Accordion group's `initiallyCollapsed` (default "true") has none there — +// and that is one of the two properties #931 turns on. Returns an empty map when +// the widget or its package cannot be found, which downgrades every diagnostic to +// a warning rather than guessing a default. +func widgetPropertyDefaults(projectPath, widgetID string) map[string]string { + if projectPath == "" || widgetID == "" { + return nil + } + cacheKey := projectPath + "\x00" + widgetID + widgetDefaultsCacheMu.Lock() + if d, ok := widgetDefaultsCache[cacheKey]; ok { + widgetDefaultsCacheMu.Unlock() + return d + } + widgetDefaultsCacheMu.Unlock() + + defaults := map[string]string{} + projectDir := projectPath + if strings.EqualFold(filepath.Ext(projectDir), ".mpr") { + projectDir = filepath.Dir(projectDir) + } + if mpkPath, err := mpk.FindMPK(projectDir, widgetID); err == nil && mpkPath != "" { + if wd, err := mpk.ParseMPKForWidget(mpkPath, widgetID); err == nil && wd != nil { + collectPropertyDefaults(defaults, "", wd.Properties) + } + } + + widgetDefaultsCacheMu.Lock() + widgetDefaultsCache[cacheKey] = defaults + widgetDefaultsCacheMu.Unlock() + return defaults +} + +// collectPropertyDefaults walks a widget's property tree, recording each declared +// default one level deep (object-list item sub-properties). +func collectPropertyDefaults(out map[string]string, listKey string, props []mpk.PropertyDef) { + for _, p := range props { + if p.DefaultValue != "" { + out[defaultsKey(listKey, p.Key)] = p.DefaultValue + } + if len(p.Children) > 0 && listKey == "" { + collectPropertyDefaults(out, p.Key, p.Children) + } + } +} diff --git a/mdl/executor/validate_widget_hidden_test.go b/mdl/executor/validate_widget_hidden_test.go new file mode 100644 index 000000000..46561ce2d --- /dev/null +++ b/mdl/executor/validate_widget_hidden_test.go @@ -0,0 +1,168 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" + "github.com/mendixlabs/mxcli/mdl/types" +) + +// accordionLikeRegistry mirrors the shape that matters for #931: a widget with an +// object list whose item sub-properties are hidden by two different conditions — +// one read off the WIDGET (collapsible), one off the ITEM (initialCollapsedState). +func accordionLikeRegistry() *WidgetRegistry { + def := &WidgetDefinition{ + WidgetID: "com.mendix.widget.web.accordion.Accordion", + MDLName: "ACCORDION", + PropertyMappings: []PropertyMapping{ + {PropertyKey: "collapsible", Operation: "primitive", Value: "true"}, + {PropertyKey: "expandBehavior", Operation: "primitive", Value: "singleExpanded"}, + }, + ObjectLists: []ObjectListMapping{{ + PropertyKey: "groups", + MDLContainer: "GROUP", + ItemProperties: []ItemPropertyMapping{ + {PropertyKey: "initialCollapsedState", Operation: "primitive", Value: "collapsed"}, + {PropertyKey: "initiallyCollapsed", Operation: "expression", Default: "true"}, + }, + }}, + PropertyVisibility: []types.WidgetVisibilityRule{ + {PropertyKey: "expandBehavior", HiddenWhen: &types.WidgetVisibilityCondition{ + PropertyKey: "collapsible", Operator: "falsy", + }}, + {PropertyKey: "initialCollapsedState", ListPropertyKey: "groups", + HiddenWhen: &types.WidgetVisibilityCondition{PropertyKey: "collapsible", Operator: "falsy"}}, + {PropertyKey: "initiallyCollapsed", ListPropertyKey: "groups", + HiddenWhen: &types.WidgetVisibilityCondition{ + PropertyKey: "initialCollapsedState", Operator: "ne", Value: "dynamic", + Scope: types.ConditionScopeItem, + }}, + }, + } + return &WidgetRegistry{byMDLName: map[string]*WidgetDefinition{"ACCORDION": def}} +} + +func accordionWidget(collapsible string, groupProps map[string]any) (*ast.WidgetV3, *ast.WidgetV3) { + group := &ast.WidgetV3{Type: "GROUP", Name: "g1", Properties: groupProps} + return &ast.WidgetV3{ + Type: "accordion", + Name: "acc1", + Properties: map[string]any{"collapsible": collapsible}, + Children: []*ast.WidgetV3{group}, + }, group +} + +func severities(vs []linter.Violation) (errors, warnings int) { + for _, v := range vs { + switch v.Severity { + case linter.SeverityError: + errors++ + case linter.SeverityWarning: + warnings++ + } + } + return errors, warnings +} + +// The value decides the severity, not the property. Measured on mxbuild 11.13: +// `collapsible: false` with the group's initialCollapsedState left at "collapsed" +// checks clean, and with "expanded" it is CE0463 — so the same hidden property is +// a warning in one script and a build failure in the other. +func TestHiddenItemPropertySeverityFollowsTheValue(t *testing.T) { + registry := accordionLikeRegistry() + mapping := ®istry.byMDLName["ACCORDION"].ObjectLists[0] + + nonDefault, group := accordionWidget("false", map[string]any{"InitialCollapsedState": "expanded"}) + vs := validateWidgetItemVisibility(nonDefault, group, mapping, registry, "page P") + errs, warns := severities(vs) + if errs != 1 || warns != 0 { + t.Fatalf("non-default on a hidden item property → %d errors %d warnings %+v, want 1 error", errs, warns, vs) + } + if !strings.Contains(vs[0].Message, "CE0463") || !strings.Contains(vs[0].Message, `"collapsed"`) { + t.Errorf("message should name the consequence and the default, got %q", vs[0].Message) + } + if !strings.Contains(vs[0].Message, "group `g1`") { + t.Errorf("message should name the item it fired on, got %q", vs[0].Message) + } + + atDefault, group := accordionWidget("false", map[string]any{"InitialCollapsedState": "collapsed"}) + vs = validateWidgetItemVisibility(atDefault, group, mapping, registry, "page P") + errs, warns = severities(vs) + if errs != 0 || warns != 1 { + t.Fatalf("default value on a hidden item property → %d errors %d warnings %+v, want 1 warning", errs, warns, vs) + } +} + +// A visible property is not flagged at all: with `collapsible` on, the whole +// State group is shown and any value is legal (measured — 0 errors on mxbuild). +func TestVisibleItemPropertyNotFlagged(t *testing.T) { + registry := accordionLikeRegistry() + mapping := ®istry.byMDLName["ACCORDION"].ObjectLists[0] + + widget, group := accordionWidget("true", map[string]any{"InitialCollapsedState": "expanded"}) + for _, v := range validateWidgetItemVisibility(widget, group, mapping, registry, "page P") { + if strings.Contains(v.Message, "initialCollapsedState") { + t.Errorf("collapsible on → initialCollapsedState is visible, got %q", v.Message) + } + } +} + +// The item-scoped condition reads the GROUP, not the widget. Evaluating it +// against the widget finds no `initialCollapsedState` key and reports nothing — +// which is how this CE0463 stayed silent even once the nested rules existed. +func TestItemScopedConditionReadsTheItem(t *testing.T) { + registry := accordionLikeRegistry() + mapping := ®istry.byMDLName["ACCORDION"].ObjectLists[0] + + widget, group := accordionWidget("true", map[string]any{ + "InitialCollapsedState": "expanded", // ≠ dynamic ⇒ initiallyCollapsed hidden + "InitiallyCollapsed": "false", // ≠ its default "true" + }) + vs := validateWidgetItemVisibility(widget, group, mapping, registry, "page P") + errs, _ := severities(vs) + if errs != 1 { + t.Fatalf("got %d errors %+v, want 1 for initiallyCollapsed", errs, vs) + } + if !strings.Contains(vs[0].Message, "initiallyCollapsed") || + !strings.Contains(vs[0].Message, "its own `initialCollapsedState`") { + t.Errorf("message should attribute the condition to the item, got %q", vs[0].Message) + } + + // dynamic ⇒ the property is shown, so the same value is fine. + ok, okGroup := accordionWidget("true", map[string]any{ + "InitialCollapsedState": "dynamic", + "InitiallyCollapsed": "false", + }) + if vs := validateWidgetItemVisibility(ok, okGroup, mapping, registry, "page P"); len(vs) != 0 { + t.Errorf("initialCollapsedState:dynamic → initiallyCollapsed is visible, got %+v", vs) + } +} + +// A top-level hidden property splits the same way — this is the pre-existing +// MDL-WIDGET10 path, which reported every case as a warning. `collapsible: false` +// with a non-default expandBehavior is CE0463 on mxbuild. +func TestHiddenTopLevelPropertySeverityFollowsTheValue(t *testing.T) { + registry := accordionLikeRegistry() + + nonDefault := &ast.WidgetV3{Type: "accordion", Name: "acc1", Properties: map[string]any{ + "collapsible": "false", + "expandBehavior": "multipleExpanded", + }} + errs, warns := severities(validateWidgetVisibility(nonDefault, registry, "page P")) + if errs != 1 || warns != 0 { + t.Errorf("non-default hidden top-level property → %d errors %d warnings, want 1 error", errs, warns) + } + + atDefault := &ast.WidgetV3{Type: "accordion", Name: "acc1", Properties: map[string]any{ + "collapsible": "false", + "expandBehavior": "singleExpanded", + }} + errs, warns = severities(validateWidgetVisibility(atDefault, registry, "page P")) + if errs != 0 || warns != 1 { + t.Errorf("default hidden top-level property → %d errors %d warnings, want 1 warning", errs, warns) + } +} diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index 496b59b55..b02f7e293 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -97,7 +97,7 @@ func ValidateWidgetPropertiesForStatement(stmt ast.Statement, registry *WidgetRe // validateWidgetTree recursively walks the AST widget tree and validates // pluggable widgets it encounters. func validateWidgetTree(widgets []*ast.WidgetV3, registry *WidgetRegistry, locationPrefix string) []linter.Violation { - return validateWidgetTreeIn(widgets, registry, locationPrefix, nil) + return validateWidgetTreeIn(widgets, registry, locationPrefix, nil, nil) } // validateWidgetTreeIn is validateWidgetTree with the *parent* widget's @@ -108,7 +108,7 @@ func validateWidgetTree(widgets []*ast.WidgetV3, registry *WidgetRegistry, locat // must be exempt from the MDL-WIDGET07 "unrecognized property, silently dropped" // warning. When the parent mapping is known, the child's enumeration // sub-properties are validated against their member keys (MDL-WIDGET08). (9a) -func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, locationPrefix string, parentObjectLists map[string]*ObjectListMapping) []linter.Violation { +func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, locationPrefix string, parentObjectLists map[string]*ObjectListMapping, parent *ast.WidgetV3) []linter.Violation { var out []linter.Violation for _, w := range widgets { if w == nil { @@ -139,11 +139,14 @@ func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, loc } if mapping != nil { out = append(out, validateObjectListItemEnums(w, mapping, locationPrefix)...) + // #931: a sub-property the widget's editorConfig hides must hold its + // default; a non-default value there is CE0463. + out = append(out, validateWidgetItemVisibility(parent, w, mapping, registry, locationPrefix)...) } // Reported once per grid, not once per column — see the rule's comment. out = append(out, validateDataGrid2ColumnNames(w, locationPrefix)...) if len(w.Children) > 0 { - out = append(out, validateWidgetTreeIn(w.Children, registry, locationPrefix, objectListMappingSet(def))...) + out = append(out, validateWidgetTreeIn(w.Children, registry, locationPrefix, objectListMappingSet(def), w)...) } } out = append(out, validateConsecutiveDynamicText(widgets, locationPrefix)...) @@ -247,9 +250,11 @@ func validateConsecutiveDynamicText(siblings []*ast.WidgetV3, locationPrefix str return nil } -// validateWidgetVisibility warns (MDL-WIDGET10) when a property the user set on a -// pluggable widget is hidden under that widget's current configuration — the -// widget's editorConfig.js suppresses it, so Studio Pro ignores the written value. +// validateWidgetVisibility flags (MDL-WIDGET10) a property the user set on a +// pluggable widget that is hidden under that widget's current configuration — the +// widget's editorConfig.js suppresses it. A value equal to the property's default +// is merely ignored (warning); a non-default one is CE0463 and fails the build +// (error) — see hiddenPropertySeverity. // Rules come from the widget's .def.json (propertyVisibility); for built-in // widgets whose def carries none they are lifted on the fly from the installed // .mpk's editorConfig.js (#574). Conservative by design: it only warns when the @@ -260,18 +265,18 @@ func validateWidgetVisibility(w *ast.WidgetV3, registry *WidgetRegistry, locatio if def == nil { return nil } - rules := def.PropertyVisibility - if len(rules) == 0 && registry != nil && registry.projectPath != "" { - rules = resolveWidgetVisibilityRules(registry.projectPath, def.WidgetID) - } + rules := visibilityRulesFor(def, registry) if len(rules) == 0 { return nil } values, explicit := widgetValueMap(w, def) + defaults := widgetPropertyDefaults(registryProjectPath(registry), def.WidgetID) var out []linter.Violation for _, rule := range rules { - if rule.HiddenWhen == nil { + // A nested rule is about a property of an object-list ITEM, not of the + // widget — validateWidgetItemVisibility evaluates those, against the item. + if rule.Nested() || rule.HiddenWhen == nil { continue } if !explicit[strings.ToLower(rule.PropertyKey)] { @@ -284,15 +289,9 @@ func validateWidgetVisibility(w *ast.WidgetV3, registry *WidgetRegistry, locatio if !rule.HiddenWhen.Hidden(map[string]string{rule.HiddenWhen.PropertyKey: condVal}) { continue } - out = append(out, linter.Violation{ - RuleID: "MDL-WIDGET10", - Severity: linter.SeverityWarning, - Message: fmt.Sprintf( - "%s: widget `%s` (%s) property `%s` is hidden when `%s` %s — the value will be ignored", - locationPrefix, w.Name, def.MDLName, rule.PropertyKey, - rule.HiddenWhen.PropertyKey, visibilityCondWord(rule.HiddenWhen), - ), - }) + out = append(out, hiddenPropertyViolation(locationPrefix, w.Name, def.MDLName, "", rule, + values[strings.ToLower(rule.PropertyKey)], + declaredDefault(defaults, def.PropertyMappings, nil, "", rule.PropertyKey))) } return out } diff --git a/mdl/executor/widget_engine.go b/mdl/executor/widget_engine.go index 3a59ac4c8..fbd4f0278 100644 --- a/mdl/executor/widget_engine.go +++ b/mdl/executor/widget_engine.go @@ -84,7 +84,14 @@ const defaultSlotContainer = "template" // matched the bare keys only, so `OnChange:` on a combobox emitted no // mapping and was dropped with no error (FINDINGS #14). Bump forces // regeneration so existing projects pick up the suffixed slots. -const WidgetDefGeneratorVersion = 15 +// 16 — lift the object-list-nested hide rules from editorConfig.js, and fix the +// polarity of a comparison guard read from a `||` connector or a ternary's +// else branch (it was recorded as `eq` where the code means `ne`). Without +// the nested rules an Accordion group's `initialCollapsedState` could be set +// to a value the widget hides, which mxbuild rejects with CE0463 and which +// makes `mx create-module-package` refuse the module (upstream #931). Bump +// forces existing projects to regenerate their defs with both. +const WidgetDefGeneratorVersion = 16 // WidgetDefinition describes how to construct a pluggable widget from MDL syntax. // Loaded from embedded JSON definition files (*.def.json). diff --git a/mdl/grammar/MDLParser.g4 b/mdl/grammar/MDLParser.g4 index 5d9714215..41a57e7fd 100644 --- a/mdl/grammar/MDLParser.g4 +++ b/mdl/grammar/MDLParser.g4 @@ -230,6 +230,7 @@ alterPageOperation : alterPageSet SEMICOLON? | alterPageInsert SEMICOLON? | alterPageDrop SEMICOLON? + | alterPageDropTemplate SEMICOLON? | alterPageReplace SEMICOLON? | alterPageAddVariable SEMICOLON? | alterPageDropVariable SEMICOLON? @@ -265,6 +266,20 @@ alterPageDrop : DROP WIDGET widgetRef (COMMA widgetRef)* ; +// DROP TEMPLATE FOR Module.Specialization IN listViewName +// +// A List View specialization template has no name — the entity it renders is +// what identifies it — so it cannot be reached through widgetRef like every +// other DROP target. Naming the list view is required, not optional: one page +// can hold two list views with a template for the same entity. +// +// There is no matching INSERT TEMPLATE. Adding one is +// `INSERT INTO { template for Module.Entity { ... } }`, which reuses +// the same block as CREATE PAGE, so a template has one spelling everywhere. +alterPageDropTemplate + : DROP TEMPLATE FOR qualifiedName IN widgetRef + ; + alterPageReplace : REPLACE widgetRef WITH LBRACE pageBodyV3 RBRACE ; @@ -305,7 +320,7 @@ navMenuItemDef // built from the same items, so this reuses navMenuItemDef rather than defining a // second item syntax. createMenuStatement - : MENU_KW qualifiedName LPAREN navMenuItemDef* RPAREN + : MENU_KW qualifiedName (FOLDER STRING_LITERAL)? LPAREN navMenuItemDef* RPAREN ; dropStatement @@ -380,15 +395,70 @@ renameTarget * ```mdl * MOVE ENUMERATION MyModule.OrderStatus TO OtherModule; * ``` + * + * @example Move an import mapping or JSON structure + * ```mdl + * MOVE IMPORT MAPPING MyModule.IMM_Order TO FOLDER 'Private/Import mappings'; + * MOVE JSON STRUCTURE MyModule.JSON_Order TO FOLDER 'Private/JSON structures'; + * ``` */ moveStatement - : MOVE (PAGE | MICROFLOW | SNIPPET | NANOFLOW | ENUMERATION | CONSTANT | DATABASE CONNECTION | JAVA ACTION | ODATA SERVICE) qualifiedName TO FOLDER STRING_LITERAL (IN (qualifiedName | IDENTIFIER))? - | MOVE (PAGE | MICROFLOW | SNIPPET | NANOFLOW | ENUMERATION | CONSTANT | DATABASE CONNECTION | JAVA ACTION | ODATA SERVICE) qualifiedName TO (qualifiedName | IDENTIFIER) + : MOVE moveDocumentType qualifiedName TO FOLDER STRING_LITERAL (IN (qualifiedName | IDENTIFIER))? + | MOVE moveDocumentType qualifiedName TO (qualifiedName | IDENTIFIER) | MOVE ENTITY qualifiedName TO (qualifiedName | IDENTIFIER) | MOVE FOLDER qualifiedName TO FOLDER STRING_LITERAL (IN (qualifiedName | IDENTIFIER))? | MOVE FOLDER qualifiedName TO (qualifiedName | IDENTIFIER) ; +/** + * The document types MOVE accepts — every top-level document, spelled as + * DESCRIBE spells it. + * + * This is a rule rather than an inline alternation so that MOVE FOLDER can be + * told from a document move by ONE check (`moveDocumentType` present or not) + * instead of by a hand-maintained negation of every doctype keyword. That + * negation is the trap mxcli-formula1 #32 flagged: each keyword added to the + * move rule had to be added to the folder discriminator too, and forgetting one + * silently turns `MOVE FOLDER …` into a document move. + * + * ENTITY is deliberately absent: an entity is not a unit, it lives inside a + * domain model, and its move converts associations rather than reparenting a + * row — so it keeps its own alternative and its own handler. + */ +moveDocumentType + : PAGE + | MICROFLOW + | NANOFLOW + | SNIPPET + | BUILDING BLOCK + | LAYOUT + | MENU_KW + | ENUMERATION + | CONSTANT + | WORKFLOW + | QUEUE + | SCHEDULED EVENT + | REGULAR EXPRESSION + | JSON STRUCTURE + | IMPORT MAPPING + | EXPORT MAPPING + | JAVA ACTION + | JAVASCRIPT ACTION + | DATABASE CONNECTION + | DATA TRANSFORMER + | IMAGE COLLECTION + | ICON COLLECTION + | REST CLIENT + | PUBLISHED REST SERVICE + | ODATA CLIENT + | ODATA SERVICE + | BUSINESS EVENT SERVICE + | MODEL + | AGENT + | KNOWLEDGE BASE + | CONSUMED MCP SERVICE + ; + // ============================================================================= // SECURITY STATEMENTS (dispatch list — rules in MDLSecurity.g4) // ============================================================================= diff --git a/mdl/grammar/domains/MDLAgent.g4 b/mdl/grammar/domains/MDLAgent.g4 index c142a6240..fb3cfc61b 100644 --- a/mdl/grammar/domains/MDLAgent.g4 +++ b/mdl/grammar/domains/MDLAgent.g4 @@ -16,6 +16,7 @@ options { tokenVocab = MDLLexer; } // ); createModelStatement : MODEL qualifiedName + (FOLDER STRING_LITERAL)? LPAREN modelProperty (COMMA modelProperty)* RPAREN ; @@ -48,6 +49,7 @@ variableDef // ); createConsumedMCPServiceStatement : CONSUMED MCP SERVICE qualifiedName + (FOLDER STRING_LITERAL)? LPAREN modelProperty (COMMA modelProperty)* RPAREN ; @@ -60,6 +62,7 @@ createConsumedMCPServiceStatement // ); createKnowledgeBaseStatement : KNOWLEDGE BASE qualifiedName + (FOLDER STRING_LITERAL)? LPAREN modelProperty (COMMA modelProperty)* RPAREN ; @@ -76,6 +79,7 @@ createKnowledgeBaseStatement // ; createAgentStatement : AGENT qualifiedName + (FOLDER STRING_LITERAL)? LPAREN modelProperty (COMMA modelProperty)* RPAREN agentBody? ; diff --git a/mdl/grammar/domains/MDLCatalog.g4 b/mdl/grammar/domains/MDLCatalog.g4 index 815f5e99e..b24462e82 100644 --- a/mdl/grammar/domains/MDLCatalog.g4 +++ b/mdl/grammar/domains/MDLCatalog.g4 @@ -70,6 +70,12 @@ showStatement | showOrList USER ROLES | showOrList DEMO USERS | showOrList ACCESS ON qualifiedName + // ENTITY is spelled out as well as implied by the bare form above: every + // other document kind names itself here, and the generated project CLAUDE.md + // documents `SHOW ACCESS ON MICROFLOW|PAGE|ENTITY`. Without this alternative + // `ENTITY` is swallowed by qualifiedName (it is in the `keyword` rule), so + // the statement ends at `ENTITY` and the real name is extraneous input. + | showOrList ACCESS ON ENTITY qualifiedName | showOrList ACCESS ON MICROFLOW qualifiedName | showOrList ACCESS ON PAGE qualifiedName | showOrList ACCESS ON WORKFLOW qualifiedName diff --git a/mdl/grammar/domains/MDLDomainModel.g4 b/mdl/grammar/domains/MDLDomainModel.g4 index 8975903b3..31e9f62e0 100644 --- a/mdl/grammar/domains/MDLDomainModel.g4 +++ b/mdl/grammar/domains/MDLDomainModel.g4 @@ -338,7 +338,7 @@ enumerationOption * quoted expression. */ createQueueStatement - : QUEUE qualifiedName queueBody? + : QUEUE qualifiedName (FOLDER STRING_LITERAL)? queueBody? ; queueBody @@ -358,7 +358,7 @@ queueProperty // document rather than a string on the rule. createRegularExpressionStatement - : REGULAR EXPRESSION qualifiedName regularExpressionBody? + : REGULAR EXPRESSION qualifiedName (FOLDER STRING_LITERAL)? regularExpressionBody? ; regularExpressionBody @@ -380,7 +380,7 @@ regularExpressionProperty // not belong to the chosen repeat. createScheduledEventStatement - : SCHEDULED EVENT qualifiedName scheduledEventBody? + : SCHEDULED EVENT qualifiedName (FOLDER STRING_LITERAL)? scheduledEventBody? ; scheduledEventBody @@ -396,7 +396,7 @@ scheduledEventProperty // ============================================================================= createImageCollectionStatement - : IMAGE COLLECTION qualifiedName imageCollectionOptions? imageCollectionBody? + : IMAGE COLLECTION qualifiedName (FOLDER STRING_LITERAL)? imageCollectionOptions? imageCollectionBody? ; imageCollectionOptions @@ -441,6 +441,7 @@ customNameMapping /** * CREATE IMPORT MAPPING Module.Name + * FOLDER 'Private/Import mappings' * WITH JSON STRUCTURE Module.JsonStructure * { * CREATE Module.Entity { @@ -451,6 +452,7 @@ customNameMapping */ createImportMappingStatement : IMPORT MAPPING qualifiedName + (FOLDER STRING_LITERAL)? importMappingWithClause? LBRACE importMappingRootElement RBRACE ; @@ -495,6 +497,7 @@ importMappingObjectHandling /** * CREATE EXPORT MAPPING Module.Name + * FOLDER 'Private/Export mappings' * WITH JSON STRUCTURE Module.JsonStructure * { * Module.Entity { @@ -504,6 +507,7 @@ importMappingObjectHandling */ createExportMappingStatement : EXPORT MAPPING qualifiedName + (FOLDER STRING_LITERAL)? exportMappingWithClause? exportMappingNullValuesClause? LBRACE exportMappingRootElement RBRACE @@ -619,7 +623,8 @@ createIndexStatement */ createDataTransformerStatement : DATA TRANSFORMER qualifiedName - SOURCE_KW (JSON | XML) STRING_LITERAL + (FOLDER folder=STRING_LITERAL)? + SOURCE_KW (JSON | XML) source=STRING_LITERAL LBRACE dataTransformerStep* RBRACE ; diff --git a/mdl/grammar/domains/MDLMicroflow.g4 b/mdl/grammar/domains/MDLMicroflow.g4 index 0c529575a..fc83efb68 100644 --- a/mdl/grammar/domains/MDLMicroflow.g4 +++ b/mdl/grammar/domains/MDLMicroflow.g4 @@ -37,6 +37,7 @@ createNanoflowStatement */ createJavaActionStatement : JAVA ACTION qualifiedName + (FOLDER STRING_LITERAL)? LPAREN javaActionParameterList? RPAREN javaActionReturnType? javaActionExposedClause? @@ -66,6 +67,7 @@ javaActionExposedClause */ createJavaScriptActionStatement : JAVASCRIPT ACTION qualifiedName + (FOLDER STRING_LITERAL)? LPAREN javaActionParameterList? RPAREN javaActionReturnType? javaActionExposedClause? @@ -200,13 +202,28 @@ enumSplitCaseValue | LPAREN EMPTY RPAREN ; +// Branch keyword is `when ... then`, matching caseStatement above and the +// caseExpression in MDLSettings.g4 — `case` used to introduce a BRANCH here +// while introducing the SUBJECT in those two, so the word meant two things +// (mxcli #913). The `case`/`else` spelling still parses: scripts in the wild +// use it, and both spellings build the identical flow. MDL065 warns. inheritanceSplitStatement : SPLIT TYPE VARIABLE - (inheritanceSplitCase+ (ELSE microflowBody)? END SPLIT)? + (inheritanceSplitCase+ inheritanceSplitElse? END SPLIT)? ; inheritanceSplitCase - : CASE qualifiedName microflowBody + : CASE qualifiedName microflowBody // legacy spelling, warns MDL065 + | WHEN qualifiedName THEN microflowBody + ; + +// The last branch is Mendix's `(empty)` outgoing flow — taken when the object +// is NULL, not when no case matched. `else` is the legacy spelling and reads +// as a default, which it is not: mxbuild still demands a flow for every +// subtype and for the base entity (CE0090) when an `else` is present. +inheritanceSplitElse + : ELSE microflowBody // legacy spelling, warns MDL065 + | WHEN LPAREN EMPTY RPAREN THEN microflowBody ; castObjectStatement diff --git a/mdl/grammar/domains/MDLPage.g4 b/mdl/grammar/domains/MDLPage.g4 index 77e9633ee..242799652 100644 --- a/mdl/grammar/domains/MDLPage.g4 +++ b/mdl/grammar/domains/MDLPage.g4 @@ -305,7 +305,17 @@ blockOverride // after a reserved keyword (e.g. "List", "Column") can be expressed. DESCRIBE // emits the quoted form for such names so its output re-parses. See issue #619. widgetV3 - : widgetTypeV3 (IDENTIFIER | QUOTED_IDENTIFIER | keyword) widgetPropertiesV3? widgetBodyV3? + // A List View specialization template: one body per specialization of the + // list view's entity. It carries an entity, not a name — Studio Pro stores + // Forms$ListViewTemplate with exactly {Entity, Widgets}. + // + // FIRST alternative on purpose. FOR is in the `keyword` rule, so without the + // ordering `template for Pages.Bus { }` could be read as a TEMPLATE widget + // named "for". A Gallery content slot named `for` must now be quoted + // (`template "for" { }`), the same escape hatch reserved names already use + // (issue #619). + : TEMPLATE FOR qualifiedName widgetBodyV3 + | widgetTypeV3 (IDENTIFIER | QUOTED_IDENTIFIER | keyword) widgetPropertiesV3? widgetBodyV3? | PLUGGABLEWIDGET STRING_LITERAL (IDENTIFIER | QUOTED_IDENTIFIER | keyword) widgetPropertiesV3? widgetBodyV3? // PLUGGABLEWIDGET 'widget.id' name | CUSTOMWIDGET STRING_LITERAL (IDENTIFIER | QUOTED_IDENTIFIER | keyword) widgetPropertiesV3? widgetBodyV3? // CUSTOMWIDGET 'widget.id' name (legacy) ; diff --git a/mdl/grammar/domains/MDLService.g4 b/mdl/grammar/domains/MDLService.g4 index ed388b5cc..ddf32c365 100644 --- a/mdl/grammar/domains/MDLService.g4 +++ b/mdl/grammar/domains/MDLService.g4 @@ -12,6 +12,7 @@ options { tokenVocab = MDLLexer; } createDatabaseConnectionStatement : DATABASE CONNECTION qualifiedName + (FOLDER STRING_LITERAL)? databaseConnectionOption+ (BEGIN databaseQuery* END)? ; diff --git a/mdl/grammar/domains/MDLWorkflow.g4 b/mdl/grammar/domains/MDLWorkflow.g4 index f7bf26b49..30486260e 100644 --- a/mdl/grammar/domains/MDLWorkflow.g4 +++ b/mdl/grammar/domains/MDLWorkflow.g4 @@ -14,12 +14,13 @@ options { tokenVocab = MDLLexer; } */ createWorkflowStatement : WORKFLOW qualifiedName + (FOLDER folder=STRING_LITERAL)? (PARAMETER VARIABLE COLON qualifiedName)? - (DISPLAY STRING_LITERAL)? - (DESCRIPTION STRING_LITERAL)? + (DISPLAY display=STRING_LITERAL)? + (DESCRIPTION description=STRING_LITERAL)? (EXPORT LEVEL (IDENTIFIER | API))? (OVERVIEW PAGE qualifiedName)? - (DUE DATE_TYPE STRING_LITERAL)? + (DUE DATE_TYPE dueDate=STRING_LITERAL)? BEGIN workflowBody END WORKFLOW SEMICOLON? SLASH? ; diff --git a/mdl/types/documentkind.go b/mdl/types/documentkind.go new file mode 100644 index 000000000..97db019fd --- /dev/null +++ b/mdl/types/documentkind.go @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: Apache-2.0 + +package types + +import ( + "strings" + "unicode" +) + +// documentKindOverrides names the document types where mxcli's own vocabulary +// differs from Mendix's stored $Type. Everything else is derived from the +// $Type itself by DocumentKind, so a document type mxcli has never heard of +// still gets a truthful name rather than falling back to "document". +// +// Kept deliberately short: an entry here is a claim that mxcli calls the thing +// something other than what the model calls it, and each one is a place the two +// vocabularies can drift. A type absent from this map is not a gap. +var documentKindOverrides = map[string]string{ + "Menus$MenuDocument": "menu", + "Rest$ConsumedRestService": "rest client", + "Rest$PublishedRestService": "published rest service", + "CustomBlobDocuments$CustomBlobDocument": "agent document", + "CustomIcons$CustomIconCollection": "icon collection", + "DatabaseConnector$DatabaseConnection": "database connection", + // The camel-case split cannot know that "OData" and "JavaScript" are each + // one word, so it produces "consumed o data service" and "java script + // action". These are the derivation's blind spot, not a vocabulary + // difference. + "Rest$ConsumedODataService": "odata client", + "ODataPublish$PublishedODataService": "odata service", + "JavaScriptActions$JavaScriptAction": "javascript action", +} + +// DocumentKind renders a unit's stored $Type as the noun mxcli uses for it — +// "JsonStructures$JsonStructure" becomes "json structure". +// +// The derivation (take the part after "$", split the camel case, lowercase it) +// is right for the large majority of document types and, crucially, degrades +// honestly: an unrecognised type yields the model's own name for it instead of +// a generic placeholder. Callers use this to report what they acted on when +// they located a document without knowing its type in advance. +func DocumentKind(unitType string) string { + if unitType == "" { + return "document" + } + if kind, ok := documentKindOverrides[unitType]; ok { + return kind + } + name := unitType + if i := strings.LastIndex(unitType, "$"); i >= 0 { + name = unitType[i+1:] + } + if name == "" { + return unitType + } + return strings.ToLower(splitCamelWords(name)) +} + +// splitCamelWords inserts a space before each interior capital that starts a +// new word, so "JsonStructure" reads as "Json Structure". A run of capitals is +// treated as one word ("XPathQuery" → "XPath Query") so acronyms survive. +func splitCamelWords(s string) string { + var b strings.Builder + runes := []rune(s) + for i, r := range runes { + if i > 0 && unicode.IsUpper(r) { + prev := runes[i-1] + nextIsLower := i+1 < len(runes) && unicode.IsLower(runes[i+1]) + if unicode.IsLower(prev) || unicode.IsDigit(prev) || (unicode.IsUpper(prev) && nextIsLower) { + b.WriteRune(' ') + } + } + b.WriteRune(r) + } + return b.String() +} diff --git a/mdl/types/documentkind_test.go b/mdl/types/documentkind_test.go new file mode 100644 index 000000000..eca25ef33 --- /dev/null +++ b/mdl/types/documentkind_test.go @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: Apache-2.0 + +package types + +import "testing" + +// TestDocumentKindDerivesTheCommonCase pins that the derivation, not the +// override table, carries most document types — the table stays short only if +// this keeps being true. +func TestDocumentKindDerivesTheCommonCase(t *testing.T) { + cases := map[string]string{ + "JsonStructures$JsonStructure": "json structure", + "ImportMappings$ImportMapping": "import mapping", + "ExportMappings$ExportMapping": "export mapping", + "Microflows$Microflow": "microflow", + "Microflows$Nanoflow": "nanoflow", + "Forms$Page": "page", + "Forms$Snippet": "snippet", + "Forms$Layout": "layout", + "Forms$BuildingBlock": "building block", + "Enumerations$Enumeration": "enumeration", + "Constants$Constant": "constant", + "Queues$Queue": "queue", + "ScheduledEvents$ScheduledEvent": "scheduled event", + "RegularExpressions$RegularExpression": "regular expression", + "Workflows$Workflow": "workflow", + "JavaActions$JavaAction": "java action", + "Images$ImageCollection": "image collection", + "DataTransformers$DataTransformer": "data transformer", + "BusinessEvents$BusinessEventService": "business event service", + } + for unitType, want := range cases { + if got := DocumentKind(unitType); got != want { + t.Errorf("DocumentKind(%q) = %q, want %q", unitType, got, want) + } + } +} + +// TestDocumentKindOverridesTheDerivationsBlindSpots covers the two reasons an +// override exists: mxcli calling something by a different name than the model +// does, and the camel-case split not knowing that "OData" and "JavaScript" are +// each a single word. +func TestDocumentKindOverridesTheDerivationsBlindSpots(t *testing.T) { + cases := map[string]string{ + "Menus$MenuDocument": "menu", + "Rest$ConsumedRestService": "rest client", + "Rest$PublishedRestService": "published rest service", + "CustomIcons$CustomIconCollection": "icon collection", + "Rest$ConsumedODataService": "odata client", + "ODataPublish$PublishedODataService": "odata service", + "JavaScriptActions$JavaScriptAction": "javascript action", + } + for unitType, want := range cases { + if got := DocumentKind(unitType); got != want { + t.Errorf("DocumentKind(%q) = %q, want %q", unitType, got, want) + } + } +} + +// TestDocumentKindDegradesToTheModelsOwnName is the property that lets callers +// report a document type nobody has taught mxcli about. An unknown type must +// yield the model's word for it, never a generic placeholder that would read as +// though mxcli recognised it. +func TestDocumentKindDegradesToTheModelsOwnName(t *testing.T) { + if got := DocumentKind("SomeFutureDomain$WidgetGallery"); got != "widget gallery" { + t.Errorf("DocumentKind of an unknown type = %q, want it derived from the type", got) + } + if got := DocumentKind(""); got != "document" { + t.Errorf("DocumentKind(\"\") = %q, want the generic fallback", got) + } +} diff --git a/mdl/types/infrastructure.go b/mdl/types/infrastructure.go index ad6c192dd..3734e8b33 100644 --- a/mdl/types/infrastructure.go +++ b/mdl/types/infrastructure.go @@ -70,6 +70,22 @@ type UnitInfo struct { Type string } +// DocumentUnit is a top-level document located by name rather than by type: +// its unit identity, where it sits, and what it turned out to be. +// +// Type is the unit's stored $Type (e.g. "JsonStructures$JsonStructure"), which +// is what makes a type-agnostic lookup reportable — the caller can name the +// kind of thing it moved without having known it in advance. Kind is that +// $Type rendered for humans ("json structure"), or the raw $Type when mxcli has +// no friendlier name for it. +type DocumentUnit struct { + ID model.ID + ContainerID model.ID + Name string + Type string + Kind string +} + // RenameHit records a single rename reference replacement. type RenameHit struct { UnitID string diff --git a/mdl/types/widget_visibility.go b/mdl/types/widget_visibility.go index c5f0e8db3..7c3b05452 100644 --- a/mdl/types/widget_visibility.go +++ b/mdl/types/widget_visibility.go @@ -12,10 +12,22 @@ package types // TextTemplate; emitting the template's populated default instead triggers // CE0463 ("the definition of this widget has changed"). See issue #574. type WidgetVisibilityRule struct { - PropertyKey string `json:"propertyKey"` - HiddenWhen *WidgetVisibilityCondition `json:"hiddenWhen,omitempty"` + PropertyKey string `json:"propertyKey"` + // ListPropertyKey names the object-list property whose ITEMS carry + // PropertyKey (e.g. an Accordion group's `initialCollapsedState` lives under + // list `groups`). Empty for a rule about a top-level widget property. + // + // Nested rules are evaluated per list item, so consumers that only understand + // top-level properties MUST skip a rule with a non-empty ListPropertyKey — + // its PropertyKey does not name a property of the widget itself. + ListPropertyKey string `json:"listPropertyKey,omitempty"` + HiddenWhen *WidgetVisibilityCondition `json:"hiddenWhen,omitempty"` } +// Nested reports whether the rule targets an item sub-property of an object list +// rather than a top-level widget property. +func (r WidgetVisibilityRule) Nested() bool { return r.ListPropertyKey != "" } + // WidgetVisibilityCondition is a single predicate evaluated against the // widget's current property values. Operators cover the dominant patterns // observed in marketplace editorConfig.js files: @@ -32,8 +44,19 @@ type WidgetVisibilityCondition struct { PropertyKey string `json:"propertyKey"` Operator string `json:"operator"` Value string `json:"value,omitempty"` + // Scope says which object PropertyKey belongs to. "" (the default) means the + // widget itself; ConditionScopeItem means the sibling sub-property of the + // same object-list item, which only makes sense on a nested rule. An + // Accordion has both: its group properties are hidden when the WIDGET's + // `collapsible` is off, and `initiallyCollapsed` is hidden unless the + // GROUP's own `initialCollapsedState` is "dynamic". + Scope string `json:"scope,omitempty"` } +// ConditionScopeItem marks a condition evaluated against the object-list item +// that carries the rule's property, rather than against the widget. +const ConditionScopeItem = "item" + // Hidden reports whether the condition matches given the widget's current // property values (keyed by property key, each value the property's primitive // string form). An unset or unrecognized operator is treated as "not hidden". diff --git a/mdl/visitor/visitor_agenteditor.go b/mdl/visitor/visitor_agenteditor.go index d528f1e48..e316ec106 100644 --- a/mdl/visitor/visitor_agenteditor.go +++ b/mdl/visitor/visitor_agenteditor.go @@ -19,6 +19,9 @@ func (b *Builder) ExitCreateModelStatement(ctx *parser.CreateModelStatementConte Name: buildQualifiedName(ctx.QualifiedName()), } stmt.Documentation = findDocCommentText(ctx) + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } for _, p := range ctx.AllModelProperty() { propCtx := p.(*parser.ModelPropertyContext) @@ -156,6 +159,9 @@ func (b *Builder) ExitCreateConsumedMCPServiceStatement(ctx *parser.CreateConsum Name: buildQualifiedName(ctx.QualifiedName()), } stmt.OuterDocumentation = findDocCommentText(ctx) + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } props := parseModelProps(ctx.AllModelProperty()) stmt.ProtocolVersion = props["protocolversion"] @@ -179,6 +185,9 @@ func (b *Builder) ExitCreateKnowledgeBaseStatement(ctx *parser.CreateKnowledgeBa Name: buildQualifiedName(ctx.QualifiedName()), } stmt.Documentation = findDocCommentText(ctx) + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } props := parseModelProps(ctx.AllModelProperty()) stmt.Provider = props["provider"] @@ -207,6 +216,9 @@ func (b *Builder) ExitCreateAgentStatement(ctx *parser.CreateAgentStatementConte Name: buildQualifiedName(ctx.QualifiedName()), } stmt.Documentation = findDocCommentText(ctx) + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } props := parseModelProps(ctx.AllModelProperty()) stmt.UsageType = props["usagetype"] diff --git a/mdl/visitor/visitor_alter_page.go b/mdl/visitor/visitor_alter_page.go index a2b79abd8..717ada197 100644 --- a/mdl/visitor/visitor_alter_page.go +++ b/mdl/visitor/visitor_alter_page.go @@ -35,6 +35,8 @@ func (b *Builder) exitAlterPageStatement(ctx *parser.AlterStatementContext) { stmt.Operations = append(stmt.Operations, b.buildAlterPageInsert(insertCtx.(*parser.AlterPageInsertContext))) } else if dropCtx := op.AlterPageDrop(); dropCtx != nil { stmt.Operations = append(stmt.Operations, b.buildAlterPageDrop(dropCtx.(*parser.AlterPageDropContext))) + } else if dropTplCtx := op.AlterPageDropTemplate(); dropTplCtx != nil { + stmt.Operations = append(stmt.Operations, b.buildAlterPageDropTemplate(dropTplCtx.(*parser.AlterPageDropTemplateContext))) } else if replaceCtx := op.AlterPageReplace(); replaceCtx != nil { stmt.Operations = append(stmt.Operations, b.buildAlterPageReplace(replaceCtx.(*parser.AlterPageReplaceContext))) } else if addVarCtx := op.AlterPageAddVariable(); addVarCtx != nil { @@ -47,6 +49,19 @@ func (b *Builder) exitAlterPageStatement(ctx *parser.AlterStatementContext) { b.statements = append(b.statements, stmt) } +// buildAlterPageDropTemplate builds a DropListViewTemplateOp: +// DROP TEMPLATE FOR Module.Specialization IN listViewName +func (b *Builder) buildAlterPageDropTemplate(ctx *parser.AlterPageDropTemplateContext) ast.AlterPageOperation { + op := &ast.DropListViewTemplateOp{} + if qn := ctx.QualifiedName(); qn != nil { + op.Specialization = qn.GetText() + } + if wr := ctx.WidgetRef(); wr != nil { + op.ListView = buildWidgetRef(wr).Widget + } + return op +} + // buildAlterPageSet builds a SetPropertyOp or SetLayoutOp from the parse tree. func (b *Builder) buildAlterPageSet(ctx *parser.AlterPageSetContext) ast.AlterPageOperation { // SET Layout = Module.LayoutName [MAP (...)] diff --git a/mdl/visitor/visitor_datatransformer.go b/mdl/visitor/visitor_datatransformer.go index 85cbb9524..d5d952368 100644 --- a/mdl/visitor/visitor_datatransformer.go +++ b/mdl/visitor/visitor_datatransformer.go @@ -25,9 +25,14 @@ func (b *Builder) ExitCreateDataTransformerStatement(ctx *parser.CreateDataTrans stmt.SourceType = "XML" } - // Source content — the STRING_LITERAL after JSON/XML - if sl := ctx.STRING_LITERAL(); sl != nil { - stmt.SourceJSON = unquoteString(sl.GetText()) + // Source content and folder are read by grammar label: the rule now has two + // direct STRING_LITERALs, so an index would depend on whether a folder was + // given. + if tok := ctx.GetSource(); tok != nil { + stmt.SourceJSON = unquoteString(tok.GetText()) + } + if tok := ctx.GetFolder(); tok != nil { + stmt.Folder = unquoteString(tok.GetText()) } // Steps diff --git a/mdl/visitor/visitor_dbconnection.go b/mdl/visitor/visitor_dbconnection.go index e6e8d7367..6864c5efb 100644 --- a/mdl/visitor/visitor_dbconnection.go +++ b/mdl/visitor/visitor_dbconnection.go @@ -16,6 +16,9 @@ func (b *Builder) ExitCreateDatabaseConnectionStatement(ctx *parser.CreateDataba stmt := &ast.CreateDatabaseConnectionStmt{ Name: buildQualifiedName(ctx.QualifiedName()), } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } // Parse options for _, optCtx := range ctx.AllDatabaseConnectionOption() { diff --git a/mdl/visitor/visitor_entity.go b/mdl/visitor/visitor_entity.go index 1949b06cd..645a0fd38 100644 --- a/mdl/visitor/visitor_entity.go +++ b/mdl/visitor/visitor_entity.go @@ -3,6 +3,7 @@ package visitor import ( + "fmt" "strconv" "strings" @@ -962,13 +963,15 @@ func (b *Builder) ExitMoveStatement(ctx *parser.MoveStatementContext) { return } - // Handle MOVE FOLDER separately — different AST type - // MOVE FOLDER is identified by having FOLDER as the first token after MOVE (no document type keyword) - if len(ctx.AllFOLDER()) > 0 && ctx.PAGE() == nil && ctx.MICROFLOW() == nil && - ctx.SNIPPET() == nil && ctx.NANOFLOW() == nil && ctx.ENTITY() == nil && - ctx.ENUMERATION() == nil && ctx.CONSTANT() == nil && ctx.DATABASE() == nil && - ctx.JAVA() == nil && ctx.ODATA() == nil { - b.exitMoveFolderStatement(ctx, names) + // MOVE FOLDER is a different AST type, and is told apart by the ABSENCE of a + // doctype. That is one check because moveDocumentType is a grammar rule: the + // discriminator used to be a hand-written negation of every doctype keyword, + // so every doctype added to MOVE had to be added here too or a folder move + // silently started parsing as a document move (mxcli-formula1 #32). + if ctx.MoveDocumentType() == nil && ctx.ENTITY() == nil { + if len(ctx.AllFOLDER()) > 0 { + b.exitMoveFolderStatement(ctx, names) + } return } @@ -976,27 +979,18 @@ func (b *Builder) ExitMoveStatement(ctx *parser.MoveStatementContext) { Name: buildQualifiedName(names[0]), } - // Determine document type - if ctx.PAGE() != nil { - stmt.DocumentType = ast.DocumentTypePage - } else if ctx.MICROFLOW() != nil { - stmt.DocumentType = ast.DocumentTypeMicroflow - } else if ctx.SNIPPET() != nil { - stmt.DocumentType = ast.DocumentTypeSnippet - } else if ctx.NANOFLOW() != nil { - stmt.DocumentType = ast.DocumentTypeNanoflow - } else if ctx.ENTITY() != nil { + if ctx.ENTITY() != nil { stmt.DocumentType = ast.DocumentTypeEntity - } else if ctx.ENUMERATION() != nil { - stmt.DocumentType = ast.DocumentTypeEnumeration - } else if ctx.CONSTANT() != nil { - stmt.DocumentType = ast.DocumentTypeConstant - } else if ctx.DATABASE() != nil { - stmt.DocumentType = ast.DocumentTypeDatabaseConnection - } else if ctx.JAVA() != nil { - stmt.DocumentType = ast.DocumentTypeJavaAction - } else if ctx.ODATA() != nil { - stmt.DocumentType = ast.DocumentTypeODataService + } else { + docType, ok := moveDocumentTypeFor(ctx.MoveDocumentType().GetText()) + if !ok { + // A doctype in the grammar with no AST constant would otherwise move + // whatever the name resolved to under an empty type. Refuse loudly: + // the grammar and this table are meant to be added to together. + b.addError(fmt.Errorf("MOVE does not support document type %q", ctx.MoveDocumentType().GetText())) + return + } + stmt.DocumentType = docType } // Parse folder path if specified @@ -1049,3 +1043,14 @@ func (b *Builder) exitMoveFolderStatement(ctx *parser.MoveStatementContext, name // ---------------------------------------------------------------------------- // ExitCreateAssociationStatement is called when exiting the createAssociationStatement production. + +// moveDocumentTypeFor maps a moveDocumentType rule's text to its AST constant. +// +// The table lives in the ast package because the executor reads it too: it has +// to decide whether the document it found is the kind the statement named, and +// a second copy of the doctype list is exactly how the MOVE FOLDER +// discriminator drifted out of step with the grammar before. +func moveDocumentTypeFor(ruleText string) (ast.DocumentType, bool) { + docType, ok := ast.MoveDocumentTypeByKeyword[strings.ToUpper(ruleText)] + return docType, ok +} diff --git a/mdl/visitor/visitor_folder_clause_test.go b/mdl/visitor/visitor_folder_clause_test.go new file mode 100644 index 000000000..063d706fc --- /dev/null +++ b/mdl/visitor/visitor_folder_clause_test.go @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// folderOf returns the Folder field of whatever create statement src produced, +// so one table can cover doctypes with unrelated AST types. +func folderOf(t *testing.T, src string) (string, bool) { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors for %q: %v", src, errs) + } + for _, s := range prog.Statements { + switch v := s.(type) { + case *ast.CreateImportMappingStmt: + return v.Folder, true + case *ast.CreateExportMappingStmt: + return v.Folder, true + case *ast.CreateQueueStmt: + return v.Folder, true + case *ast.CreateRegularExpressionStmt: + return v.Folder, true + case *ast.CreateScheduledEventStmt: + return v.Folder, true + case *ast.CreateImageCollectionStmt: + return v.Folder, true + case *ast.CreateMenuStmt: + return v.Folder, true + case *ast.CreateJavaActionStmt: + return v.Folder, true + case *ast.CreateJavaScriptActionStmt: + return v.Folder, true + case *ast.CreateDatabaseConnectionStmt: + return v.Folder, true + case *ast.CreateDataTransformerStmt: + return v.Folder, true + case *ast.CreateWorkflowStmt: + return v.Folder, true + case *ast.CreateModelStmt: + return v.Folder, true + case *ast.CreateKnowledgeBaseStmt: + return v.Folder, true + case *ast.CreateConsumedMCPServiceStmt: + return v.Folder, true + case *ast.CreateAgentStmt: + return v.Folder, true + } + } + return "", false +} + +// TestCreateStatementsAcceptAFolderClause covers the doctypes that had no way +// to be placed at all: #932 reported mappings, but a dozen more documents could +// only ever be created at the module root. +func TestCreateStatementsAcceptAFolderClause(t *testing.T) { + const want = "Private/Filed" + cases := map[string]string{ + "import mapping": `create import mapping M.IMM folder 'Private/Filed' + with json structure M.JS { create M.E { A = a } };`, + "export mapping": `create export mapping M.EXM folder 'Private/Filed' + with json structure M.JS { M.E { a = A } };`, + "queue": `create queue M.Q folder 'Private/Filed' ( Parallelism: 3 );`, + "regular expression": `create regular expression M.RE folder 'Private/Filed' ( Expression: '\d+' );`, + "scheduled event": `create scheduled event M.SE folder 'Private/Filed' ( Microflow: M.ACT_Run, Repeat: Day );`, + "image collection": `create image collection M.IC folder 'Private/Filed';`, + "menu": `create menu M.Menu folder 'Private/Filed' ( menu item 'Home' page M.Home; );`, + "java action": "create java action M.JA folder 'Private/Filed' () returns string as $$return null;$$;", + "javascript action": "create javascript action M.JSA folder 'Private/Filed' () returns string as $$return null;$$;", + "database connection": `create database connection M.DB folder 'Private/Filed' + type 'PostgreSQL' connection string 'jdbc:postgresql://h/db';`, + "data transformer": `create data transformer M.DT folder 'Private/Filed' + source json '{"a": 1}' { JSLT '{ "b": .a }'; };`, + "workflow": `create workflow M.WF folder 'Private/Filed' + begin + user task Approve 'Approve the order'; + end workflow;`, + "model": `create model M.Model folder 'Private/Filed' ( Provider: MxCloudGenAI );`, + "knowledge base": `create knowledge base M.KB folder 'Private/Filed' ( Provider: MxCloudGenAI );`, + "consumed mcp service": `create consumed mcp service M.MCP folder 'Private/Filed' ( Version: '0.0.1' );`, + "agent": `create agent M.Agent folder 'Private/Filed' ( UsageType: Task );`, + } + for kind, src := range cases { + t.Run(kind, func(t *testing.T) { + got, found := folderOf(t, src) + if !found { + t.Fatalf("no create statement produced for %s", kind) + } + if got != want { + t.Errorf("Folder = %q, want %q", got, want) + } + }) + } +} + +// TestCreateStatementsWithoutAFolderClauseAreSilent is the control. An absent +// clause must leave Folder empty, because the executor reads emptiness as +// "leave placement alone" — a visitor that defaulted it to anything would unfile +// every foldered document on the next CREATE OR MODIFY. +func TestCreateStatementsWithoutAFolderClauseAreSilent(t *testing.T) { + cases := map[string]string{ + "import mapping": `create import mapping M.IMM with json structure M.JS { create M.E { A = a } };`, + "queue": `create queue M.Q ( Parallelism: 3 );`, + "java action": "create java action M.JA () returns string as $$return null;$$;", + "model": `create model M.Model ( Provider: MxCloudGenAI );`, + } + for kind, src := range cases { + t.Run(kind, func(t *testing.T) { + got, found := folderOf(t, src) + if !found { + t.Fatalf("no create statement produced for %s", kind) + } + if got != "" { + t.Errorf("Folder = %q, want empty for a statement with no folder clause", got) + } + }) + } +} + +// TestWorkflowHeaderClausesAreReadByLabel pins the trap that adding a FOLDER +// clause to the workflow rule created. The header's optional strings used to be +// read by counting STRING_LITERALs in order, so a folder path — now the first +// string in the rule — would have been picked up as the display name. +func TestWorkflowHeaderClausesAreReadByLabel(t *testing.T) { + src := `create workflow M.WF + folder 'Private/Filed' + display 'Approve Order' + description 'Two-step approval' + due date '[%DayLength%]' + begin + user task Approve 'Approve the order'; + end workflow;` + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + var wf *ast.CreateWorkflowStmt + for _, s := range prog.Statements { + if w, ok := s.(*ast.CreateWorkflowStmt); ok { + wf = w + } + } + if wf == nil { + t.Fatal("no CreateWorkflowStmt produced") + } + if wf.Folder != "Private/Filed" { + t.Errorf("Folder = %q", wf.Folder) + } + if wf.DisplayName != "Approve Order" { + t.Errorf("DisplayName = %q — the folder path leaked into it", wf.DisplayName) + } + if wf.Description != "Two-step approval" { + t.Errorf("Description = %q", wf.Description) + } + if wf.DueDate != "[%DayLength%]" { + t.Errorf("DueDate = %q", wf.DueDate) + } +} + +// TestDataTransformerSourceIsReadByLabel is the same trap in the other rule that +// gained a second direct STRING_LITERAL. +func TestDataTransformerSourceIsReadByLabel(t *testing.T) { + src := `create data transformer M.DT folder 'Private/Filed' + source json '{"a": 1}' { JSLT '{ "b": .a }'; };` + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + for _, s := range prog.Statements { + if dt, ok := s.(*ast.CreateDataTransformerStmt); ok { + if dt.Folder != "Private/Filed" { + t.Errorf("Folder = %q", dt.Folder) + } + if dt.SourceJSON != `{"a": 1}` { + t.Errorf("SourceJSON = %q — the folder path was read as the source", dt.SourceJSON) + } + return + } + } + t.Fatal("no CreateDataTransformerStmt produced") +} diff --git a/mdl/visitor/visitor_imagecollection.go b/mdl/visitor/visitor_imagecollection.go index 45ff4ac58..4d8f9c51a 100644 --- a/mdl/visitor/visitor_imagecollection.go +++ b/mdl/visitor/visitor_imagecollection.go @@ -13,6 +13,9 @@ func (b *Builder) ExitCreateImageCollectionStatement(ctx *parser.CreateImageColl Name: buildQualifiedName(ctx.QualifiedName()), ExportLevel: "Hidden", } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } // Extract /** ... */ doc comment (same as other create statements) stmt.Comment = findDocCommentText(ctx) diff --git a/mdl/visitor/visitor_import_export_mapping.go b/mdl/visitor/visitor_import_export_mapping.go index 07b327b7f..79a848a9d 100644 --- a/mdl/visitor/visitor_import_export_mapping.go +++ b/mdl/visitor/visitor_import_export_mapping.go @@ -15,6 +15,9 @@ func (b *Builder) ExitCreateImportMappingStatement(ctx *parser.CreateImportMappi stmt := &ast.CreateImportMappingStmt{ Name: buildQualifiedName(ctx.QualifiedName()), } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } // Parse WITH clause if wc := ctx.ImportMappingWithClause(); wc != nil { @@ -126,6 +129,9 @@ func (b *Builder) ExitCreateExportMappingStatement(ctx *parser.CreateExportMappi stmt := &ast.CreateExportMappingStmt{ Name: buildQualifiedName(ctx.QualifiedName()), } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } // Parse WITH clause if wc := ctx.ExportMappingWithClause(); wc != nil { diff --git a/mdl/visitor/visitor_inheritance_split_spelling_test.go b/mdl/visitor/visitor_inheritance_split_spelling_test.go new file mode 100644 index 000000000..7785f1f76 --- /dev/null +++ b/mdl/visitor/visitor_inheritance_split_spelling_test.go @@ -0,0 +1,146 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// The two spellings of a type split must produce the same AST apart from the +// flags that exist only to drive the MDL065 deprecation warning. If they ever +// diverge, the "both build the identical flow" promise in the warning's own +// text stops being true. (#913) +func TestInheritanceSplit_BothSpellingsProduceTheSameAST(t *testing.T) { + const legacy = `CREATE MICROFLOW Sample.Route ($Input: Sample.Animal) +RETURNS String +BEGIN + split type $Input + case Sample.Dog + return 'woof'; + case Sample.Cat + return 'meow'; + else + return 'none'; + end split; +END;` + + const modern = `CREATE MICROFLOW Sample.Route ($Input: Sample.Animal) +RETURNS String +BEGIN + split type $Input + when Sample.Dog then + return 'woof'; + when Sample.Cat then + return 'meow'; + when (empty) then + return 'none'; + end split; +END;` + + oldSplit := parseInheritanceSplit(t, legacy) + newSplit := parseInheritanceSplit(t, modern) + + if oldSplit.Variable != newSplit.Variable { + t.Errorf("variable: legacy %q, modern %q", oldSplit.Variable, newSplit.Variable) + } + if len(oldSplit.Cases) != len(newSplit.Cases) { + t.Fatalf("case count: legacy %d, modern %d", len(oldSplit.Cases), len(newSplit.Cases)) + } + for i := range oldSplit.Cases { + if got, want := newSplit.Cases[i].Entity.String(), oldSplit.Cases[i].Entity.String(); got != want { + t.Errorf("case %d entity: legacy %q, modern %q", i, want, got) + } + if got, want := len(newSplit.Cases[i].Body), len(oldSplit.Cases[i].Body); got != want { + t.Errorf("case %d body length: legacy %d, modern %d", i, want, got) + } + } + if got, want := len(newSplit.ElseBody), len(oldSplit.ElseBody); got != want { + t.Errorf("empty-branch body length: legacy %d, modern %d", want, got) + } + if len(newSplit.ElseBody) == 0 { + t.Error("`when (empty) then` did not populate ElseBody — the empty branch was dropped") + } +} + +// The spelling flags are what MDL065 keys off. A parser that stopped setting +// them would silently retire the warning, and nothing else would fail. +func TestInheritanceSplit_SpellingFlagsRecordTheSource(t *testing.T) { + tests := []struct { + name string + branch, empty string + wantCaseLegacy bool + wantElseLegacy bool + }{ + {"both legacy", "case Sample.Dog", "else", true, true}, + {"both modern", "when Sample.Dog then", "when (empty) then", false, false}, + {"legacy branch, modern empty", "case Sample.Dog", "when (empty) then", true, false}, + {"modern branch, legacy empty", "when Sample.Dog then", "else", false, true}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + src := `CREATE MICROFLOW Sample.Route ($Input: Sample.Animal) +RETURNS String +BEGIN + split type $Input + ` + tc.branch + ` + return 'woof'; + ` + tc.empty + ` + return 'none'; + end split; +END;` + split := parseInheritanceSplit(t, src) + if split.LegacyCaseKeyword != tc.wantCaseLegacy { + t.Errorf("LegacyCaseKeyword = %v, want %v", split.LegacyCaseKeyword, tc.wantCaseLegacy) + } + if split.LegacyElseKeyword != tc.wantElseLegacy { + t.Errorf("LegacyElseKeyword = %v, want %v", split.LegacyElseKeyword, tc.wantElseLegacy) + } + }) + } +} + +// Mixing the spellings within one statement parses. Nothing depends on it, but +// a grammar change that made the two alternatives mutually exclusive would +// reject scripts mid-migration, so the tolerance is pinned deliberately. +func TestInheritanceSplit_MixedSpellingsParse(t *testing.T) { + split := parseInheritanceSplit(t, `CREATE MICROFLOW Sample.Route ($Input: Sample.Animal) +RETURNS String +BEGIN + split type $Input + case Sample.Dog + return 'woof'; + when Sample.Cat then + return 'meow'; + when (empty) then + return 'none'; + end split; +END;`) + if len(split.Cases) != 2 { + t.Fatalf("case count = %d, want 2", len(split.Cases)) + } + if !split.LegacyCaseKeyword { + t.Error("LegacyCaseKeyword = false, want true — one branch used `case`") + } + if split.LegacyElseKeyword { + t.Error("LegacyElseKeyword = true, want false — the empty branch used the modern spelling") + } +} + +func parseInheritanceSplit(t *testing.T, src string) *ast.InheritanceSplitStmt { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + mf, ok := prog.Statements[0].(*ast.CreateMicroflowStmt) + if !ok { + t.Fatalf("statement 0: got %T, want *ast.CreateMicroflowStmt", prog.Statements[0]) + } + split, ok := mf.Body[0].(*ast.InheritanceSplitStmt) + if !ok { + t.Fatalf("body statement 0: got %T, want *ast.InheritanceSplitStmt", mf.Body[0]) + } + return split +} diff --git a/mdl/visitor/visitor_javaaction.go b/mdl/visitor/visitor_javaaction.go index a3bf2a249..d1fff3a01 100644 --- a/mdl/visitor/visitor_javaaction.go +++ b/mdl/visitor/visitor_javaaction.go @@ -17,6 +17,9 @@ func (b *Builder) ExitCreateJavaActionStatement(ctx *parser.CreateJavaActionStat if qn := ctx.QualifiedName(); qn != nil { stmt.Name = buildQualifiedName(qn) } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } // Get parameters if paramList := ctx.JavaActionParameterList(); paramList != nil { @@ -113,6 +116,9 @@ func (b *Builder) ExitCreateJavaScriptActionStatement(ctx *parser.CreateJavaScri if qn := ctx.QualifiedName(); qn != nil { stmt.Name = buildQualifiedName(qn) } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } if paramList := ctx.JavaActionParameterList(); paramList != nil { for _, paramCtx := range paramList.AllJavaActionParameter() { diff --git a/mdl/visitor/visitor_menu.go b/mdl/visitor/visitor_menu.go index 295466adc..5d85999f4 100644 --- a/mdl/visitor/visitor_menu.go +++ b/mdl/visitor/visitor_menu.go @@ -19,6 +19,9 @@ func (b *Builder) ExitCreateMenuStatement(ctx *parser.CreateMenuStatementContext } stmt := &ast.CreateMenuStmt{Name: buildQualifiedName(qn)} + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } for _, itemCtx := range ctx.AllNavMenuItemDef() { stmt.Items = append(stmt.Items, buildNavMenuItemDef(itemCtx)) } diff --git a/mdl/visitor/visitor_microflow_statements.go b/mdl/visitor/visitor_microflow_statements.go index 152823165..2c73160e0 100644 --- a/mdl/visitor/visitor_microflow_statements.go +++ b/mdl/visitor/visitor_microflow_statements.go @@ -665,13 +665,22 @@ func buildInheritanceSplitStatement(ctx parser.IInheritanceSplitStatementContext } for _, caseCtx := range splitCtx.AllInheritanceSplitCase() { c := caseCtx.(*parser.InheritanceSplitCaseContext) + // `case X ` and `when X then ` are the same branch; only + // the spelling differs, and it is recorded for MDL065. + if c.CASE() != nil { + stmt.LegacyCaseKeyword = true + } stmt.Cases = append(stmt.Cases, ast.InheritanceSplitCase{ Entity: buildQualifiedName(c.QualifiedName()), Body: buildMicroflowBody(c.MicroflowBody()), }) } - if splitCtx.ELSE() != nil { - stmt.ElseBody = buildMicroflowBody(splitCtx.MicroflowBody()) + if elseCtx := splitCtx.InheritanceSplitElse(); elseCtx != nil { + e := elseCtx.(*parser.InheritanceSplitElseContext) + if e.ELSE() != nil { + stmt.LegacyElseKeyword = true + } + stmt.ElseBody = buildMicroflowBody(e.MicroflowBody()) } return stmt } diff --git a/mdl/visitor/visitor_move_alldoctypes_test.go b/mdl/visitor/visitor_move_alldoctypes_test.go new file mode 100644 index 000000000..434c60e95 --- /dev/null +++ b/mdl/visitor/visitor_move_alldoctypes_test.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// parseMove parses one MOVE statement and returns it. +func parseMove(t *testing.T, src string) *ast.MoveStmt { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors for %q: %v", src, errs) + } + for _, s := range prog.Statements { + if m, ok := s.(*ast.MoveStmt); ok { + return m + } + } + t.Fatalf("no MoveStmt produced for %q", src) + return nil +} + +// TestMoveStatement_AcceptsEveryDocumentType is #932's second half: MOVE listed +// nine doctypes, so import mappings, JSON structures and twenty others could not +// be foldered from MDL at all ("no viable alternative at input 'MOVEIMPORT'"). +// +// Driven from ast.MoveDocumentTypeByKeyword rather than a list written out here, +// so a doctype added to the grammar and the registry is covered without anyone +// remembering to extend this test — and a doctype added to the registry but not +// the grammar fails it. +func TestMoveStatement_AcceptsEveryDocumentType(t *testing.T) { + for keyword, want := range ast.MoveDocumentTypeByKeyword { + t.Run(keyword, func(t *testing.T) { + // The MDL spelling is the DocumentType value, which is the keyword + // with its word breaks restored. + src := "move " + strings.ToLower(string(want)) + " Mv.Thing to folder 'Private';" + got := parseMove(t, src) + if got.DocumentType != want { + t.Errorf("%q gave DocumentType %q, want %q", src, got.DocumentType, want) + } + if got.Name.Module != "Mv" || got.Name.Name != "Thing" { + t.Errorf("%q gave name %q", src, got.Name.String()) + } + if got.Folder != "Private" { + t.Errorf("%q gave folder %q, want Private", src, got.Folder) + } + }) + } +} + +// TestMoveStatement_ReportedStatementsParse is the issue's own text, verbatim. +func TestMoveStatement_ReportedStatementsParse(t *testing.T) { + got := parseMove(t, "MOVE IMPORT MAPPING Module.IMM_Example TO FOLDER 'Private/Import mappings';") + if got.DocumentType != ast.DocumentTypeImportMapping { + t.Errorf("DocumentType = %q, want IMPORT MAPPING", got.DocumentType) + } + if got.Folder != "Private/Import mappings" { + t.Errorf("Folder = %q", got.Folder) + } +} + +// TestMoveStatement_ToModuleWithoutFolder pins the second alternative for the +// new doctypes: moving to a module root rather than into a folder. +func TestMoveStatement_ToModuleWithoutFolder(t *testing.T) { + got := parseMove(t, "move json structure Mv.JSON_Order to OtherModule;") + if got.DocumentType != ast.DocumentTypeJsonStructure { + t.Errorf("DocumentType = %q, want JSON STRUCTURE", got.DocumentType) + } + if got.Folder != "" { + t.Errorf("Folder = %q, want empty for a move to a module root", got.Folder) + } + if got.TargetModule != "OtherModule" { + t.Errorf("TargetModule = %q, want OtherModule", got.TargetModule) + } +} + +// TestMoveStatement_FolderMoveSurvivesEveryDoctype is the regression this +// restructure exists to prevent, and the reason moveDocumentType is a grammar +// rule instead of an inline alternation. +// +// MOVE FOLDER is identified by the ABSENCE of a doctype. That used to be a +// hand-written negation of every doctype keyword, so each keyword added to MOVE +// had to be added to the discriminator too — and with twenty-two more keywords +// the odds of that staying in step were nil. The discriminator is now one check +// against the sub-rule, which cannot go stale; this test is the proof. +func TestMoveStatement_FolderMoveSurvivesEveryDoctype(t *testing.T) { + for _, src := range []string{ + "move folder Mv.Old to folder 'New';", + "move folder Mv.Old to OtherModule;", + "move folder Mv.Old to folder 'New' in OtherModule;", + } { + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors for %q: %v", src, errs) + } + sawFolderMove := false + for _, s := range prog.Statements { + if m, ok := s.(*ast.MoveStmt); ok { + t.Fatalf("%q produced a document MoveStmt (type %q)", src, m.DocumentType) + } + if _, ok := s.(*ast.MoveFolderStmt); ok { + sawFolderMove = true + } + } + if !sawFolderMove { + t.Errorf("%q produced no MoveFolderStmt", src) + } + } +} + +// TestMoveStatement_EntityIsStillItsOwnThing guards the one doctype that is not +// a unit: an entity lives inside a domain model, and its move converts +// associations rather than reparenting a row, so it must not be swept into the +// generic document path. +func TestMoveStatement_EntityIsStillItsOwnThing(t *testing.T) { + got := parseMove(t, "move entity Mv.Customer to OtherModule;") + if got.DocumentType != ast.DocumentTypeEntity { + t.Errorf("DocumentType = %q, want ENTITY", got.DocumentType) + } + if got.TargetModule != "OtherModule" { + t.Errorf("TargetModule = %q", got.TargetModule) + } +} diff --git a/mdl/visitor/visitor_page_listview_template_test.go b/mdl/visitor/visitor_page_listview_template_test.go new file mode 100644 index 000000000..888a5e9d2 --- /dev/null +++ b/mdl/visitor/visitor_page_listview_template_test.go @@ -0,0 +1,175 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// listViewOf parses a one-statement page script and returns its list view. +func listViewOf(t *testing.T, src string) *ast.WidgetV3 { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + for _, err := range errs { + t.Errorf("parse error: %v", err) + } + t.FailNow() + } + if len(prog.Statements) != 1 { + t.Fatalf("expected 1 statement, got %d", len(prog.Statements)) + } + stmt, ok := prog.Statements[0].(*ast.CreatePageStmtV3) + if !ok { + t.Fatalf("expected CreatePageStmtV3, got %T", prog.Statements[0]) + } + var find func(ws []*ast.WidgetV3) *ast.WidgetV3 + find = func(ws []*ast.WidgetV3) *ast.WidgetV3 { + for _, w := range ws { + if w.Type == "listview" { + return w + } + if got := find(w.Children); got != nil { + return got + } + } + return nil + } + lv := find(stmt.Widgets) + if lv == nil { + t.Fatal("no listview in the parsed page") + } + return lv +} + +// TestListViewSpecializationTemplateParses pins the surface syntax: a template +// is identified by the entity it renders, source order is the stored order, and +// the list view's own widgets stay separate from its templates. +func TestListViewSpecializationTemplateParses(t *testing.T) { + lv := listViewOf(t, ` +create page Pages.Vehicle_Overview (Title: 'Vehicles') { + listview vehicleListView (DataSource: database from Pages.Vehicle) { + dynamictext defaultVehicle (Content: 'v') + template for Pages.Bus { dynamictext busLabel (Content: 'b') } + template for Pages.Truck { dynamictext truckLabel (Content: 't') } + } +};`) + + if len(lv.Children) != 3 { + t.Fatalf("expected 3 body children (1 default widget + 2 templates), got %d", len(lv.Children)) + } + if got := lv.Children[0]; got.Specialization != "" || got.Name != "defaultVehicle" { + t.Errorf("first child = %+v, want the plain default widget with no specialization", got) + } + + // Order is authored, not derived: TestApp's four templates are Bus, Truck, + // Car, SUV — neither alphabetical nor domain-model order. + want := []string{"Pages.Bus", "Pages.Truck"} + for i, w := range want { + child := lv.Children[i+1] + if child.Type != "template" { + t.Errorf("child %d Type = %q, want template", i+1, child.Type) + } + if child.Specialization != w { + t.Errorf("child %d Specialization = %q, want %q", i+1, child.Specialization, w) + } + if child.Name != "" { + t.Errorf("child %d Name = %q, want empty — a list view template has no name", i+1, child.Name) + } + if len(child.Children) != 1 { + t.Errorf("child %d has %d widget(s), want 1", i+1, len(child.Children)) + } + } +} + +// TestGalleryNamedTemplateStillParses is the collision guard. +// +// `template { }` already existed as a Gallery content slot before +// `template for { }` was added, and FOR is in the parser's `keyword` +// rule — so a Gallery template could be swallowed by the new alternative, or +// vice versa. Both forms must keep their own meaning. +func TestGalleryNamedTemplateStillParses(t *testing.T) { + lv := listViewOf(t, ` +create page P.G (Title: 'G') { + listview outer (DataSource: database from P.E) { + gallery g (DataSource: database from P.E) { + template tmpl1 { dynamictext a (Content: 'x') } + } + } +};`) + gallery := lv.Children[0] + if gallery.Type != "gallery" { + t.Fatalf("expected a gallery, got %q", gallery.Type) + } + tmpl := gallery.Children[0] + if tmpl.Name != "tmpl1" { + t.Errorf("gallery template Name = %q, want tmpl1", tmpl.Name) + } + if tmpl.Specialization != "" { + t.Errorf("gallery template Specialization = %q, want empty — a named slot is "+ + "not a specialization template", tmpl.Specialization) + } +} + +// TestAlterPageDropTemplateParses pins the ALTER PAGE half. A template has no +// name, so it cannot be reached through widgetRef like every other DROP target — +// it is addressed by the entity it renders plus the list view holding it, since +// one page can carry two list views with a template for the same entity. +func TestAlterPageDropTemplateParses(t *testing.T) { + prog, errs := Build(`ALTER PAGE Pages.Vehicle_Overview { + DROP TEMPLATE FOR Pages.SUV IN vehicleListView + };`) + if len(errs) > 0 { + for _, err := range errs { + t.Errorf("parse error: %v", err) + } + t.FailNow() + } + stmt, ok := prog.Statements[0].(*ast.AlterPageStmt) + if !ok { + t.Fatalf("expected AlterPageStmt, got %T", prog.Statements[0]) + } + if len(stmt.Operations) != 1 { + t.Fatalf("expected 1 operation, got %d", len(stmt.Operations)) + } + op, ok := stmt.Operations[0].(*ast.DropListViewTemplateOp) + if !ok { + t.Fatalf("expected DropListViewTemplateOp, got %T", stmt.Operations[0]) + } + if op.Specialization != "Pages.SUV" { + t.Errorf("Specialization = %q, want Pages.SUV", op.Specialization) + } + if op.ListView != "vehicleListView" { + t.Errorf("ListView = %q, want vehicleListView", op.ListView) + } +} + +// TestAlterPageInsertTemplateParses pins that adding a template reuses INSERT +// INTO with the same `template for` block CREATE PAGE uses — one spelling of a +// template everywhere, rather than a second INSERT TEMPLATE form. +func TestAlterPageInsertTemplateParses(t *testing.T) { + prog, errs := Build(`ALTER PAGE Pages.Vehicle_Overview { + INSERT INTO vehicleListView { + template for Pages.Motorcycle { dynamictext mcLabel (Content: 'm') } + } + };`) + if len(errs) > 0 { + for _, err := range errs { + t.Errorf("parse error: %v", err) + } + t.FailNow() + } + stmt := prog.Statements[0].(*ast.AlterPageStmt) + op, ok := stmt.Operations[0].(*ast.InsertWidgetOp) + if !ok { + t.Fatalf("expected InsertWidgetOp, got %T", stmt.Operations[0]) + } + if len(op.Widgets) != 1 { + t.Fatalf("expected 1 inserted node, got %d", len(op.Widgets)) + } + if got := op.Widgets[0].Specialization; got != "Pages.Motorcycle" { + t.Errorf("Specialization = %q, want Pages.Motorcycle", got) + } +} diff --git a/mdl/visitor/visitor_page_v3.go b/mdl/visitor/visitor_page_v3.go index 1c8cd910a..a1db9e2cb 100644 --- a/mdl/visitor/visitor_page_v3.go +++ b/mdl/visitor/visitor_page_v3.go @@ -511,6 +511,20 @@ func buildWidgetV3(ctx parser.IWidgetV3Context, b *Builder) *ast.WidgetV3 { Children: []*ast.WidgetV3{}, } + // A List View specialization template: `template for Module.Entity { ... }`. + // It has no name — the entity is what identifies it — so this returns before + // the name lookup below, which would otherwise find nothing. + if wCtx.FOR() != nil && wCtx.TEMPLATE() != nil { + widget.Type = "template" + if qn := wCtx.QualifiedName(); qn != nil { + widget.Specialization = qn.GetText() + } + if bodyCtx := wCtx.WidgetBodyV3(); bodyCtx != nil { + widget.Children = buildWidgetBodyV3(bodyCtx, b) + } + return widget + } + // Get widget type if wCtx.PLUGGABLEWIDGET() != nil { widget.Type = "pluggablewidget" diff --git a/mdl/visitor/visitor_query.go b/mdl/visitor/visitor_query.go index 19b819d9f..5b384ce42 100644 --- a/mdl/visitor/visitor_query.go +++ b/mdl/visitor/visitor_query.go @@ -78,7 +78,12 @@ func (b *Builder) ExitShowStatement(ctx *parser.ShowStatementContext) { } } b.statements = append(b.statements, stmt) - } else if ctx.ENTITY() != nil { + } else if ctx.ENTITY() != nil && ctx.ACCESS() == nil { + // SHOW ENTITY Module.Entity. The ACCESS guard is load-bearing: this + // branch runs long before the ACCESS one below, so without it + // `SHOW ACCESS ON ENTITY Module.Entity` would answer with the entity's + // definition instead of its access rules — a wrong answer rather than an + // error, which is the harder kind to notice. if qn := ctx.QualifiedName(); qn != nil { name := buildQualifiedName(qn) b.statements = append(b.statements, &ast.ShowStmt{ @@ -184,12 +189,18 @@ func (b *Builder) ExitShowStatement(ctx *parser.ShowStatementContext) { } } b.statements = append(b.statements, stmt) - } else if ctx.PAGE() != nil { + } else if ctx.PAGE() != nil && ctx.ACCESS() == nil { // `SHOW PAGE Module.Page` (singular) is an alias for DESCRIBE PAGE, the // same way SHOW ENTITY / SHOW ASSOCIATION read as their describe. The // grammar alternative existed with no visitor branch, so the statement // parsed to nothing and the command exited 0 printing nothing — which // reads as "this page is empty", not "this command does nothing". + // + // The ACCESS guard is the sibling of the one on the ENTITY branch above: + // this branch sits ~200 lines before the ACCESS one, so without it + // `SHOW ACCESS ON PAGE Module.Page` printed the page's whole definition + // instead of its allowed roles. Found by the #925 test sweep — the + // spelling parsed, which is why it was reported as working. if qn := ctx.QualifiedName(); qn != nil { b.statements = append(b.statements, &ast.DescribeStmt{ ObjectType: ast.DescribePage, @@ -399,7 +410,9 @@ func (b *Builder) ExitShowStatement(ctx *parser.ShowStatementContext) { // SHOW DEMO USERS b.statements = append(b.statements, &ast.ShowStmt{ObjectType: ast.ShowDemoUsers}) } else if ctx.ACCESS() != nil { - // SHOW ACCESS ON [MICROFLOW|PAGE|WORKFLOW|NANOFLOW] Module.Entity + // SHOW ACCESS ON [ENTITY|MICROFLOW|PAGE|WORKFLOW|NANOFLOW] Module.Name. + // A bare name and an explicit ENTITY both fall through to ShowAccessOn, + // which reports entity access rules. if qn := ctx.QualifiedName(); qn != nil { name := buildQualifiedName(qn) if ctx.MICROFLOW() != nil { diff --git a/mdl/visitor/visitor_queue.go b/mdl/visitor/visitor_queue.go index a7ff58cc1..a90fdebb0 100644 --- a/mdl/visitor/visitor_queue.go +++ b/mdl/visitor/visitor_queue.go @@ -16,6 +16,9 @@ func (b *Builder) ExitCreateQueueStatement(ctx *parser.CreateQueueStatementConte Name: buildQualifiedName(ctx.QualifiedName()), Documentation: findDocCommentText(ctx), } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } if createStmt := findParentCreateStatement(ctx); createStmt != nil { if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { stmt.CreateOrModify = true diff --git a/mdl/visitor/visitor_regularexpression.go b/mdl/visitor/visitor_regularexpression.go index f81e63fcc..3007746c9 100644 --- a/mdl/visitor/visitor_regularexpression.go +++ b/mdl/visitor/visitor_regularexpression.go @@ -16,6 +16,9 @@ func (b *Builder) ExitCreateRegularExpressionStatement(ctx *parser.CreateRegular Name: buildQualifiedName(ctx.QualifiedName()), Documentation: findDocCommentText(ctx), } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } if createStmt := findParentCreateStatement(ctx); createStmt != nil { if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { stmt.CreateOrModify = true diff --git a/mdl/visitor/visitor_scheduledevent.go b/mdl/visitor/visitor_scheduledevent.go index 5a52640b3..8c945376c 100644 --- a/mdl/visitor/visitor_scheduledevent.go +++ b/mdl/visitor/visitor_scheduledevent.go @@ -22,6 +22,9 @@ func (b *Builder) ExitCreateScheduledEventStatement(ctx *parser.CreateScheduledE Name: buildQualifiedName(ctx.QualifiedName()), Documentation: findDocCommentText(ctx), } + if lit := ctx.STRING_LITERAL(); lit != nil { + stmt.Folder = unquoteString(lit.GetText()) + } if createStmt := findParentCreateStatement(ctx); createStmt != nil { if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { stmt.CreateOrModify = true diff --git a/mdl/visitor/visitor_show_access_test.go b/mdl/visitor/visitor_show_access_test.go new file mode 100644 index 000000000..a329694a6 --- /dev/null +++ b/mdl/visitor/visitor_show_access_test.go @@ -0,0 +1,98 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + mdlast "github.com/mendixlabs/mxcli/mdl/ast" +) + +// #925: `SHOW ACCESS ON ENTITY Mod.E` was a parse error while the MICROFLOW and +// PAGE spellings parsed, even though the CLAUDE.md `mxcli init` writes into every +// project documents all three. The cause is that `ENTITY` is in the grammar's +// `keyword` rule, so `ACCESS ON qualifiedName` matched the word ENTITY as the +// whole name and the real name became extraneous input: +// +// line 1:27 extraneous input '.' expecting the start of a statement +// +// The two halves of the fix are covered here: the grammar alternative (without +// it these do not parse at all) and the visitor's ACCESS guard (without it the +// long-preceding ENTITY branch answers with the entity's DEFINITION instead of +// its access rules — a wrong answer rather than an error). +func TestShowAccessOnEntity(t *testing.T) { + for _, src := range []string{ + "SHOW ACCESS ON ENTITY Mod.Customer", + "LIST ACCESS ON ENTITY Mod.Customer", + `SHOW ACCESS ON ENTITY "Mod"."Customer"`, + "show access on entity Mod.Customer", + } { + s := parseOneShow(t, src) + if s.ObjectType != mdlast.ShowAccessOn { + t.Errorf("%q: ObjectType = %v, want ShowAccessOn (entity access rules)", + src, s.ObjectType) + } + if s.Name == nil || s.Name.String() != "Mod.Customer" { + t.Errorf("%q: Name = %v, want Mod.Customer", src, s.Name) + } + } +} + +// The explicit spelling must be the bare one's synonym, not a near-miss: both +// reach listAccessOnEntity, so a divergence here would be two commands with the +// same name and different answers. +func TestShowAccessOnEntityMatchesBareForm(t *testing.T) { + bare := parseOneShow(t, "SHOW ACCESS ON Mod.Customer") + explicit := parseOneShow(t, "SHOW ACCESS ON ENTITY Mod.Customer") + + if bare.ObjectType != explicit.ObjectType { + t.Errorf("bare = %v, explicit = %v — the two spellings must agree", + bare.ObjectType, explicit.ObjectType) + } + if bare.Name.String() != explicit.Name.String() { + t.Errorf("bare name = %q, explicit name = %q", + bare.Name.String(), explicit.Name.String()) + } +} + +// False-positive control for the ACCESS guard: the plain SHOW ENTITY branch must +// still produce ShowEntity, and the other ACCESS ON spellings must keep +// their own object types. Without this the guard could "pass" by routing every +// ENTITY statement to the access path. +func TestShowAccessOnKindsStayDistinct(t *testing.T) { + for _, tc := range []struct { + src string + want mdlast.ShowObjectType + }{ + {"SHOW ENTITY Mod.Customer", mdlast.ShowEntity}, + {"SHOW ACCESS ON Mod.Customer", mdlast.ShowAccessOn}, + {"SHOW ACCESS ON ENTITY Mod.Customer", mdlast.ShowAccessOn}, + {"SHOW ACCESS ON MICROFLOW Mod.ACT_Do", mdlast.ShowAccessOnMicroflow}, + {"SHOW ACCESS ON PAGE Mod.Home", mdlast.ShowAccessOnPage}, + {"SHOW ACCESS ON WORKFLOW Mod.Approve", mdlast.ShowAccessOnWorkflow}, + {"SHOW ACCESS ON NANOFLOW Mod.NAV_Go", mdlast.ShowAccessOnNanoflow}, + } { + if got := parseOneShow(t, tc.src).ObjectType; got != tc.want { + t.Errorf("%q: ObjectType = %v, want %v", tc.src, got, tc.want) + } + } +} + +func parseOneShow(t *testing.T, src string) *mdlast.ShowStmt { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse %q: %v", src, errs) + } + if prog == nil { + t.Fatalf("%q: no program", src) + } + if len(prog.Statements) != 1 { + t.Fatalf("%q: got %d statements, want 1", src, len(prog.Statements)) + } + s, ok := prog.Statements[0].(*mdlast.ShowStmt) + if !ok { + t.Fatalf("%q: got %T, want *ast.ShowStmt", src, prog.Statements[0]) + } + return s +} diff --git a/mdl/visitor/visitor_workflow.go b/mdl/visitor/visitor_workflow.go index eb39e8b05..3b6a727b3 100644 --- a/mdl/visitor/visitor_workflow.go +++ b/mdl/visitor/visitor_workflow.go @@ -29,20 +29,19 @@ func (b *Builder) ExitCreateWorkflowStatement(ctx *parser.CreateWorkflowStatemen } } - // Parse DISPLAY, DESCRIPTION, EXPORT LEVEL, DUE DATE using positional STRING_LITERAL indexing - allStrings := ctx.AllSTRING_LITERAL() - stringIdx := 0 - - // DISPLAY 'text' - if ctx.DISPLAY() != nil && stringIdx < len(allStrings) { - stmt.DisplayName = unquoteString(allStrings[stringIdx].GetText()) - stringIdx++ + // Each optional string clause is read by its grammar LABEL, not by counting + // STRING_LITERALs. The positional version worked only while the clauses + // happened to be the rule's only strings: adding the FOLDER clause would + // have made allStrings[0] the folder path whenever one was given, so a + // foldered workflow would have silently taken its display name from it. + if tok := ctx.GetFolder(); tok != nil { + stmt.Folder = unquoteString(tok.GetText()) } - - // DESCRIPTION 'text' - if ctx.DESCRIPTION() != nil && stringIdx < len(allStrings) { - stmt.Description = unquoteString(allStrings[stringIdx].GetText()) - stringIdx++ + if tok := ctx.GetDisplay(); tok != nil { + stmt.DisplayName = unquoteString(tok.GetText()) + } + if tok := ctx.GetDescription(); tok != nil { + stmt.Description = unquoteString(tok.GetText()) } // EXPORT LEVEL (Identifier | API) @@ -71,11 +70,9 @@ func (b *Builder) ExitCreateWorkflowStatement(ctx *parser.CreateWorkflowStatemen _ = overviewPageIdx // Parse DUE DATE 'expression' - if ctx.DUE() != nil && stringIdx < len(allStrings) { - stmt.DueDate = unquoteString(allStrings[stringIdx].GetText()) - stringIdx++ + if tok := ctx.GetDueDate(); tok != nil { + stmt.DueDate = unquoteString(tok.GetText()) } - _ = stringIdx // Parse CREATE OR MODIFY createStmt := findParentCreateStatement(ctx) diff --git a/modelsdk/gen/pages/storagename_listviewtemplate_test.go b/modelsdk/gen/pages/storagename_listviewtemplate_test.go new file mode 100644 index 000000000..7fd10ec1e --- /dev/null +++ b/modelsdk/gen/pages/storagename_listviewtemplate_test.go @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: Apache-2.0 + +package pages + +import ( + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// TestListViewTemplateUsesStorageName pins the BSON key of a List View +// specialization template's entity. +// +// The generator that produced this package bound the property by its SDK name, +// "Specialization". Mendix stores it as "Entity". Both the in-repo generator and +// real documents agree: +// +// generated/metamodel/types.go +// Specialization model.QualifiedName `json:"entity"` +// ^ SDK name ^ storage name +// +// ako/TestApp, Pages.Vehicle_Overview — all four Studio Pro authored templates +// {$ID, $Type, Entity, Widgets} +// +// Encode and decode are separate literals and have drifted before, so both are +// pinned here. Getting this wrong is silent in the usual way: a template written +// under "Specialization" carries a property Mendix does not have, and a reader +// keyed on it returns an empty entity for every real document — a template that +// matches nothing, with `mx check` still reporting 0 errors. +func TestListViewTemplateUsesStorageName(t *testing.T) { + const storageKey = "Entity" + + t.Run("encode", func(t *testing.T) { + o := NewListViewTemplate() + o.SetSpecializationQualifiedName("Pages.Bus") + + var found bool + for _, p := range o.Properties() { + if p.Name() == storageKey { + found = true + } + if p.Name() == "Specialization" { + t.Errorf("property is bound as %q — that is the SDK name, not the key on disk", p.Name()) + } + } + if !found { + t.Errorf("no %q property; this key is written to the .mxunit", storageKey) + } + }) + + t.Run("decode", func(t *testing.T) { + raw, err := bson.Marshal(bson.D{ + {Key: "$Type", Value: "Forms$ListViewTemplate"}, + {Key: storageKey, Value: "Pages.Bus"}, + }) + if err != nil { + t.Fatalf("marshal: %v", err) + } + + o := initListViewTemplate() + o.InitFromRaw(bson.Raw(raw)) + + if got := o.SpecializationQualifiedName(); got != "Pages.Bus" { + t.Errorf("decoded %q, want Pages.Bus — InitFromRaw is reading the wrong key, so "+ + "every stored template reads back with no specialization", got) + } + }) +} diff --git a/modelsdk/gen/pages/types.go b/modelsdk/gen/pages/types.go index bee94d7b1..8ac0e6e0a 100644 --- a/modelsdk/gen/pages/types.go +++ b/modelsdk/gen/pages/types.go @@ -15633,7 +15633,8 @@ func (o *ListViewTemplate) RemoveWidgets(index int) { // InitFromRaw populates lazy-decoded property holders from raw BSON. func (o *ListViewTemplate) InitFromRaw(raw bson.Raw) { - if val, err := raw.LookupErr("Specialization"); err == nil { + // STORAGE-NAME OVERRIDE: see initListViewTemplate. Key is "Entity". + if val, err := raw.LookupErr("Entity"); err == nil { if s, ok := val.StringValueOK(); ok { o.specialization.SetFromDecode(s) } @@ -30486,7 +30487,14 @@ func NewListViewSearch() *ListViewSearch { func initListViewTemplate() *ListViewTemplate { o := &ListViewTemplate{} o.SetTypeName("Forms$ListViewTemplate") - o.specialization = property.NewByNameRef[element.Element]("Specialization", "DomainModels$Entity") + // STORAGE-NAME OVERRIDE: BSON key is "Entity", not the SDK name + // "Specialization". generated/metamodel agrees — + // Specialization model.QualifiedName `json:"entity"` + // where the tag is the storage name — and all four templates in the + // Vehicle_Overview page of ako/TestApp (Mendix 10.24, Studio Pro authored) + // store {$ID, $Type, Entity, Widgets}. This generator kept only the SDK name. + // Must match the InitFromRaw decode key. + o.specialization = property.NewByNameRef[element.Element]("Entity", "DomainModels$Entity") o.specialization.Bind(&o.Base, 0) o.widget = property.NewPart[element.Element]("Widget") o.widget.Bind(&o.Base, 1) @@ -31543,7 +31551,20 @@ func initPage() *Page { o.style.Bind(&o.Base, 10) o.appearance = property.NewPart[element.Element]("Appearance") o.appearance.Bind(&o.Base, 11) - o.allowedRoles = property.NewByNameRefListV3[element.Element]("AllowedModuleRoles", "Security$ModuleRole") // STORAGE-NAME OVERRIDE: real BSON key is AllowedModuleRoles + // STORAGE-NAME OVERRIDE: real BSON key is AllowedModuleRoles. + // LIST-MARKER OVERRIDE: version 1, not 3. Studio Pro writes marker 1 on every + // page of a blank 11.13 app (16 of 16, empty lists included), and marker 3 with + // a NON-EMPTY role list is what makes `mx create-module-package` abort with + // "Unable to cast object of type 'Newtonsoft.Json.Linq.JValue' to type + // 'Newtonsoft.Json.Linq.JObject'" in MprDocumentHasher — the marker tells the + // reader what the entries are, and the entries here are qualified-name STRINGS. + // The empty case never crashed because there was nothing to mis-cast, which is + // why only a module with a page carrying a role hit it (upstream #931). + // The comment on NewByNameRefListV3 claims marker 1 raises CE0557: measured on + // 11.13 at security Off, Prototype AND Production, marker 1 with a role set is + // 0 errors and exports; marker 3 is 0 errors and crashes the exporter. The + // legacy engine (sdk/mpr/writer_security.go) has always written 1. + o.allowedRoles = property.NewByNameRefList[element.Element]("AllowedModuleRoles", "Security$ModuleRole") o.allowedRoles.Bind(&o.Base, 12) o.popupCloseAction = property.NewPrimitive[string]("PopupCloseAction", property.DecodeString) o.popupCloseAction.Bind(&o.Base, 13) diff --git a/modelsdk/mpr/writer_core.go b/modelsdk/mpr/writer_core.go index a71e7bfef..3a060b94e 100644 --- a/modelsdk/mpr/writer_core.go +++ b/modelsdk/mpr/writer_core.go @@ -578,15 +578,38 @@ func (w *Writer) DeleteUnit(unitID string) error { // MoveUnit reparents a unit to a new container (used by MOVE for top-level // documents like enumerations/constants/microflows). The container lives in the // Unit table; contents files are keyed by UnitID, so only the row changes. +// +// Counted in WriteStats like any other write, and elided when the unit is +// already in that container. Both matter: a move is the one mutation that +// changes a document's placement without changing a byte of its contents, so +// without the count ReportMutation calls a real move "Unchanged", and without +// the elision re-running an already-applied script dirties the .mpr — the two +// halves of ADR-0008's guarantee, applied to the row instead of the blob. func (w *Writer) MoveUnit(unitID, newContainerID string) error { + w.writesOffered++ + target := uuidToBlob(newContainerID) + if stored, err := w.containerOfUnit(unitID); err == nil && bytes.Equal(stored, target) { + return nil + } _, err := w.reader.db.Exec(`UPDATE Unit SET ContainerID = ? WHERE UnitID = ?`, - uuidToBlob(newContainerID), uuidToBlob(unitID)) + target, uuidToBlob(unitID)) if err == nil { + w.writesLanded++ w.reader.InvalidateCache() } return err } +// containerOfUnit reads a unit's stored ContainerID blob. Compared as bytes +// rather than as a formatted UUID so the check cannot disagree with the write +// about spelling or byte order. +func (w *Writer) containerOfUnit(unitID string) ([]byte, error) { + var stored []byte + err := w.reader.db.QueryRow(`SELECT ContainerID FROM Unit WHERE UnitID = ?`, + uuidToBlob(unitID)).Scan(&stored) + return stored, err +} + func (w *Writer) deleteUnit(unitID string) error { // Convert UUID string to 16-byte blob unitIDBlob := uuidToBlob(unitID) diff --git a/modelsdk/property/reference.go b/modelsdk/property/reference.go index c35213094..ce69313d9 100644 --- a/modelsdk/property/reference.go +++ b/modelsdk/property/reference.go @@ -43,9 +43,15 @@ func NewByNameRefList[T element.Element](name, targetType string) *ByNameRefList } // NewByNameRefListV3 creates a ByNameRefList with BSON version marker int32(3). -// Required for AllowedRoles on Forms$Page (document-level access control). -// Mendix Studio Pro uses version 3 for page AllowedRoles; using version 1 causes -// CE0557 ("At least one allowed role must be selected") even when roles are set. +// +// NOT for AllowedRoles on Forms$Page, despite what this comment used to say. The +// claim that marker 1 raises CE0557 there does not reproduce: measured on mxbuild +// 11.13 at security Off, Prototype and Production, a page with marker 1 and a +// role set is 0 errors and exports cleanly, while marker 3 is 0 errors and aborts +// `mx create-module-package` in MprDocumentHasher with "Unable to cast object of +// type 'Newtonsoft.Json.Linq.JValue' to type 'Newtonsoft.Json.Linq.JObject'". +// Studio Pro writes marker 1 on all 16 pages of a blank app, empty lists included. +// See the LIST-MARKER OVERRIDE in gen/pages initPage (upstream #931). func NewByNameRefListV3[T element.Element](name, targetType string) *ByNameRefList[T] { return &ByNameRefList[T]{propertyBase: propertyBase{name: name}, targetType: targetType, versionMarker: 3} } diff --git a/sdk/mpr/writer_domainmodel.go b/sdk/mpr/writer_domainmodel.go index 027fcd79e..f89478ffa 100644 --- a/sdk/mpr/writer_domainmodel.go +++ b/sdk/mpr/writer_domainmodel.go @@ -3,6 +3,7 @@ package mpr import ( + "bytes" "fmt" "sort" "strings" @@ -351,12 +352,25 @@ func (w *Writer) MoveUnitByID(unitID string, newContainerID string) error { return w.moveUnitByID(unitID, newContainerID) } +// Counted in WriteStats and elided when the unit already sits in that +// container, for the reasons on the modelsdk engine's MoveUnit: a move changes +// placement without changing contents, so it is invisible to both halves of +// ADR-0008 unless the row update accounts for itself. func (w *Writer) moveUnitByID(unitID string, newContainerID string) error { + w.writesOffered++ unitIDBlob := uuidToBlob(unitID) containerIDBlob := uuidToBlob(newContainerID) + var stored []byte + if err := w.reader.db.QueryRow(`SELECT ContainerID FROM Unit WHERE UnitID = ?`, unitIDBlob).Scan(&stored); err == nil { + if bytes.Equal(stored, containerIDBlob) { + return nil + } + } + _, err := w.reader.db.Exec(`UPDATE Unit SET ContainerID = ? WHERE UnitID = ?`, containerIDBlob, unitIDBlob) if err == nil { + w.writesLanded++ w.reader.InvalidateCache() } return err diff --git a/sdk/mpr/writer_listview_template_test.go b/sdk/mpr/writer_listview_template_test.go new file mode 100644 index 000000000..c0724e427 --- /dev/null +++ b/sdk/mpr/writer_listview_template_test.go @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "testing" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" + "go.mongodb.org/mongo-driver/bson" +) + +// TestSerializeListViewTemplateCarriesTheSpecialization pins the legacy engine's +// half of #940. +// +// The writer already emitted Forms$ListViewTemplate elements, but with only +// {$ID, $Type, Widgets} — no entity — so every template it wrote matched nothing +// and rendered never. Studio Pro's own documents (ako/TestApp, +// Pages.Vehicle_Overview) carry the entity under the storage name "Entity", not +// the SDK name "Specialization". +func TestSerializeListViewTemplateCarriesTheSpecialization(t *testing.T) { + lv := &pages.ListView{ + BaseWidget: pages.BaseWidget{ + BaseElement: model.BaseElement{ID: model.ID("lv"), TypeName: "Forms$ListView"}, + Name: "vehicleListView", + }, + Templates: []*pages.ListViewTemplate{ + {BaseElement: model.BaseElement{ID: model.ID("t1")}, Specialization: "Pages.Bus"}, + {BaseElement: model.BaseElement{ID: model.ID("t2")}, Specialization: "Pages.Truck"}, + }, + } + + doc := serializeListView(lv) + + var templates bson.A + for _, e := range doc { + if e.Key == "Templates" { + templates, _ = e.Value.(bson.A) + } + } + // The first element is the typed-array marker, not a template. + if len(templates) != 3 { + t.Fatalf("Templates has %d element(s) (marker + templates), want 3", len(templates)) + } + + want := []string{"Pages.Bus", "Pages.Truck"} + for i, wantEntity := range want { + tpl, ok := templates[i+1].(bson.D) + if !ok { + t.Fatalf("template %d is %T, want bson.D", i, templates[i+1]) + } + var got string + var keys []string + for _, e := range tpl { + keys = append(keys, e.Key) + if e.Key == "Entity" { + got, _ = e.Value.(string) + } + } + if got != wantEntity { + t.Errorf("template %d Entity = %q, want %q (keys present: %v)", i, got, wantEntity, keys) + } + // Key order matches Studio Pro's documents. + if len(keys) != 4 || keys[0] != "$ID" || keys[1] != "$Type" || keys[2] != "Entity" || keys[3] != "Widgets" { + t.Errorf("template %d keys = %v, want [$ID $Type Entity Widgets]", i, keys) + } + } +} diff --git a/sdk/mpr/writer_placement.go b/sdk/mpr/writer_placement.go new file mode 100644 index 000000000..fa4ef8b89 --- /dev/null +++ b/sdk/mpr/writer_placement.go @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + "go.mongodb.org/mongo-driver/bson" +) + +// MoveDocument reparents a top-level document unit, whatever its type. +// The idempotence and write accounting live in moveUnitByID. +func (w *Writer) MoveDocument(unitID, containerID model.ID) error { + if unitID == "" || containerID == "" { + return fmt.Errorf("MoveDocument: unit and container are both required") + } + return w.moveUnitByID(string(unitID), string(containerID)) +} + +// FindDocumentUnit locates a document by module and name through the unit +// table, whatever its type. Mirrors the modelsdk engine's implementation. +// +// Only units contained as "Documents" are considered: a module also holds its +// domain model, security and settings, and folders share the table, so the +// containment filter is what stops a folder named like a document from being +// returned as one. +func (w *Writer) FindDocumentUnit(moduleName, name string) (*types.DocumentUnit, error) { + modules, err := w.reader.ListModules() + if err != nil { + return nil, fmt.Errorf("FindDocumentUnit: list modules: %w", err) + } + var moduleID string + for _, m := range modules { + if m.Name == moduleName { + moduleID = string(m.ID) + break + } + } + if moduleID == "" { + return nil, nil + } + containers := buildContainerSet(w.reader, moduleID) + + var found *types.DocumentUnit + err = w.eachDocumentUnit(func(doc *types.DocumentUnit) bool { + if doc.Name != name || !containers[string(doc.ContainerID)] { + return true + } + found = doc + return false + }) + if err != nil { + return nil, fmt.Errorf("FindDocumentUnit: %w", err) + } + return found, nil +} + +// ListDocumentUnits returns every top-level document in the project. +func (w *Writer) ListDocumentUnits() ([]*types.DocumentUnit, error) { + var out []*types.DocumentUnit + if err := w.eachDocumentUnit(func(doc *types.DocumentUnit) bool { + out = append(out, doc) + return true + }); err != nil { + return nil, fmt.Errorf("ListDocumentUnits: %w", err) + } + return out, nil +} + +// eachDocumentUnit walks every "Documents" unit, decoding just enough of each +// to name it, and stops early when visit returns false. A unit whose contents +// will not decode is skipped rather than failing the whole walk. +func (w *Writer) eachDocumentUnit(visit func(*types.DocumentUnit) bool) error { + units, err := w.reader.listUnitsByType("") + if err != nil { + return fmt.Errorf("list units: %w", err) + } + for _, unit := range units { + if unit.ContainmentName != "Documents" { + continue + } + contents, err := w.reader.resolveContents(unit.ID, unit.Contents) + if err != nil || len(contents) == 0 { + continue + } + var raw bson.D + if err := bson.Unmarshal(contents, &raw); err != nil { + continue + } + name := "" + for _, elem := range raw { + if elem.Key == "Name" { + if s, ok := elem.Value.(string); ok { + name = s + } + break + } + } + if name == "" { + continue + } + if !visit(&types.DocumentUnit{ + ID: model.ID(unit.ID), + ContainerID: model.ID(unit.ContainerID), + Name: name, + Type: unit.Type, + Kind: types.DocumentKind(unit.Type), + }) { + return nil + } + } + return nil +} diff --git a/sdk/mpr/writer_widgets_display.go b/sdk/mpr/writer_widgets_display.go index 22b2dd7d0..a864fe8de 100644 --- a/sdk/mpr/writer_widgets_display.go +++ b/sdk/mpr/writer_widgets_display.go @@ -147,9 +147,13 @@ func serializeListView(lv *pages.ListView) bson.D { templateWidgets = append(templateWidgets, serializeWidget(w)) } } + // Key order and names match Studio Pro's own documents: $ID, $Type, + // Entity, Widgets. "Entity" is the storage name of the SDK's + // Specialization property — see pages.ListViewTemplate. template := bson.D{ {Key: "$ID", Value: idToBsonBinary(string(t.ID))}, {Key: "$Type", Value: "Forms$ListViewTemplate"}, + {Key: "Entity", Value: t.Specialization}, {Key: "Widgets", Value: templateWidgets}, } templates = append(templates, template) diff --git a/sdk/pages/pages_widgets_data.go b/sdk/pages/pages_widgets_data.go index 070c76710..09fd471c4 100644 --- a/sdk/pages/pages_widgets_data.go +++ b/sdk/pages/pages_widgets_data.go @@ -65,10 +65,17 @@ type ListView struct { Templates []*ListViewTemplate `json:"templates,omitempty"` } -// ListViewTemplate represents a template in a list view. +// ListViewTemplate represents a template in a list view: the body rendered for +// one specialization of the list view's entity. +// +// The tag is the STORAGE name, matching generated/metamodel — Mendix's SDK name +// for this property is Specialization but every Studio Pro document stores it +// under "Entity". A template carries nothing else: Studio Pro writes exactly +// {$ID, $Type, Entity, Widgets}. type ListViewTemplate struct { model.BaseElement - Widgets []Widget `json:"widgets,omitempty"` + Specialization string `json:"entity,omitempty"` + Widgets []Widget `json:"widgets,omitempty"` } // TemplateGrid represents a template grid widget.