bregctl: explain and check report structural paths with no file or line to go with them

[jeremi]

Problem

Both bregctl check --format json findings and bregctl explain output identify configuration by structural path, for example entities[2].fields[5].constraints[0]. Neither says which file the construct came from, nor where in that file it sits.

For a single-file project a human can resolve this by reading. For a project assembled from several files, or one that a tool wants to link back to source, the structural path is not enough. Every consumer has to re-parse the project files itself and re-implement the mapping from structural path to file position, which means re-implementing assumptions about how the loader merges files.

Request

Either of:

1. Source provenance inline: add file and range (start and end line/column) next to path in explain output and in check findings.
2. A separate map: bregctl explain locations --format json returning structuralPath -> { file, range } for every addressable construct.

Option 2 is cheaper to add without disturbing existing payload shapes, and it lets consumers opt in.

Provenance should also distinguish authored configuration from values that came from a module or from an engine default, since "this field is not in your file because the engine supplied it" is a different answer from "this is at line 40".

Why it matters

Any tool that shows a project's configuration and wants to let a reader jump to where a rule is written needs this. Without it, the tool is guessing.

[jeremi]

Assessed this against the compiler, and then put the cost split to a consumer that actually reads projects through the CLIs. The answer that came back reorders the ticket: the location half is the part not worth building, and the authored-versus-default half is the part that is.

The three asks have wildly different costs

1. Which file (registry.yaml vs modules/<id>/module.yaml): days. The compiler already threads origins through the entity and module merge, and the branch behind #1271 (not yet on main) puts module ids on the compiled model as CompiledEntity::source_module and module_origins. Adding an optional file to Diagnostic and generalising derived_source_path in crates/registry-bregctl/src/lib.rs:9098, today special-cased to derived.sql.* and implemented by re-deriving the path string and string-matching it across every module, is roughly 4-6 files and 150-300 lines. No parser change.

2. Which line (range): not available at all. serde_norway 0.9.42 is the only YAML parser registry-breg uses, and it exports no Spanned type and nothing span-like beyond Error::location(), a single point attached to a parse failure. Every position is discarded at deserialize time, before any diagnostic code runs. Getting line and column means either hand-rolling a span-aware deserializer across the ~144 authoring types in contract.rs, or adopting tree-sitter-yaml (already a workspace dependency, already used by registry-language-server for Relay V2 and Evidence authoring) and writing a CST-to-path resolver. That is a parser-layer project on the scale of a second language-server front end, not a bounded PR.

3. Authored versus engine default: architecturally independent of 2. Solving spans does nothing for it. There are 350 #[serde(default...)] sites in contract.rs; a field the author wrote and a field the engine filled deserialize to bit-identical Rust values, with no presence flag anywhere.

What the consumer said, which changes the order

Paraphrasing an answer given with numbers attached, from a tool that reads a project through bregctl check and the compiled model:

• File-only attribution is worth very little, and most of its value already landed. The source_module and module_origins work on the feat: close six authoring and contract tickets across BReg and Casework #1271 branch is what let that tool stop text-searching module files to decide where a member came from. A file on a diagnostic adds nothing it cannot already reach through the object, and in a single-registry.yaml project it tells the reader nothing at all.
• It already has file and line, without any upstream span work. It indexes the compiled model by id-keyed selector, resolves each finding to an object by stripping trailing path segments until a prefix matches, and reads the position from its own YAML index. Across four projects that resolved 37 findings out of 37. So the finding-to-position path is already closed, through the object, and an inline file would duplicate something derived more precisely elsewhere.
• A precise path matters more than file and range combined. That resolution rate depends entirely on the [id=x] selector dialect.
• The authored-versus-default half is correctness, not convenience. The failure mode is concrete: a value the engine supplied renders identically to a value the author wrote, so a screen tells a reader "your project sets retention to 30 days" when the project says nothing and the engine chose 30. For a compliance reader that is worse than showing nothing, because it is confidently wrong about what they are accountable for, and no spot-check catches it.

The real blocker is that there is no one path grammar

This is the part of the ticket I would fix first, and it is not what the ticket asks for.

The path format quoted above, entities[2].fields[5].constraints[0], comes from the serde_path_to_error route in contract.rs:3196-3207, which only fires when a document fails to parse. The semantic diagnostics, all 542 non-test Diagnostic::error/Diagnostic::finding construction sites in crates/registry-breg/src, each write their own path literal by hand. A census of those literals finds at least four incompatible dialects:

Dialect	Example	Distinct literals
Empty bracket, no index at all	entities[].primaryDataset, actions[].effects[].target.entity	150
Id-keyed with an explicit id=	entities[id={}].accessProfiles[id={}].readPaths	13
Keyed by some other key name	actions[id={}].permissions[profile={}].targets[entity={}].rowBoundaries	(subset of the above)
Bare id, no key name	entities[household].derived[demographics].sql	seen in the compiler's own contract tests

Only the second is resolvable by a consumer. entities[].primaryDataset resolves to entities, which is not an object, so the finding renders with no link and no position, and no file field repairs that: knowing it is in registry.yaml is not knowing which entity. The bare-id spelling fails to resolve too, and fails silently, producing an unlinked finding rather than an error.

So the single most valuable constraint to settle here is: every semantic diagnostic path is id-keyed, never empty-bracketed and never bare-id. That is worth more to a consumer than file and range together, and it is a prerequisite for option 2 in the ticket, since a structuralPath -> {file, range} map built on today's literals would have keys that do not line up with what check and explain actually emit.

A cheaper shape for the authored-versus-default half

Offered by the same consumer, and it looks right to me on inspection, though I have not prototyped it: the need is not per-field provenance inside the typed model, so it does not require touching 350 #[serde(default)] sites or adding presence wrappers. What a consumer needs is the set of paths the document actually contained. A second pass that deserializes the same bytes into a generic serde_norway::Value and walks it yields exactly that, with no parser change and no span work, and bregctl already retains the bytes (project_bytes on the captured source, lib.rs:1534, and the same per module). Emit it as a flat authoredPaths list beside the compiled model; anything in the model and absent from that list came from a default.

Two caveats I would want settled before building it:

• It has to be per-document, or carry the owning file, because a field supplied by a module is authored, just not in registry.yaml. That is the one place where the file question from ask 1 comes back, and it comes back cheaply, because you know which document you are walking.
• It only works once the path grammar above is canonical, and rendering an id-keyed selector while walking a generic Value requires knowing which key is the identity at each level. That is mechanical for the authoring shapes, which carry id, but it is not free.

If that shape holds, ask 3 is closer in cost to ask 1 than to ask 2, which inverts the build order the ticket implies.

Where this leaves the ticket

Proposed, not decided:

1. Settle one canonical id-keyed path grammar for semantic diagnostics, and migrate the 150 empty-bracket literals. This unblocks everything else and is independently the biggest win for a consumer.
2. Then authoredPaths, per document, for the authored-versus-default distinction.
3. Do not build inline file, or build it only as a by-product of step 2.
4. Line and column is a parser-layer project. It should be its own ticket with that named as its cost, not carried inside this one.

Facts verified directly: the serde_norway API surface, the 542 diagnostic sites, the 350 #[serde(default...)] sites, the dialect census, and that the raw bytes are retained. The consumer's figures (four projects, 37 of 37 findings resolved) are its measurements, not mine. It also notes it has never actually received an empty-bracket or bare-id path in practice, so the claim that those break it is read off its resolver's dialect rather than observed.

[jeremi]

Correcting my comment above

The consumer whose figures I quoted has since gone back over its own census and
corrected me on three points. Two sharpen the argument; one weakens a claim I made,
so I am recording all three rather than only the convenient ones.

1. The 37-of-37 number does not support the weight I put on it

I wrote that a precise path grammar is "worth more to a consumer than file and
range together", and leaned on 37 findings resolved out of 37 to say so.

For 8 of those 37 the limiting factor was granularity, not dialect. Those findings
attach to something that is not a distinct object in the compiled model: per-profile
target permissions inside an action are payload hanging off the action, not addressable
things in their own right. An id-keyed selector pointing at them has nothing to point
to. Canonicalising the grammar would not have resolved those 8 any better than
today's literals do.

So the grammar is worth more than file, and it remains a prerequisite for option 2
in the ticket. But "more than file and range combined" is overstated as posted,
and the resolution rate was never evidence for it, because a rate of 37/37 measured
through the consumer's own YAML index says nothing about what our path literals would
have achieved unaided.

2. I mischaracterised one of the two failure modes

I wrote that the bare-id spelling "fails silently, producing an unlinked finding
rather than an error." That blurs two genuinely different behaviours, and the
consumer was right to split them:

• A path that resolves to a non-object ancestor (entities[].primaryDataset →
  entities) renders attached to the wrong thing. That is the dangerous one, because
  it is confidently wrong rather than absent.
• A path that does not resolve at all (actions[].effects[effect={id}].clear)
  renders with no link. That is visible to the reader as a missing link. Unhelpful,
  but honest, and a reader can tell something is missing.

Calling the second "silent" was wrong. Only the first fails quietly, and the argument
for fixing the grammar rests on that one.

3. There is a fifth spelling, and I verified it

My census table listed four dialects. There is a fifth: applyTargets[entity={}],
at crates/registry-breg/src/change_request.rs:2033 and
crates/registry-breg/src/access.rs:407. It is a different collection from
targets[entity={}] in access.rs, not a synonym, so a consumer matching on
collection name has to know both. I checked this myself rather than taking it: two
occurrences, two distinct collections.

A sixth observation, mine, which I think is the better argument

While building a reproduction for the consumer I produced all four of the original
dialects from a single bregctl check run, by making three deliberate edits to a
committed acceptance project:

access.requirements.empty      entities[id=establishment].accessRequirements
constraint.field.unknown       entities[].constraints[]
index.fields.invalid           entities[].indexes[].fields
module.lock.digest_mismatch    project.modules[].digest

The third row is worse than anything in my original analysis.
entities[].indexes[].fields (crates/registry-breg/src/compiler.rs:3055,3066) drops
two identities: which entity, and which index. A consumer keying entities by id
cannot recover either, because the path never carried them. This is not degradation to
a resolvable ancestor. It is unresolvable, and the entity root is anonymous here, so it
is the ordinary shape rather than a corner case.

That is a better argument for canonicalising the grammar than any resolution rate,
because it does not depend on one consumer's measurements at all. The mechanism is
general: 542 hand-written path literals with nothing gating their shape drift toward
whatever was convenient at each call site. The same failure in a neighbouring product
is now #1274, where eighteen caseworkctl payloads have no envelope, no schema and no
conformance test, and one concept is already spelled two ways across surfaces as a
result.

Canonicalising the grammar fixes today's 150 empty-bracket literals. A gate is what
stops the next 150.

What stands

The ordering in my comment above is unchanged: grammar first, then authoredPaths,
inline file only as a by-product, line-and-column as its own ticket with a
parser-layer cost named. The authored-versus-default half is still correctness rather
than convenience. What changes is that the case for step 1 should be argued from the
unresolvable-path mechanism, not from 37 of 37.