fix(db-lookup): refuse write-shaped statements and see a composed statement in the lint (BACKLOG #1574, #1658) - #1206
wshallwshall wants to merge 4 commits into
Conversation
…comment (BACKLOG #1574, #1658)
_require_read_only read the statement's first six characters and then scanned
only for a chained ';'. Opening with SELECT or WITH never implied the rest was a
read, and T-SQL needs no ';' between statements, so the gate admitted:
SELECT * INTO staging_copy FROM patients
WITH doomed AS (...) DELETE FROM patients ...
WITH c AS (...) UPDATE patients SET mrn='X' FROM c
SELECT 1 UPDATE patients SET mrn='X'
SELECT 1 EXEC sp_who
All five were driven through the shipped predicate and passed it.
The statement is now tokenized first, so comments, string literals and quoted
identifiers (' ', " ", [ ], ` `) are skipped, and any write or authority keyword
outside them is refused. The head check is a whole word now, so SELECTX and
WITHOUT no longer pass as SELECT and WITH. An unterminated literal or comment is
refused rather than guessed at.
Two shapes stay admitted deliberately: a keyword inside a literal or a quoted
identifier is data, and MySQL's scalar INSERT()/TRUNCATE() are calls. EXEC and
EXECUTE get no such tolerance, because T-SQL EXEC('...') is the dynamic-SQL
shape this gate exists to refuse.
The shipped benign-CTE contract stays green: WITH cte AS (SELECT 1 AS c) SELECT
* FROM cte still passes, and so does a trailing ';'.
This closes the statement-shape limbs of both rows. It does NOT close #1574's
core ask, which is read-only AUTHORITY at the database boundary; see the PR body.
…at built it (BACKLOG #1658) _unsafe_lookup_hit handed only the call's own statement expression to _is_dynamic_string, which understood a JoinedStr, a BinOp and a .format at the call site and nothing else. An ast.Name fell through to False, so every shape that composed the statement a line earlier went unseen. Thirteen one-handler modules were driven through the shipped rule: six shapes flagged, six missed. The six missed ones, now flagged and pinned as named arms: stmt = f"..." then db_lookup(c, stmt) stmt = "..." % value then db_lookup(c, stmt) stmt = "..."; stmt += "..." + value dedent(f"...") " ".join([... f-string ...]) f"..." if cond else "..." A module-level constant is now visible inside a handler body too, so a concat against one still reads as composed. _is_dynamic_string now takes the enclosing scope's assignments and follows a Name through every value bound to it, treats an AugAssign with a non-literal right side as dynamic, reads through a wrapping call by its arguments and its receiver, and descends into a conditional expression and into list/tuple/set and comprehension elements. A name bound through itself terminates the walk. Two deliberate choices, both recorded in the docstrings. Every binding of a name is read, not the last one, because a statement interpolated in one branch of an if has no meaningful last binding in source order; this rule is a filter, not a boundary (ADR 0144), so it over-reports rather than miss the branch that interpolates. And a wrapper is followed by its arguments rather than by a list of blessed wrapper names, because such a list is always missing one. A call in a signature (a decorator or a default argument) has no scope entry and is read exactly as before, from its own expression, so no coverage is lost.
… cannot meet (BACKLOG #1574)
DatabaseLookupExecutor's class docstring said "Read-only is enforced (ADR 0010),
not merely documented". Three things make that false, and two of them survive
the statement-gate fix in this branch:
- ApplicationIntent=ReadOnly is honored only by a SQL Server Always-On read
replica and is a no-op elsewhere, which _build_dsn's own docstring already
conceded;
- the pools are opened autocommit=True, so a write that got past the
statement test commits rather than rolls back;
- a statement test is a shape test on text, and cannot bound a linked-server
pass-through or an over-privileged account.
ADR 0010 states the weaker and correct shape: "A read-only convention is by
design -- the executor neither commits nor exposes a write path." A docstring
asserting enforcement is a compensating control resting on a false premise,
which CLAUDE.md section 11 (SDS-3.7) forbids, and #1574's closing steps ask for
exactly this correction.
Three docstrings now say what each layer does and name the account privilege as
the control that actually refuses a write:
- DatabaseLookupExecutor (the enforcement claim itself), which also now records
that db_lookup is SQL-Server-only -- __init__ calls _build_dsn directly
rather than the _build_connection dialect dispatcher, so the generic ODBC
dialect is unreachable from here -- and why the pools are autocommit: T-SQL
has no SET TRANSACTION READ ONLY, so there is no read-only transaction to
open in its place;
- DatabaseLookupExecutor.query ("read-only enforced");
- _build_dsn, where "the statement guard is the load-bearing control" now says
that neither layer is authority.
The public db_lookup() docstring gains the same sentence, because that is the
surface a Handler author reads.
Prose only. No behaviour changes in this commit.
… (BACKLOG #1658) BACKLOG #1658's third closing step. A new subsection beside the db_lookup scope note says plainly that the account's privilege is what makes a lookup read-only, and that the engine's two in-process layers are defence in depth rather than authority: the statement gate reads text, and ApplicationIntent is honored only by an Always-On read replica. It also records why there is no read-only transaction to fall back on. T-SQL has no SET TRANSACTION READ ONLY; SQL Server's read-only mechanisms are a read-only database or filegroup, a snapshot, or ApplicationIntent against an availability- group replica, and every one of those is operator provisioning. One separation is called out because the two are easy to conflate: a DatabaseLookup credential is a partner-database principal, not the [store] login the engine writes its own messages with. Kept deliberately local to the db_lookup rows -- PR 1178 holds this file.
LANDER INSPECTION -- blocking on one bytePosted under the korus The tokenizer is a real improvement and closes every evasion it was written against. Leading The blocker: a carriage return reopens comment masking
nl = statement.find("\n", i) # database.py:614Driven against this PR's own predicate, lifted from the PR head: Payload: SQL Server's lexer ends a Why CR rather than any exotic byte: HL7 v2 uses carriage return as its segment terminator. A CR It is also the same class as the PR's own test Fail-closed, except exactly here
Second change, cheaper: the docs claim more than the predicate deliversThe vocabulary is fourteen words, not "writes". Admitted as a chained second statement today:
This PR invokes CLAUDE.md section 11 against exactly this shape of overclaim in the docstring it What is clean
To unblock
Verdict: block. |
Closes the statement-shape and lint limbs of BACKLOG #1574 and BACKLOG #1658. They are one edit described twice, which is why they are one PR: the rewrite that refuses a write keyword outside literals and comments closes #1574's
SELECT INTOand CTE-terminal-write limbs and #1658's no-semicolon-chain limb in the same change.Filed together at source: #1658 says "this item and #1574 close together; neither is complete without the other."
What was wrong, measured before the fix
_require_read_onlyreadstripped[:6].upper()forSELECT/WITHand then scanned only for a chained;. Driven directly, the shipped predicate ADMITTED all of these:_unsafe_lookup_hithanded only the call's own expression to_is_dynamic_string, so anast.Namefell through toFalse. Thirteen one-handler modules: six composition shapes flagged, six missed.What this PR does
1. The statement gate (
messagefoundry/transports/database.py,201866d65). The statement is tokenized first, so comments, string literals and quoted identifiers (' '," ",[ ],` `) are skipped, and then any write or authority keyword outside them is refused. The head check is a whole word now, soSELECTXandWITHOUTno longer pass asSELECTandWITH. An unterminated literal or comment is refused rather than guessed at.Two shapes stay admitted on purpose: a keyword inside a literal or a quoted identifier is data, and MySQL's scalar
INSERT()/TRUNCATE()are calls, not statements.EXECandEXECUTEget no such tolerance, because T-SQLEXEC('...')is the dynamic-SQL shape this gate exists to refuse.The shipped benign-CTE contract stays green.
WITH cte AS (SELECT 1 AS c) SELECT * FROM ctestill passes, and so does a trailing;.2. The lint (
messagefoundry/checks.py,69419b1f4)._is_dynamic_stringnow takes the enclosing scope's assignments and follows aNamethrough every value bound to it, treats anAugAssignwith a non-literal right side as dynamic, reads through a wrapping call by its arguments and its receiver, and descends into a conditional expression and into list/tuple/set and comprehension elements. A name bound through itself terminates the walk.Two deliberate choices, both recorded in the docstrings:
ifhas no meaningful last binding in source order, and this rule is a filter, not a boundary (ADR 0144), so it over-reports rather than miss the branch that interpolates. That is a wider reading than #1658's suggested single-assignment walk.A call in a signature (a decorator or a default argument) has no scope entry and is read exactly as before, from its own expression, so no coverage is lost.
3. Docstring corrections (
a5baa2ed2) and 4. the operator recommendation (932532e1b) — see the next two sections.What is NOT closed, and needs the owner
#1574's core ask is read-only AUTHORITY at the database boundary, and this PR does not deliver it. The row asks to "enforce read-only authority at the database boundary -- a read-only connection or transaction per dialect" and to "confirm the account itself cannot write", with acceptance on live backends with disposable fixtures. Its own Verification-limits paragraph concedes no live database ran. That still holds here: nothing in this PR was driven against a real SQL Server.
Two findings that bound what a code change could even do:
SET TRANSACTION READ ONLY, and nothing in the engine issues one. SQL Server's read-only mechanisms are a read-only database or filegroup, a snapshot, orApplicationIntentagainst an availability-group replica. Every one of those is operator provisioning, not something the connector can assert. That is now written into the docstrings and intodocs/CONNECTIONS.mdrather than left as a gap.db_lookupis SQL-Server-only, so there is no per-dialect story to write.DatabaseLookupExecutor.__init__calls_build_dsn(dict(s), read_only=True)directly rather than the_build_connectiondialect dispatcher, so thegenericODBC dialect the DATABASE connector accepts is unreachable from a lookup. ADR 0010 says "SQL Server backend only." SQLite and PostgreSQL are store backends and a different question.So the remaining live option is provisioning a
db_datareader-class login, which is an operator act. This PR documents it (docs/CONNECTIONS.md); it cannot enforce it. Leave both rows open.One separate finding, recorded not fixed
ADR 0010's stated basis for read-only is false against the shipped pool. The ADR says "the executor neither commits nor exposes a write path", and
DatabaseLookupExecutor._get_poolopens its pools withautocommit=True. A write that got past the predicate would commit, not roll back. That is the ADR's premise, independent of the predicate fix in this PR and of the authority question above.It appears to be unfiled. Naming the subject rather than a number, deliberately: a citation to an unallocated number resolves to nothing today and to unrelated work the day someone allocates it.
This PR corrects the docstrings that repeated the overclaim —
DatabaseLookupExecutorsaid "Read-only is enforced (ADR 0010), not merely documented", which CLAUDE.md section 11 (SDS-3.7) forbids as a compensating control resting on a false premise. It does not amend the ADR.Collision to resolve at merge
docs/CONNECTIONS.mdis held by open PR 1178, and rows #1249 and #1624 are also recorded as contending for it. My edit is one new subsection beside the existingdb_lookupscope note, in its own commit, last (932532e1b), so it can be read and re-placed in one look. Nothing else in this PR touches that file.Ledger
docs/BACKLOG.mdis untouched. It is a deliberate stub in this repository since 2026-09-13 and the real ledger is vault-only; a builder does not flip banners. The required "a PR that implements BACKLOG #N must update BACKLOG.md" context is expected to pass without the edit, as it did on PRs 1160 and 1161. If it refuses, that is a finding, not something to work around.Checks
Run in this worktree's own venv, all green:
ruff format --check .andruff check .over the whole treemypy messagefoundry(strict) — 275 source files, no issues_require_read_only,db_lookup,DatabaseLookup,handler-security,unsafe-db-lookuporCONNECTIONS.md(22 modules, includingtest_db_lookup,test_checks_handler_security,test_checks,test_database_transport,test_database_connector_integration,test_db_lookup_live_runner,test_semgrep_handler_rules,test_static_credential_db_hops,test_dryrun_trace,test_sandbox,test_wiring_serve) — 1189 passed, 91 skipped-k "connections_doc or docs or operator_docs") — 462 passed, 2 skippedSkipped: the full suite. It was started and was roughly 8 percent in after several minutes, so it could not finish inside the session and was stopped in favour of the targeted sweep above. Nothing in this change reaches the store, a transport listener or the pipeline.
Hosted legs a reviewer must read after this: the full
pytestmatrix on every platform, andwindows-service-smoke(NSSM), which never runs locally.Open questions
INSERT/UPDATE/DELETE/MERGE/INTO/EXEC) plus DDL, or is DDL scope creep? I includedDROP/CREATE/ALTER/TRUNCATE/GRANT/REVOKE/DENYbecause leavingSELECT 1 DROP TABLE tadmitted after fixing the no-semicolon chain for the other six looked like an obvious residual.test_real_samples_config_is_cleanpasses), but a site running--strict-handler-securitycould see a new finding on a shape that is in fact safe.