Skip to content

Add a ty typing corpus and stub-vs-runtime conformance for plain-postgres - #132

Merged
davegaeddert merged 4 commits into
masterfrom
typing-corpus
Sep 19, 2026
Merged

davegaeddert merged 4 commits into
masterfrom
typing-corpus

Conversation

@davegaeddert

Copy link
Copy Markdown
Member

Most of what typed construction (#83) and the typed where() API (#84) promise is static: "this call is a type error." Nothing in the pytest suite could test that, so the promises were being pinned ad hoc — ty: ignore markers scattered through tests/public/, alongside the runtime contract. This makes the mechanism first-class, and adds the thing the mechanism structurally cannot catch.

The mechanism: unused suppression as the assertion

plain-postgres/tests/typing/ is checker input, not pytest input. Nothing in it runs; uv run ty check reading the files is the test, and no file is named test_*.py, so pytest never collects them.

Must-reject claims write the offending call and mark it with the exact diagnostic it has to produce:

DefaultsExample(name=123)  # ty: ignore[invalid-argument-type]

unused-ignore-comment is promoted from warning to error in the root pyproject.toml, so the marker is the assertion: the day that call stops being a type error, the suppression goes unused and the build fails. A wrong code fails too — the real diagnostic goes unsuppressed and the marker goes unused.

Must-accept claims are assert_type(...), plus any call that simply has to keep type-checking clean.

12 files, one per claim area: construction (required fields, value types, unknown kwargs, mixins), conditions (value types, the encrypted block), field access, relations (forward FK, reverse), queryset access, field constructors. Seeded from the plan plus everything that already existed as a static marker in tests/public/.

What the corpus cannot catch, and the test that does

ty believes types.pyi. A stub that promises a keyword argument the runtime constructor doesn't take, or declares a method under TYPE_CHECKING with nothing behind it, type-checks perfectly and then raises — and the corpus can't see it either, because the corpus is checked by the same checker reading the same stub.

tests/internal/test_stub_runtime_conformance.py parses the stub with ast and compares it against the live objects:

  • every keyword argument the stub declares across a constructor's overloads must exist on that constructor's runtime signature (PEP 681's pseudo-parameters — init=, default=, … — excepted); extras are reported by name;
  • the self-restricted conditions on Field must be exactly STRING_CONDITION_METHODS, and every field the stub types as string-valued must register those lookups at runtime — unless it blocks them statically too, which is how EncryptedTextField is exempt;
  • every method EncryptedField declares under TYPE_CHECKING must name a real condition and have a runtime attribute behind it;
  • ModelBase's field_specifiers == ModelMixin's == the stub's constructors == what plain.postgres.types exports.

Both failure modes have actually happened on these branches: the only_empty_default overloads promised default= combinations DefaultableField.__init__ rejects, and GenericIPAddressField type-checked .contains(...) while raising at runtime.

Proof both halves bite

Widening one stub overload with a bogus db_collation kwarg — ./scripts/type-check plain-postgres still says All checks passed!, and the conformance test fails:

AssertionError: types.pyi declares ['db_collation'] on TextField(), which
plain.postgres.fields.text.TextField.__init__ does not accept. A call using one
type-checks and then raises TypeError.
    runtime parameters: ['allow_null', 'choices', 'default', 'max_length', 'required', 'validators']

Loosening one Never parameter (EncryptedField.startswith) — the corpus fails, exit 1:

error[unused-ignore-comment]: Unused `ty: ignore` directive
  --> plain-postgres/tests/typing/conditions_encrypted.py:63:42
   |
63 |     SecretStore.api_key.startswith("x")  # ty: ignore[invalid-argument-type]
   |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

tests/public/ is the runtime contract again

The static-marker tests moved out; each one that also carried a runtime assertion kept it. Private-surface imports went with them, into new tests/internal/ files (test_encrypted_internals.py, test_meta_related_objects.py, test_m2m_value_from_object.py, test_random_string_sql.py). An incidental # ty: ignore on a deliberately-wrong call inside a pytest.raises block stays put — the raises is the assertion there.

How it runs

No new script. ./scripts/type-validate already runs ty check over every fully typed path, plain-postgres included, and CI's lint job already runs type-validate — so the corpus is inside a gate that exists. plain-postgres/tests is on ty's extra-paths, so corpus files import the example app's models the same way the runtime tests do.

Adding a claim

  1. Find the file for the area, or add one — plain module name, no test_ prefix.
  2. Write the smallest call that expresses the claim, inside a def.
  3. Must-reject: run ./scripts/type-check plain-postgres with no marker first, read the code ty reports, paste that code into the marker. Never guess it.
  4. Must-accept: assert_type(...), or leave the line unmarked.
  5. A runtime half goes in tests/public/ or tests/internal/ per the tests-layout rule, cross-referenced.

One gotcha, documented in the file it bites: the blocked encrypted conditions are declared -> Never, so a second call in the same function body sits in unreachable code and is never analyzed — the marker would read as satisfied while proving nothing. One call per function there.

tests/typing/README.md and the tests-layout rule carry all of this. pyproject.toml notes that a ty bump has to pass the corpus rather than land as a silent lockfile update.

Gates

./scripts/fix --check, ./scripts/type-check ., ./scripts/type-validate (26/26, plain-postgres 100% coverage), full ./scripts/test — 2655 passed, 5 skipped.

Static promises -- "this call is a type error" -- can only be tested by
writing the bad call and asserting the checker rejects it. tests/typing/
does that: each must-reject claim carries a `# ty: ignore[<code>]` marker,
and `unused-ignore-comment` is promoted to an error so the day the call
stops erroring the marker goes unused and the build fails. Must-accept
claims are `assert_type` calls.

Nothing in tests/typing/ is named test_*.py, so pytest never collects it;
ty reading the files is the test, and ./scripts/type-check plain-postgres
already walks the directory.
The static-marker tests in tests/public/ -- bodies that were nothing but
`assert_type` calls or a TYPE_CHECKING block of `ty: ignore` markers -- now
live in tests/typing/, where the checker is the assertion. Each one that also
carried a runtime assertion keeps it: the pattern-conditions-are-registered-on-
every-Field fact moved to tests/internal/test_typed_where_internals.py, and the
encrypted blocks were already covered by their parametrized raises tests.

Private surface followed the same rule. tests/public/ no longer imports
underscore-prefixed names or internal modules; the tests that needed them are
change detectors and moved to tests/internal/:

- test_encrypted_internals.py (_encrypt/_decrypt/_get_fernet)
- test_meta_related_objects.py (ForeignKeyRel, _model_meta.related_objects)
- test_m2m_value_from_object.py (_model_meta.get_forward_field)
- test_random_string_sql.py (compile_database_default_sql)

The rest is spelling: Q, QuerySet and F come from plain.postgres, and field
constructors from plain.postgres.types.
ty believes the stub, so the typing corpus cannot catch a stub that has
drifted from the runtime -- it is checked by the same checker reading the
same stub. This parses types.pyi (and fields/base.py, fields/encrypted.py)
with `ast` and compares the declarations against the live objects:

- every keyword argument the stub declares across a constructor's overloads
  must exist on that constructor's runtime signature, PEP 681's pseudo-
  parameters (init=, default=, ...) excepted; extras are reported by name;
- the `self`-restricted conditions on Field must be exactly
  STRING_CONDITION_METHODS, and every field the stub types as string-valued
  must register those lookups at runtime -- unless it blocks them statically
  too, which is how EncryptedTextField is exempt;
- every method EncryptedField declares under TYPE_CHECKING must name a real
  condition and have a runtime attribute behind it;
- ModelBase's field_specifiers == ModelMixin's == the stub's constructors ==
  what plain.postgres.types exports.

Both failure modes have happened: overloads promising `default=` combinations
DefaultableField.__init__ rejects, and GenericIPAddressField type-checking
.contains(...) while raising at runtime.

The specifier-list assertions move here from
test_typed_construction_preflight.py, which keeps the preflight checks.
No new script: ./scripts/type-validate already runs `ty check` over every
fully typed path, plain-postgres included, and CI's lint job already runs
type-validate -- so the corpus is inside a gate that exists. This documents
that, adds tests/typing/ to the tests-layout rule so the next static claim
lands in the right place, and notes in pyproject that a ty bump has to pass
the corpus rather than land as a lockfile update.
@pullapprove5

pullapprove5 Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor
PASS: 1 review scope passed
Scope Progress
code 1/1

View in PullApprove

Next steps:

@davegaeddert
davegaeddert enabled auto-merge (squash) September 19, 2026 18:09
@davegaeddert
davegaeddert merged commit 9cbaf7c into master Sep 19, 2026
9 checks passed
@davegaeddert
davegaeddert deleted the typing-corpus branch September 19, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant