Add a ty typing corpus and stub-vs-runtime conformance for plain-postgres - #132
Merged
Merged
Conversation
Static promises -- "this call is a type error" -- can only be tested by writing the bad call and asserting the checker rejects it. tests/typing/ does that: each must-reject claim carries a `# ty: ignore[<code>]` marker, and `unused-ignore-comment` is promoted to an error so the day the call stops erroring the marker goes unused and the build fails. Must-accept claims are `assert_type` calls. Nothing in tests/typing/ is named test_*.py, so pytest never collects it; ty reading the files is the test, and ./scripts/type-check plain-postgres already walks the directory.
The static-marker tests in tests/public/ -- bodies that were nothing but `assert_type` calls or a TYPE_CHECKING block of `ty: ignore` markers -- now live in tests/typing/, where the checker is the assertion. Each one that also carried a runtime assertion keeps it: the pattern-conditions-are-registered-on- every-Field fact moved to tests/internal/test_typed_where_internals.py, and the encrypted blocks were already covered by their parametrized raises tests. Private surface followed the same rule. tests/public/ no longer imports underscore-prefixed names or internal modules; the tests that needed them are change detectors and moved to tests/internal/: - test_encrypted_internals.py (_encrypt/_decrypt/_get_fernet) - test_meta_related_objects.py (ForeignKeyRel, _model_meta.related_objects) - test_m2m_value_from_object.py (_model_meta.get_forward_field) - test_random_string_sql.py (compile_database_default_sql) The rest is spelling: Q, QuerySet and F come from plain.postgres, and field constructors from plain.postgres.types.
ty believes the stub, so the typing corpus cannot catch a stub that has drifted from the runtime -- it is checked by the same checker reading the same stub. This parses types.pyi (and fields/base.py, fields/encrypted.py) with `ast` and compares the declarations against the live objects: - every keyword argument the stub declares across a constructor's overloads must exist on that constructor's runtime signature, PEP 681's pseudo- parameters (init=, default=, ...) excepted; extras are reported by name; - the `self`-restricted conditions on Field must be exactly STRING_CONDITION_METHODS, and every field the stub types as string-valued must register those lookups at runtime -- unless it blocks them statically too, which is how EncryptedTextField is exempt; - every method EncryptedField declares under TYPE_CHECKING must name a real condition and have a runtime attribute behind it; - ModelBase's field_specifiers == ModelMixin's == the stub's constructors == what plain.postgres.types exports. Both failure modes have happened: overloads promising `default=` combinations DefaultableField.__init__ rejects, and GenericIPAddressField type-checking .contains(...) while raising at runtime. The specifier-list assertions move here from test_typed_construction_preflight.py, which keeps the preflight checks.
No new script: ./scripts/type-validate already runs `ty check` over every fully typed path, plain-postgres included, and CI's lint job already runs type-validate -- so the corpus is inside a gate that exists. This documents that, adds tests/typing/ to the tests-layout rule so the next static claim lands in the right place, and notes in pyproject that a ty bump has to pass the corpus rather than land as a lockfile update.
Contributor
|
Next steps:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Most of what typed construction (#83) and the typed
where()API (#84) promise is static: "this call is a type error." Nothing in the pytest suite could test that, so the promises were being pinned ad hoc —ty: ignoremarkers scattered throughtests/public/, alongside the runtime contract. This makes the mechanism first-class, and adds the thing the mechanism structurally cannot catch.The mechanism: unused suppression as the assertion
plain-postgres/tests/typing/is checker input, not pytest input. Nothing in it runs;uv run ty checkreading the files is the test, and no file is namedtest_*.py, so pytest never collects them.Must-reject claims write the offending call and mark it with the exact diagnostic it has to produce:
unused-ignore-commentis promoted from warning to error in the rootpyproject.toml, so the marker is the assertion: the day that call stops being a type error, the suppression goes unused and the build fails. A wrong code fails too — the real diagnostic goes unsuppressed and the marker goes unused.Must-accept claims are
assert_type(...), plus any call that simply has to keep type-checking clean.12 files, one per claim area: construction (required fields, value types, unknown kwargs, mixins), conditions (value types, the encrypted block), field access, relations (forward FK, reverse), queryset access, field constructors. Seeded from the plan plus everything that already existed as a static marker in
tests/public/.What the corpus cannot catch, and the test that does
ty believes
types.pyi. A stub that promises a keyword argument the runtime constructor doesn't take, or declares a method underTYPE_CHECKINGwith nothing behind it, type-checks perfectly and then raises — and the corpus can't see it either, because the corpus is checked by the same checker reading the same stub.tests/internal/test_stub_runtime_conformance.pyparses the stub withastand compares it against the live objects:init=,default=, … — excepted); extras are reported by name;self-restricted conditions onFieldmust be exactlySTRING_CONDITION_METHODS, and every field the stub types as string-valued must register those lookups at runtime — unless it blocks them statically too, which is howEncryptedTextFieldis exempt;EncryptedFielddeclares underTYPE_CHECKINGmust name a real condition and have a runtime attribute behind it;ModelBase'sfield_specifiers==ModelMixin's == the stub's constructors == whatplain.postgres.typesexports.Both failure modes have actually happened on these branches: the
only_empty_defaultoverloads promiseddefault=combinationsDefaultableField.__init__rejects, andGenericIPAddressFieldtype-checked.contains(...)while raising at runtime.Proof both halves bite
Widening one stub overload with a bogus
db_collationkwarg —./scripts/type-check plain-postgresstill says All checks passed!, and the conformance test fails:Loosening one
Neverparameter (EncryptedField.startswith) — the corpus fails, exit 1:tests/public/is the runtime contract againThe static-marker tests moved out; each one that also carried a runtime assertion kept it. Private-surface imports went with them, into new
tests/internal/files (test_encrypted_internals.py,test_meta_related_objects.py,test_m2m_value_from_object.py,test_random_string_sql.py). An incidental# ty: ignoreon a deliberately-wrong call inside apytest.raisesblock stays put — theraisesis the assertion there.How it runs
No new script.
./scripts/type-validatealready runsty checkover every fully typed path,plain-postgresincluded, and CI'slintjob already runstype-validate— so the corpus is inside a gate that exists.plain-postgres/testsis on ty'sextra-paths, so corpus files import the example app's models the same way the runtime tests do.Adding a claim
test_prefix.def../scripts/type-check plain-postgreswith no marker first, read the code ty reports, paste that code into the marker. Never guess it.assert_type(...), or leave the line unmarked.tests/public/ortests/internal/per the tests-layout rule, cross-referenced.One gotcha, documented in the file it bites: the blocked encrypted conditions are declared
-> Never, so a second call in the same function body sits in unreachable code and is never analyzed — the marker would read as satisfied while proving nothing. One call per function there.tests/typing/README.mdand thetests-layoutrule carry all of this.pyproject.tomlnotes that a ty bump has to pass the corpus rather than land as a silent lockfile update.Gates
./scripts/fix --check,./scripts/type-check .,./scripts/type-validate(26/26, plain-postgres 100% coverage), full./scripts/test— 2655 passed, 5 skipped.