Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ad3f25a
Add typed where() with field-method conditions
davegaeddert May 23, 2026
1f02cf0
Block typed-query comparisons on encrypted fields
davegaeddert May 23, 2026
0fc51f2
Close kwarg-path foot-gun and tighten encrypted-field block
davegaeddert May 25, 2026
f76757c
Add typed FK traversal for where() clauses
davegaeddert May 25, 2026
5a78e38
Fix FK traversal review findings
davegaeddert May 25, 2026
1807490
Close assignment-typing gap on Field.__set__
davegaeddert May 25, 2026
bc619bb
Merge master into typed-where
davegaeddert May 25, 2026
9cf1090
Merge remote-tracking branch 'origin/master' into typed-where
davegaeddert Jul 23, 2026
f32c3ad
Fix FK descriptor shadowing in typed where() traversal
davegaeddert Jul 23, 2026
85717fd
Add is_in membership condition to typed where()
davegaeddert Jul 23, 2026
3da7f53
Document typed where() conditions
davegaeddert Jul 23, 2026
c9e8beb
Consolidate typed where() traversal onto delegation and metadata reso…
davegaeddert Jul 23, 2026
a289af1
Merge origin/master into typed-where
davegaeddert Sep 18, 2026
af0eb97
Fix typed where() against master: new lint rules, ty overrides, encry…
davegaeddert Sep 18, 2026
4d3cb34
Merge origin/master into typed-where
davegaeddert Sep 18, 2026
d6ec8de
Merge remote-tracking branch 'origin/master' into typed-where
davegaeddert Sep 18, 2026
69c757e
Anchor the reset test on the leaf migration's own models
davegaeddert Sep 18, 2026
e0c3c7f
Pin the encrypted-field block from the type checker's side
davegaeddert Sep 18, 2026
6e0bbd1
Converge the reset test on the master-bound version
davegaeddert Sep 18, 2026
fcc201a
Merge origin/master into typed-where
davegaeddert Sep 19, 2026
dd38bf1
Carry the typed query surface through #83's Field[T] annotations
davegaeddert Sep 19, 2026
fc66c73
Give every string-valued field the pattern conditions Field declares
davegaeddert Sep 19, 2026
f37b02c
Address Codex review on #84: FK conditions, README example, descripto…
davegaeddert Sep 19, 2026
cb5bc3c
Put the whole condition surface on Field
davegaeddert Sep 19, 2026
89f8a76
Class-level FK access returns the descriptor again
davegaeddert Sep 19, 2026
cb7db9a
Address code-review findings on the typed read surface
davegaeddert Sep 19, 2026
25f06e8
Address xhigh review: encrypted empty string, traversal edges, detach…
davegaeddert Sep 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion .claude/rules/plain-postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,21 @@ class Article(postgres.Model):
- **String forward-ref FKs** (`"self"`, `"OtherModel"`) keep a _value-type_
annotation — the checker can't resolve the string to a model:
`parent: Foo | None = types.ForeignKeyField("self", on_delete=postgres.CASCADE, allow_null=True, default=None)`.
That annotation is a model instance, not a field, so `where()` traversal
(`Child.parent.name.equals(...)`) doesn't type-check through it — declare the
target above and pass the class when you want traversal typed.
- **Encrypted fields** are annotated `EncryptedField[T]` (imported from
`plain.postgres` alongside `Field`), not `Field[T]`. It's a `Field[T]`
subclass, so the constructor is typed identically, but it also carries the
`Never`-typed blocks that reject `Model.secret.equals(...)` at the call site.
A plain `Field[T]` hides them and the comparison only fails at runtime.
- **JSON**: `JSONField`/`EncryptedJSONField` return `Any` from the stub (the
runtime class isn't generic over its value shape), so the annotation is what
preserves typing: `Field[dict]` / `Field[dict[str, Any]]`.
preserves typing: `Field[dict]` / `EncryptedField[dict[str, Any]]`. Never
annotate a field `Field[Any]` — `Any` satisfies the model-valued `__get__`
overload, so class access types as `type[Any]` and the whole condition
surface disappears silently. Use the concrete shape, or `Field[object]` when
the column really does hold arbitrary JSON.
- **Custom querysets**: declare `query: ClassVar[MyQuerySet] = MyQuerySet()`
(`ClassVar` so it isn't treated as a field). Default-queryset models declare
nothing — `Model.query` is typed automatically.
Expand Down Expand Up @@ -90,7 +102,11 @@ Run `uv run plain docs postgres` for full workflow details.

Use `Model.query` to build querysets (e.g., `User.query.filter(is_active=True)`).

- `where()` takes typed conditions built off fields (`User.query.where(User.role.equals("admin"))`) instead of `filter()`'s string kwargs, so a typo or wrong value type is caught at the call site.
- **Conditions on a relation go through its key**: `Post.query.where(Post.author.id.equals(author.id))`, `.id.is_in([...])`, `.id.is_null()` — the typed spelling of `filter(author=author)`, same SQL. `Post.author.equals(author)` raises `AttributeError`: `Post.author` is `type[Author]` to the checker (which is what makes `Post.author.email.equals(...)` work), so it offers the related model's fields, not conditions. Traversal only _starts_ from a forward FK; a many-to-many is traversable as a later hop (`WidgetTag.widget.tags.name`), but `Widget.tags` and reverse accessors are not entry points — use `filter(tags__name=...)` there.
- Encrypted fields can't be looked up at all: `get_or_create(secret=...)` raises — put the value in `defaults=`.
- Use `select_related()` for FK access in loops, `prefetch_related()` for reverse/M2N
- A foreign key with no value raises `RelatedObjectDoesNotExist`; catch it as `Related.DoesNotExist` (it subclasses that and `AttributeError`) — `Model.fk.RelatedObjectDoesNotExist` is a type error now that class access types as the related model.
- A foreign key returns a partial related object: `obj.author` and `obj.author.id` are query-free; other fields load on first access. There is no `obj.author_id` — use `obj.author.id`
- Use `.annotate(Count(...))` instead of calling `.count()` per row
- Fetch all data in the view — templates should never trigger queries
Expand Down
10 changes: 8 additions & 2 deletions plain-cache/plain/cache/models.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from __future__ import annotations

from datetime import datetime
from typing import Any, ClassVar, Self
from typing import ClassVar, Self

from plain.postgres import Field, types
from plain.runtime import settings
Expand Down Expand Up @@ -38,7 +38,13 @@ def forever(self) -> Self:
@postgres.register_model
class CachedItem(postgres.Model):
key: Field[str] = types.TextField(max_length=255)
value: Field[Any] = types.JSONField(required=False, allow_null=True, default=None)
# `object`, not `Any`: the cache stores any JSON-serializable value, and
# `Field[Any]` would make `Any` satisfy the model-valued `__get__` overload,
# so `CachedItem.value` would type as `type[Any]` and lose the field surface
# entirely.
value: Field[object] = types.JSONField(
required=False, allow_null=True, default=None
)
expires_at: Field[datetime | None] = types.DateTimeField(
required=False, allow_null=True, default=None
)
Expand Down
6 changes: 3 additions & 3 deletions plain-oauth/plain/oauth/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import psycopg
from app.users.models import User
from plain.exceptions import ValidationError
from plain.postgres import Field, transaction, types
from plain.postgres import EncryptedField, Field, transaction, types
from plain.utils import timezone

from plain import postgres
Expand Down Expand Up @@ -36,8 +36,8 @@ class OAuthConnection(postgres.Model):
provider_user_id: Field[str] = types.TextField(max_length=100)

# Token data
access_token: Field[str] = types.EncryptedTextField(max_length=2000)
refresh_token: Field[str] = types.EncryptedTextField(
access_token: EncryptedField[str] = types.EncryptedTextField(max_length=2000)
refresh_token: EncryptedField[str] = types.EncryptedTextField(
max_length=2000, required=False, default=""
)
access_token_expires_at: Field[datetime | None] = types.DateTimeField(
Expand Down
83 changes: 79 additions & 4 deletions plain-postgres/plain/postgres/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,63 @@ first_10_users = User.query.all()[:10]

For more advanced querying options, see the [`QuerySet`](./query.py#QuerySet) class.

### Typed conditions with where()

`where()` is a typed alternative to `filter()`. Instead of string keyword lookups, you build each condition from a field, so a type checker catches a misspelled field or a wrong value type at the call site:

```python
from plain import postgres
from plain.postgres import Field, types


@postgres.register_model
class User(postgres.Model):
email: Field[str] = types.EmailField()
role: Field[str] = types.TextField(max_length=20)
age: Field[int | None] = types.IntegerField(allow_null=True, default=None)


# Each argument is a condition; multiple arguments are ANDed together.
admins = User.query.where(
User.role.equals("admin"),
User.age.gte(18),
)
```

Every field exposes `equals`, `not_equal`, `gt`, `gte`, `lt`, `lte`, `is_null`, and `is_in`. Text fields add `contains`, `icontains`, `startswith`, and `endswith`. Each returns a `Q`, so you can combine them with `|` and `&` or negate with `~`:

```python
# Membership, negation, and OR
User.query.where(User.role.is_in(["admin", "staff"]))
User.query.where(~User.role.equals("guest"))
User.query.where(User.email.endswith("@example.com") | User.role.equals("admin"))
```

Conditions traverse foreign keys — accessing a field through a relation builds the joined lookup:

```python
# Q(author__email="a@example.com")
Post.query.where(Post.author.email.equals("a@example.com"))
```

A relation is a path to traverse, not a field, so it carries no conditions of its own. To match on the relation itself, traverse to the key it points at — that's the typed spelling of `filter(author=author)`, and it compiles to the same SQL:

```python
Post.query.where(Post.author.id.equals(author.id))
Post.query.where(Post.author.id.is_in([a.id for a in authors]))
Post.query.where(Post.author.id.is_null()) # nullable relation
```

`Post.author.equals(author)` raises `AttributeError` naming this spelling (an `AttributeError`, so `hasattr` and `getattr(..., default)` keep behaving). It isn't an oversight: to the type checker `Post.author` is `type[Author]`, which is what makes `Post.author.email.equals(...)` type-check, and a condition method there would be a runtime method the checker rejects.

Traversal starts from a **forward foreign key**. Once inside one, every relation you pass through is another hop, many-to-many included — `WidgetTag.widget.tags.name.equals("metal")` builds `Q(widget__tags__name="metal")` — and the same rule applies to the relation itself: `WidgetTag.widget.tags.equals(tag)` points you at `WidgetTag.widget.tags.id.equals(tag.id)`.

A class-level many-to-many (`Widget.tags`) is _not_ an entry point: it has no traversal wiring, and it is typed `ManyToManyManager[Tag]`, so it could not be typed as one either. Use the string path there — `Widget.query.filter(tags__name="metal")`. Reverse relations aren't traversable for the same reason (a reverse accessor is a `ClassVar`, so there is nothing for the related model to offer the checker), and the error says so.

A traversed field _is_ the related field, carrying the relation path as its name — so it offers exactly the conditions that field offers, including an encrypted field's refusals.

[Encrypted fields](#encrypted-fields) reject value comparisons because their ciphertext is non-deterministic — only `is_null()` is available, and any other condition method (`equals`, `is_in`, …) raises `TypeError`.

### Custom QuerySets

You can customize [`QuerySet`](./query.py#QuerySet) classes to provide specialized query methods. Define a custom QuerySet and assign it to your model's `query` attribute as a `ClassVar` (so it isn't treated as a constructor field):
Expand Down Expand Up @@ -991,28 +1048,44 @@ This is **not** for passwords or tokens you issue — those should be hashed (on

```python
from plain import postgres
from plain.postgres import Field, types
from plain.postgres import EncryptedField, Field, types


@postgres.register_model
class Integration(postgres.Model):
name: Field[str] = types.TextField(max_length=100)
api_key: Field[str] = types.EncryptedTextField(max_length=200)
credentials: Field[dict | None] = types.EncryptedJSONField(
api_key: EncryptedField[str] = types.EncryptedTextField(max_length=200)
credentials: EncryptedField[dict | None] = types.EncryptedJSONField(
required=False, allow_null=True, default=None
)
```

Annotate encrypted fields `EncryptedField[T]`, not `Field[T]`. The annotation is
what the type checker reads, and `EncryptedField[T]` is the `Field[T]` subclass
that declares the blocked conditions — with a plain `Field[T]`,
`Integration.api_key.equals("x")` type-checks its way to a runtime `TypeError`
instead of being rejected at the call site. It types the constructor exactly as
`Field[T]` does.

Values are encrypted using Fernet (AES-128-CBC + HMAC-SHA256) with a key derived from `SECRET_KEY`. The `cryptography` package is required — install it with `pip install cryptography`.

**Available fields:**

- `EncryptedField[T]` — the annotation type; also the shared base the two fields below derive from.
- `EncryptedTextField` — encrypts text, stored as `text` in the database regardless of `max_length` (ciphertext is longer than plaintext). `max_length` is enforced on the plaintext value during validation.
- `EncryptedJSONField` — serializes to JSON, encrypts, and stores as `text`. Supports custom `encoder` and `decoder` parameters (same as `JSONField`).

**Limitations:**

- **No lookups** — encrypted values are non-deterministic (same plaintext produces different ciphertext each time), so filtering on encrypted fields doesn't work. Only `isnull` lookups are supported.
- **No lookups** — encrypted values are non-deterministic (same plaintext produces different ciphertext each time), so filtering on encrypted fields doesn't work. Only `isnull` lookups are supported. Comparing against a value raises `TypeError` rather than silently matching nothing — both `filter(api_key="x")` and the typed [condition methods](#typed-conditions-with-where) (`equals`, `contains`, …), which are also rejected at the call site when the field is annotated `EncryptedField[T]`. `filter(api_key=None)` still rewrites to `IS NULL`.
- **`get_or_create()` must not look up an encrypted field.** `get_or_create(api_key="k")` raises, and the error says to move the value into `defaults=`. This is a deliberate break: it previously "worked" by creating a new row on every call, because the lookup could never match existing ciphertext. An encrypted value can be written, just not looked up:

```python
Integration.query.get_or_create(name="acme", defaults={"api_key": "k"})
```

The same applies to `update_or_create()`, and to an expression right-hand side like `filter(api_key=F("name"))` — the column is still ciphertext.

- **No indexes or constraints** — encrypted fields cannot be used in indexes or unique constraints. Preflight checks will catch this.
- **Only `default=""`** — on `EncryptedTextField` (paired with `required=False`), the empty string is stored as plaintext `''`, so it's the one value expressible as a column `DEFAULT` (declare it to add the field to a populated table). Any other default would need ciphertext, which is non-deterministic. `EncryptedJSONField` has no persistent default at all — even `{}` serializes to text that would need ciphertext — so pair `allow_null=True` with `default=None`, which stores nothing and just marks the field optional in the constructor.

Expand Down Expand Up @@ -1053,6 +1126,8 @@ book.author.name # one query — loads the rest of the row

The first access to any non-key field loads the whole row in a single query. There is no separate `author_id` attribute — `book.author.id` is the foreign key value, and it is type-checked because `book.author` is an `Author`. In loops, use `select_related()` to load related rows up front and avoid a query per row.

A foreign key with no value raises `RelatedObjectDoesNotExist` on access. That attribute still lives on the descriptor at runtime, but class-level access is now typed as the related model (that is what makes `Book.author.name.equals(...)` work), so `Book.author.RelatedObjectDoesNotExist` is a type error. Catch it as `Author.DoesNotExist` — the exception subclasses both that and `AttributeError` — or as `AttributeError`.

The partial-instance shortcut is safe because Plain always creates a database foreign-key constraint, so the referenced row is guaranteed to exist.

### Constraints are checked immediately
Expand Down
5 changes: 5 additions & 0 deletions plain-postgres/plain/postgres/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
URLField,
UUIDField,
)
from .fields.encrypted import EncryptedField
from .fields.json import JSONField
from .fields.timezones import TimeZoneField
from .fields.related import (
Expand Down Expand Up @@ -70,6 +71,10 @@
"DecimalField",
"DurationField",
"EmailField",
# The typed descriptor base for encrypted fields, for annotating them:
# api_key: EncryptedField[str] = types.EncryptedTextField()
# It blocks the value-comparison conditions a plain Field[T] would allow.
"EncryptedField",
"F",
# The typed descriptor base, for annotating model fields:
# name: Field[str] = types.TextField()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,21 @@ class Article(postgres.Model):
- **String forward-ref FKs** (`"self"`, `"OtherModel"`) keep a _value-type_
annotation — the checker can't resolve the string to a model:
`parent: Foo | None = types.ForeignKeyField("self", on_delete=postgres.CASCADE, allow_null=True, default=None)`.
That annotation is a model instance, not a field, so `where()` traversal
(`Child.parent.name.equals(...)`) doesn't type-check through it — declare the
target above and pass the class when you want traversal typed.
- **Encrypted fields** are annotated `EncryptedField[T]` (imported from
`plain.postgres` alongside `Field`), not `Field[T]`. It's a `Field[T]`
subclass, so the constructor is typed identically, but it also carries the
`Never`-typed blocks that reject `Model.secret.equals(...)` at the call site.
A plain `Field[T]` hides them and the comparison only fails at runtime.
- **JSON**: `JSONField`/`EncryptedJSONField` return `Any` from the stub (the
runtime class isn't generic over its value shape), so the annotation is what
preserves typing: `Field[dict]` / `Field[dict[str, Any]]`.
preserves typing: `Field[dict]` / `EncryptedField[dict[str, Any]]`. Never
annotate a field `Field[Any]` — `Any` satisfies the model-valued `__get__`
overload, so class access types as `type[Any]` and the whole condition
surface disappears silently. Use the concrete shape, or `Field[object]` when
the column really does hold arbitrary JSON.
- **Custom querysets**: declare `query: ClassVar[MyQuerySet] = MyQuerySet()`
(`ClassVar` so it isn't treated as a field). Default-queryset models declare
nothing — `Model.query` is typed automatically.
Expand Down Expand Up @@ -90,7 +102,11 @@ Run `uv run plain docs postgres` for full workflow details.

Use `Model.query` to build querysets (e.g., `User.query.filter(is_active=True)`).

- `where()` takes typed conditions built off fields (`User.query.where(User.role.equals("admin"))`) instead of `filter()`'s string kwargs, so a typo or wrong value type is caught at the call site.
- **Conditions on a relation go through its key**: `Post.query.where(Post.author.id.equals(author.id))`, `.id.is_in([...])`, `.id.is_null()` — the typed spelling of `filter(author=author)`, same SQL. `Post.author.equals(author)` raises `AttributeError`: `Post.author` is `type[Author]` to the checker (which is what makes `Post.author.email.equals(...)` work), so it offers the related model's fields, not conditions. Traversal only _starts_ from a forward FK; a many-to-many is traversable as a later hop (`WidgetTag.widget.tags.name`), but `Widget.tags` and reverse accessors are not entry points — use `filter(tags__name=...)` there.
- Encrypted fields can't be looked up at all: `get_or_create(secret=...)` raises — put the value in `defaults=`.
- Use `select_related()` for FK access in loops, `prefetch_related()` for reverse/M2N
- A foreign key with no value raises `RelatedObjectDoesNotExist`; catch it as `Related.DoesNotExist` (it subclasses that and `AttributeError`) — `Model.fk.RelatedObjectDoesNotExist` is a type error now that class access types as the related model.
- A foreign key returns a partial related object: `obj.author` and `obj.author.id` are query-free; other fields load on first access. There is no `obj.author_id` — use `obj.author.id`
- Use `.annotate(Count(...))` instead of calling `.count()` per row
- Fetch all data in the view — templates should never trigger queries
Expand Down
Loading
Loading