Replace pytest with plain.testing, a Plain-native test engine - #130
Draft
davegaeddert wants to merge 25 commits into
Draft
davegaeddert wants to merge 25 commits into
davegaeddert wants to merge 25 commits into
Conversation
… test runner Documentation-driven development for replacing pytest with a Plain-native engine. The README is the spec — written entirely from the user's perspective, covering the authoring API (bare asserts, declarative decorators, context-manager helpers), automatic database/email/cache lifecycle, built-in parallelism with template-cloned databases, flake classification, trace-aware performance assertions, route coverage, agent-oriented output (--json, --changed), browser testing, and the built-in test suite. The 'How it works' section proposes the core split: plain.test (core) grows into the authoring vocabulary users import, plain.testing (dev dependency) is the engine that runs tests, and other packages implement a TestLifecycle protocol registered via the plain.testing entry point group — packages never import the engine. No runnable code yet: __init__ and a no-op plain.setup entrypoint only. plain test is still provided by plain.pytest. Design context: futures repo, plain/testing-engine arc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
…onale Adds a 'Testing code outside the app' section: tests for supporting workspace packages run under the app runtime (lazy lifecycle costs nothing), and projects with no Plain app at all get library mode — kernel features work, app-dependent helpers error clearly. Adds a FAQ answering why the engine can't be named plain.test: the module is load-bearing in core at runtime (plain request is built on Client in plain/cli/request.py), so a dev-only package can't own it. Install plain.testing, import from plain.test, run plain test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
…omits Drops the vs-pytest negations (no fixtures, no conftest, no marks system, no db fixture, no retry-until-green) from the feature sections and rephrases each positively. The pytest contrast now lives only in the Migrating from pytest section and the FAQs, where readers arrive with that question. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
If a test file imports it, it lives in plain.test (core); if it runs test files, it lives in plain.testing. Growth is one-way into core — nothing there today moves to the engine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
plain.pytest will be removed when this ships; the migration section now frames upgrading as a one-time automated rewrite rather than a transition with both runners installed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
Request side speaks Plain's request vocabulary end to end (form_data=, json_data=, files=, query_params= — same names views receive), with keyword-only args and follow_redirects off by default. Responses are assertable data instead of assertion methods: status_code, headers, text/body, json_data, redirect_to, and response.request (post-middleware, so response.request.user replaces the old response.user attribute). Updates all examples to the new vocabulary. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
….postgres.test plain.test and the runner stay package-agnostic. Package-domain helpers ship with their packages: HTMX request helpers in plain.htmx.test (not grown on the client), query budgets in plain.postgres.test, job execution in plain.jobs.test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
…, full migration The engine (plain-testing package): - Collection: tests/**/test_*.py, test_* functions, Test* classes (fresh instance per method), async tests natively. Test modules import under synthetic names with the tests root and file dir on sys.path. - Assertion rewriting: an AST pass over test modules so bare assert failures show both sides of single-operator comparisons. Compiled with dont_inherit=True so the collector's __future__ flags don't leak into test modules. - Runner drives TestLifecycle implementations discovered from the plain.testing entry point group; reporter ends every failure with its exact re-run command. Flags: -k, -x, -v, --tag, --exclude-tag. - plain test re-execs into python -m plain.testing so PLAIN_ENV=test is set before settings load. Library mode (no app/) runs kernel-only. Core plain.test additions: - raises (with match= and .exception), cases/skip/tag decorators, override_settings, patch, capture_spans/capture_metrics, and the TestLifecycle base class. - Client/RequestFactory redesigned: form_data=/json_data=/files=/body=+ content_type=/query_params=, keyword-only, follow_redirects=; responses expose text, body, json_data, redirect_to, request. response.user and json() removed. - plain request CLI updated for the new client kwargs. Lifecycles: - plain.postgres: worker-scoped test database, per-test rolled-back transaction, @isolated_db tag for DDL-heavy tests, capture_queries and max_queries helpers. The db/isolated_db/setup_db fixtures and the cursor guard are gone. - plain.email: EMAIL_BACKEND routed to locmem for the run, outbox cleared per test, exposed as plain.email.test.outbox. CLI fixes surfaced by the work: - Registered commands (plain test) now dispatch without an app/ directory; CLIRegistryGroup skips app-module imports in that path. - _ensure_registry_loaded tolerates an already-set-up runtime. Migration: - plain-pytest deleted; pytest removed from every dependency group; scripts/test runs plain test. - All 28 suites (~150 files) migrated: fixtures to lifecycle/helpers, pytest.raises to raises, parametrize to @Cases, monkeypatch to patch, conftest.py files to explicit helper modules, client calls to the new vocabulary. Full run: 1867 passed, 1 skipped, 0 failed. - plain-oauth and plain-postgres dev deps gained cryptography (previously transitive via plain.pytest); plain dev deps gained plain.dev. - Docs and rules updated: core plain.test README rewritten, plain-test rule replaced, package lists and references swept. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
…nup sweep From a 4-angle review (reuse/simplification/efficiency/altitude) of the engine branch: - TestLifecycle.required_package replaces per-lifecycle _active guards; load_lifecycles() gates on INSTALLED_PACKAGES once, so inactive lifecycles never load and implementations carry no boilerplate. - Collection: CollectedTest drops dead/derivable fields (path, name, case_args — cases bind via functools.partial), the lineno-sort machinery goes (dict order is definition order), discovery uses os.walk with directory pruning instead of rglob-then-filter, per-file parent sys.path insertion removed (root only), and the Plain 'app' exclusion moves from the kernel to the runner. - main.py lets plain.runtime.setup()/AppPathNotFound decide app vs library mode instead of re-deriving the app path. - scripts/test invokes python -m plain.testing directly, skipping the throwaway plain-CLI parent process per package. - Client: ClientResponse loses the object.__getattribute__ dance, the redundant __setattr__, and the legacy url property (redirect handling uses redirect_to); RequestFactory.request() gets explicit parameters. - capture_metrics() yields CapturedMetrics with .points(name) — the flattening helper two suites had hand-rolled; CLI-internal capture_spans renamed capture_trace_spans to end the name collision with plain.test.capture_spans. - OTel SDK imports deferred so importing plain.test doesn't pay them; assertion rewriter locates only the injected import instead of a second whole-tree pass; email lifecycle clears the outbox once. - Tests: ~70 manual save/try/finally-restore blocks became patch()/ override_settings() blocks, _swap_router collapsed onto override_settings, make_admin_client deduped into helpers.py, and the stub tests/app dirs in plain-cloud/tunnel/dev are gone (library mode covers them). plain-oauth's provider-class strings now use f-string __name__ paths instead of relying on per-file sys.path. Deferred (recorded in the futures repo): the runtime setup() split that retires the CLI's special cases, per-test connection reuse, and per-test span-exporter clearing. Full suite: 1867 passed, 1 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
…essions From a high-effort correctness review (line-by-line, removed-behavior, cross-file, conventions): Collection: - Inherited test_* methods are now collected (MRO walk, subclass overrides win) — mixin/base-class test methods were silently dropped. - path::TestClass targets select the class's tests instead of matching nothing and exiting 5. - One unimportable file no longer aborts the run: per-file collection errors are reported alongside results and fail the exit code while every other file's tests still run. Isolation: - override_settings snapshots every original before applying anything, so an unknown name can't leave earlier overrides applied. - Postgres per-test teardown wraps check_constraints in try/finally — a deferred-constraint violation no longer skips the rollback and leaks an open transaction into the next test. - Runner tears down completed lifecycles when a later setup_worker raises (no leaked test databases), and one teardown failure no longer skips the rest. - capture_metrics actually isolates now: the reader uses delta temporality for synchronous instruments (the entry drain was a no-op for cumulative counters, leaking prior tests' points into every block), and CapturedMetrics accumulates drains so collect()+points() don't double-collect. New .clear() forgets the block so far. - install_test_tracer/meter raise loudly when another global provider won the one-shot install (silent empty captures + test traffic exporting to a real backend otherwise). Client / CLI: - plain request reads get_request_user(response.request) instead of the removed response.user (auth state showed anonymous for everyone), and the JSON body pretty-print uses response.json_data instead of the removed json() method. - Raw body= strings encode with the charset the content type declares. - scripts/type-validate validates plain-testing instead of the deleted plain-pytest. Plus: plain-testing now has its own test suite (28 kernel tests run in library mode, wired into scripts/test) locking in the collection, runner-lifecycle, assertion-rewriter, and vocabulary behavior. Full suite: 1894 passed, 2 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
- Raw bytes bodies pass through untouched — force_bytes with an encoding transcodes (or crashes on) bytes input, so only str bodies get charset-encoded. - override_settings applies inside the try, so a type-rejected value on a known setting still restores the overrides applied before it. Both found by adversarial verification of the previous fix commits. Full suite: 1894 passed, 2 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFQWBn2hag3Wq32iZ8q8M6
Brings 204 commits of master onto the branch that replaced pytest with plain.testing. The rule for resolution: the branch wins for anything about the test engine, plain.test, the test-runner CLI, and test-file conventions; master wins for everything else; where master changed a file the branch had also rewritten, master's intent is carried onto the branch's form. Non-trivial resolutions: - plain-redirection: master removed the package; removed here too. - plain-pytest: the branch removed the package; master's changes to it are dropped. Its `plain.pytest` references in docs, rules, and the packages list are now `plain.testing`. - `plain test` registration: the branch needed a CLIRegistryGroup that could load without an app. Master's new `plain.cli` entry point group does that properly (commands there run before `plain.runtime.setup()`), so the command moved to it and the branch's `import_app_modules` workaround is gone. `plain/plain/cli/core.py` keeps only the `SetupError` tolerance, so the CLI still works in-process when the runner already ran setup(). - `plain/plain/cli/_trace.py` and `request.py`: master's rewrite taken wholesale, with the branch's `capture_spans` -> `capture_trace_spans` rename (the name now belongs to `plain.test`) and the new Client API (`follow_redirects=`, `body=`, `response.json_data`, `get_request_user(response.request)` instead of `response.user`). - `plain/plain/test/client.py`: the branch's redesign kept, with master's bodiless-response handling, `_to_buffered_response`, and the delegating `__setattr__` that keeps `status_code` read-only. Wrapper-private state is written through `object.__setattr__`; `user` is no longer a test attribute. - `plain/plain/test/README.md`: master's "Capturing OpenTelemetry signals" section is superseded by the branch's `capture_spans` documentation. - Every `conftest.py` is gone; master's additions to them are ported into importable helper modules next to the tests.
Master added ~35 test files (and 5 conftest.py files) between July and now, all written for pytest. They merged in cleanly and had never been converted. - `import pytest` / `pytest.raises` / `pytest.mark.*` become `plain.test`'s `raises` / `@cases` / `@skip` / `@tag`. - Fixtures become explicit imports: `monkeypatch` -> `patch(...)`, the `settings` fixture -> `override_settings(...)`, `db` / `isolated_db` -> nothing / the `@isolated_db` decorator, `tmp_path` -> `TemporaryDirectory`, `capsys` -> `redirect_stdout`. - The five conftest.py files become importable helper modules at the tests root: `plain-dev/tests/helpers.py` (one `sandbox()` folding in both autouse conftests), `plain-mcp/tests/helpers.py` (the MCP request envelope), `plain-postgres/tests/migration_helpers.py`, `plain-templates/tests/clients.py`, and `plain/tests/log_helpers.py`. - Client calls move to the redesigned API: `data=` -> `form_data=` / `json_data=` / `files=` / `body=`, `follow=` -> `follow_redirects=`, `response.json()` -> `response.json_data`. There is no `caplog` equivalent, so tests that assert on log records attach a recording handler through a local `capture_logs` helper. Three of those exist now (plain, plain-mcp, plain-jobs) — see the report; it wants to be `plain.test.capture_logs`.
Post-merge fixes that aren't conflict resolutions: - `plain.email`'s README documented the `mailoutbox` pytest fixture; it now documents `plain.email.test.outbox` and the test lifecycle behind it. `plain.dev`'s README no longer describes a pytest plugin setting PLAIN_ENV, and `plain.cli`'s README points at plain.testing. - `plain-code`'s shipped ruff defaults drop the `PT` (flake8-pytest-style) ruleset. - `PYTEST_CURRENT_TEST` was how the CLI knew to turn colors off. The runner now sets `PLAIN_TEST_RUNNING` and `plain/plain/cli/formatting.py` reads it. - `plain/plain/cli/core.py` tolerates `SetupError` when loading the registry. The runner calls `plain.runtime.setup()` before collection, so every in-process CLI invocation from a test would otherwise fail to load the app. - The postgres test lifecycle no longer calls `conn.check_constraints()` — master made foreign keys NOT DEFERRABLE and removed the method, so there is nothing left to check before the rollback. `use_test_database` drops its hand-rolled maintenance-cursor DDL for master's new `plain.postgres.databases.create_database` / `drop_database`. - Lint fixes for ruff 0.16.7's expanded defaults: `Self` on `raises.__enter__`, a tuple `startswith` and an `S102` noqa in the collection kernel.
Five ad-hoc logging.Handler subclasses had accumulated across the suites
because there was no log-capture vocabulary. This is the one implementation.
with capture_logs() as logs:
Client().get("/broken/")
assert "Server error" in logs.messages
assert logs[0].path == "/broken/"
With no arguments it captures the `plain` and `app` trees — Plain's loggers
don't propagate to the root logger, so a handler attached there sees nothing.
Name loggers to narrow it. Levels and any global `logging.disable()` are
lowered for the block and restored on exit.
`logs.span_context_for(message)` returns the OpenTelemetry span context that
was current when that record was emitted, which is the assertion the OTel
guidance keeps asking for: an exception log has to land *inside* its error
span, or it exports with empty trace/span ids and the one failure gets
reported twice — the span's exception event plus an orphaned error log.
The span context is stored beside the record, not on it. A LogRecord's
__dict__ is how Plain carries structured context, so an extra attribute there
would surface as a key=value pair in every other handler's output, including
the OTel LoggingHandler's exported attributes.
Replaces the handlers in plain/tests (test_view_hooks, test_urls_instrumentation,
test_server_worker_shutdown, test_server_connection_errors — log_helpers.py is
gone), plain-mcp/tests/helpers.py, and plain-jobs/tests/internal/test_otel.py.
`raises` was annotated `exception: BaseException | None`, so every test that reached for an attribute of the exception it just caught — ValidationError's `.messages`, `.error_dict`, `.error_list` — was a type error. That was 18 of the 20 diagnostics `./scripts/type-check plain-postgres` reported. `raises[E: BaseException]` infers E from the types passed in, and `.exception` returns E. Reading it before the block exits now raises AttributeError with a message saying why, instead of returning a None that fails somewhere further down the test.
`PREFIX` was a fixed string, so every run of this suite handed out the same database names. Two runs sharing a cluster — a CI matrix, parallel worktrees, two agents — then destroy each other: one run's `make()` opens with `drop_database(force=True)`, which lands between the other run's `create_database` and the assertion that follows it. The symptom is a ~40% flake in `test_template_requires_idle_source`, failing with `InvalidCatalogName: template database ... does not exist`. The prefix now carries a token generated once per process. The names are still recognizable, so anything a crashed run leaves behind is still identifiable as this suite's. Verified by running two suites concurrently against separate databases: six concurrent runs, all green, where the fixed prefix flaked.
Parametrized tests reported by position only — `[0]`, `[1]` — so a failure
and its re-run command said nothing about which case broke. pytest had
readable ids and the conversion lost them.
@Cases(
case("a@example.com", True, id="plain address"),
case("nope", False, id="no at sign"),
)
def test_email_validation(email, valid): ...
reports `test_email_validation[no at sign]`, and `plain test
'file.py::test_email_validation[no at sign]'` runs just that one.
The id goes on the case rather than in a parallel `ids=[...]` list, so it
can't drift onto the wrong values when a case is added or reordered, and you
read the name next to the values it names. Duplicated ids are rejected —
two tests with the same id would make the re-run command ambiguous. Cases
without a `case()` wrapper keep their index, so nothing else changes.
Applied to the two tests that had pytest ids: the h1 shutdown drain modes
and the MCP classic header combinations, whose byte-payload and dict cases
were the least legible as bare numbers.
Also lists capture_logs in the testing rule alongside the other `with`-block
helpers, missed when it was added.
`form_data=` always produced multipart, so a view under test saw a content type it would rarely see in production — browsers and htmx send urlencoded unless there's a file input. Forms are now urlencoded, and multipart only when `files=` is given (form fields sent alongside files ride along in the same body, as before). `content_type=` with no body at all now says what to pass. It already rejected `content_type=` next to `form_data=`/`json_data=`, but `post(path, content_type="application/json")` — which is how this reads naturally — got a message about a raw body it hadn't mentioned. Building an empty body instead would hand the view a `b""` that its content type claims is parseable, so the failure would surface somewhere further in. Content headers now follow the content type rather than the byte count: a POST of an empty form declares what it is with Content-Length: 0, which is what a browser submitting an empty form sends. Requests with no body source at all (a GET) still get neither header. Without this, an empty form — previously non-empty multipart boundary bytes, now genuinely empty — would have silently lost its Content-Type. test_urls_trailing_slash's bodiless-POST test asserted multipart; its point is that a 308 preserves the initial request's content headers, which it still does.
`plain test` re-exec'd into `python -m plain.testing` for two reasons, and master's CLI has since removed both. The command is contributed through the `plain.cli` entry point group, whose commands are marked `without_runtime_setup`, so nothing has called `plain.runtime.setup()` by the time it runs — the runner still owns the app-vs-library decision. And `_PLAIN_ENV_DEFAULTS` in plain/cli/core.py already sets `PLAIN_ENV=test` before dispatch, so the dotenv ladder sees it either way. The third reason, cwd on sys.path for `from helpers import ...`, was never the re-exec's doing: collection inserts the root explicitly. `python -m plain.testing` still works and is still what `coverage run -m` uses; this only changes what the `plain test` command does. Verified: app mode, library mode, flag passthrough, and exit codes (1 on failure, 2 on a missing target, 0 on success).
Five packages had a `tests/helpers.py` and two had a `tests/clients.py`.
Each tests root is its own sys.path entry at runtime, so the collision was
invisible there — but the type checker sees them as one flat namespace, and
`from helpers import sandbox` resolved to whichever root `extra-paths` listed
first. That was 18 of the 46 diagnostics `./scripts/type-validate` reported.
Renamed after the `oauth_helpers.py` precedent, and every tests root that
holds helpers is now listed in `extra-paths`:
plain-admin/tests/helpers.py -> admin_test_helpers.py
plain-cloud/tests/helpers.py -> cloud_test_helpers.py
plain-dev/tests/helpers.py -> dev_test_helpers.py
plain-mcp/tests/helpers.py -> mcp_test_helpers.py
plain-postgres/tests/helpers.py -> postgres_test_helpers.py
plain-templates/tests/clients.py -> templates_test_clients.py
plain-templates/tests/error_routers.py -> templates_error_routers.py
The rest of the diagnostics, now that the right modules resolve:
- `get_request_user()` returns `User | None`; tests that read attributes off
it now assert it isn't None first, which also makes a lost session fail
where it happened. plain-admin's five sites share one `acting_user_id`
helper.
- `send_mail(html_message=...)` puts an `EmailMultiAlternatives` in the
outbox, which is typed `list[EmailMessage]` — narrowed with isinstance,
since that promotion is part of what the test checks.
- migration_helpers carried mypy's `# type: ignore[method-assign]`, which ty
doesn't read.
`./scripts/type-validate` now passes 26/26 with no errors.
What renaming can't reach: `plain.auth.requests.get_request_user` is
annotated with `app.users.models.User`, and `app` is a flat module name that
every test app and the example app claim. Checking the workspace resolves it
to the example app's model, so plain-oauth's reads of its own User model are
unresolvable at type time though correct at runtime. Those seven are
suppressed in place with the reason; fixing it properly means changing how
that annotation names the app's user model, which is a framework decision.
An `object()` sentinel made the saved values `str | object`, so restoring them to os.environ was a type error. An env var is never legitimately None, so None is the marker that says "wasn't set" and narrows the restore to str. Last of the diagnostics in a repo-root `ty check .` — it now runs clean.
Picks up "Clean up after plain migrations reset" (4eb466f) and three release commits. Three conflicts, all in tests this branch had converted to plain.testing — the branch's form kept, master's substance applied: - plain-dev/tests/internal/test_postgres_guard.py: master replaced `pending_migration_count` with `pending_migrations` returning a `PendingMigrations(run=, record=)`. Carried onto the branch's `patch(...)` form; `refuse()`'s return annotation follows. - plain-postgres/tests/internal/test_baselines.py: master's new assertion that a migration's own dependencies survive a resolved retired-name edge, reindented into the branch's `with migrations_dir()` block. - plain-postgres/tests/internal/test_migrations_reset.py: the whole file conflicted because the branch turned master's `migrations_dir` / `repo` fixtures into context managers, so no hunk lines up. Resolved by keeping the branch's file — same test set, verified name by name — and applying master's renames to it: `since` -> `shipped_in` (attribute, written source, and the `--shipped-in` CLI flag) and `detect_model_changes(..., {...})` -> `package_labels={...}`. Also adopted master's pending generalization of that file (228a19f on test-fixtures-per-checkout) while it was open: LEAF, NEXT_NUMBER, EXAMPLES_COUNT and LEAF_CREATED_MODELS now come from the migrations directory instead of literals, so adding an examples migration doesn't break every assertion — and so that branch's merge is a no-op here. `test_writer_operation_options.py`, new in the same master commit, needed no conversion: plain functions and bare asserts already. This branch's own test_databases.py per-run naming is unchanged.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| def test_config_when_logged_in(runner): | ||
| save(Credentials(api_url="https://example.com", token="tok")) | ||
| assert result.exit_code == 0 | ||
| assert "https://example.com" in result.output |
# Conflicts: # plain-dev/tests/conftest.py # plain-dev/tests/public/conftest.py # plain-dev/tests/public/test_dotenv.py # plain-dev/tests/public/test_dotenv_encryption.py # plain-postgres/tests/internal/test_autodetector_not_null_errors.py # plain-postgres/tests/internal/test_db_expression_defaults.py # plain-postgres/tests/internal/test_fk_characterization.py # plain-postgres/tests/internal/test_migrations_reset.py # plain-postgres/tests/public/test_databases.py # plain-postgres/tests/public/test_delete_behaviors.py # plain-postgres/tests/public/test_integrity_error_mapping.py # plain-postgres/tests/public/test_manager_assignment.py # plain-postgres/tests/public/test_related.py
#83 arrived with tests written for pytest: the EncryptedJSONField default cases in test_encrypted_fields.py use pytest.raises and pytest.mark.parametrize, and the typed-construction conformance file names pytest as the thing that doesn't check its TYPE_CHECKING half. Move them onto raises/@Cases and say 'the test runner' instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plain owns its test runner.
plain-pytestis deleted and every suite in the repo (28 suites, ~2,500 tests) runs onplain.testing.What stays familiar
tests/**/test_*.py, functions namedtest_*,Test*classes, async tests, bareassertwith rich diffs on single-op comparisons,-k/-x/-v/--tag, and re-run commands in the failure output.What's different
@cases,@skip,@tagattach static metadata. Runtime state enters throughwithblocks and plain function calls, so scope is visible as indentation. There is no fixture engine, no autouse, no conftest, no scopes, no plugin hook system. Shared setup is an importable helper module with a unique name.TestLifecycleentry point. Postgres (worker test database, per-test rollback,@isolated_db) and email are implemented. Package-specific helpers live inplain.<pkg>.test, never inplain.testor the client.plain.testhelpers:raises[E](typed.exception),capture_spans,capture_logs(records plus the span context current at emit time, kept beside the record rather than on it so other handlers' output is untouched),override_settings,patch,case(value, id="…")for named cases.Client: request vocabulary mirrors the view side (form_data=,json_data=,body=,follow_redirects=),response.json_data,get_request_user(response). Forms urlencode unless files are present, and content headers follow the content type, so an empty form POST looks like a browser's.plain testruns in-process through theplain.clientry point group, beforeplain.runtime.setup(), so it works in app and library checkouts alike.python -m plain.testingstill works forcoverage run -m.Not built yet
--lf,--pdb,@timeout, parallelism with template-database clones, seeded shuffle and flake classification,--jsonand--changed, browser testing, model factories, and the built-in framework suite. None of the ~40 test files added to master since July needed any of them.Known
plain.auth.requests.get_request_useris annotated asapp.users.models.User.appis a flat module name every test app and the example app claim, so a workspace-wide type check resolves it to whichever wins; seven suppressions carry that. How the annotation names the app's user model is a framework decision outside this PR.plain-code's shipped ruff defaults drop the flake8-pytest-style rules.Review shape
The kernel (
plain-testing/plain/testing/: collection, runner, reporter, assertion rewriter, lifecycle) andplain/plain/test/carefully; the converted test files sampled.plain-testing/plain/testing/README.mdand.claude/rules/plain-test.mddescribe the surface;.claude/rules/tests-layout.mdcarries the helper-module naming rule.Verification
./scripts/fix --checkclean,./scripts/type-validate26/26 with zero errors./scripts/test— 2487 passed, 2 skipped (both deliberate@skip), 28 suites