Skip to content

Add plain.html, a Plain-native HTML template engine, and remove Jinja - #129

Draft
davegaeddert wants to merge 122 commits into
plain-formsfrom
plain-html
Draft

davegaeddert wants to merge 122 commits into
plain-formsfrom
plain-html

Conversation

@davegaeddert

Copy link
Copy Markdown
Member

Stacked on #128 (plain-forms); retarget to master when that merges.

plain.html is an HTML-aware template engine that replaces Jinja across the repo. plain-templates and plain-elements are deleted. Every in-repo template is ported, plain.pages renders through it, and the checker runs in CI.

The engine

  • Templates are HTML with {{ expr }} expressions and {% if %} / {% for %} / {% slot %} / {% fragment %} blocks. Expressions are real Python, not a filter language: .lower(), str(), or "", comprehensions. No {% set %}, no inheritance, no macros, no registry, no autoescape opt-out — components and Python cover those.
  • Every template declares its names in YAML frontmatter (attrs:, components:, slots:). Nothing is ambient.
  • Components are templates invoked as tags (<Pagination page_obj="{{ page_obj }}" />) with typed attrs: and named slots. Call sites are checked against the component's declaration.
  • Compiled to Python ahead of time with a disk cache; tracebacks map back to template source; contextual autoescape by construction.
  • {% fragment "name" %}…{% endfragment %} renders inline normally, and render(..., fragment="name") returns only that region. HTMX fragment requests are rebuilt on it, including status_code= forwarding. Fragments inside components are reachable.
  • Tooling: plain html check [--typecheck], plain html format, plain html compile. --typecheck extracts every expression into a synthesized module and runs ty over it against the declared names, so a misspelled attribute or a Jinja-ism like is false fails before render. --template-dir runs both commands standalone with no app or database; a plain-html console script exposes that mode. ./scripts/html-check and ./scripts/html-format walk every package and every test app and run in the Postgres-less CI lint job. plain html format --check is part of ./scripts/fix --check.

Porting notes worth knowing

  • Jinja's silent fallbacks (dict attribute access, StrictUndefined sentinels, is none, filters) all became caught errors under --typecheck; several had shipped as latent bugs.
  • Logic that Jinja templates carried with {% set %} moved into Python. The admin list's three-state sort header is now get_column_headers() on the view, which also fixed a bug where an unsorted list rendered no sort links.
  • lazy=True on HTMX fragments is not carried over: component slot content renders eagerly, so it would defer nothing. The README points at dedicated templates instead.
  • plain-html/ROADMAP.md lists deferred follow-ups: global attribute pass-through, always-on component validation, :values enums, repeatable slots, the static/dynamic compiled representation as the optimization path for fragments, and lint tiers. {% let %} is recorded as the candidate if a second real case appears that Python can't absorb.

Review shape

The engine (plain-html/plain/html/: parser, compiler, typecheck, loader, format) carefully; plain-html/README.md for the authoring surface; the ported templates sampled, with admin/list.html and the oauthserver template as the two largest.

Verification

  • ./scripts/fix --check clean (ruff, ty, oxlint, oxfmt, prettier, html format --check)
  • ./scripts/html-check — 120 templates across every package and test app, 0 errors
  • ./scripts/type-validate 25/25
  • ./scripts/test — 3421 passed, 1 skipped, across 25 packages plus the example app

claude and others added 30 commits May 12, 2026 14:05
Captures the design spec for the new HTML-aware template engine and
a phased plan to migrate the repo from Jinja. The plan adds a tracer-bullet
phase before infrastructure work, moves parity verification into an
automated harness from Phase 7 onward, pins upfront decisions (cache
location, Markup reuse, plain.templates shim, presenters convention,
CLI naming), and defers Tier 2/3 HTML lint rules to a post-migration
phase.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
Adds a minimum-viable plain.html renderer (tokenizer → tag-tree parser →
tree-walking interpreter + contextual HTML escape + frontmatter parser)
and a parity harness that renders paired Jinja/.plain fixtures side by
side and writes diffs to plain-html/tests/parity/results/.

Phase 0 scope:
- Plain text, {expr} interpolation, attribute interpolation, mixed-segment
  attribute values, boolean-attribute coercion, :if/:for directives,
  tuple-unpacked iteration, <template> fragments, HTML/template comments,
  void elements, {{ }} brace escape.
- 19 unit tests in plain-html/tests/internal/test_engine.py.

Parity results:
- greeting (3 scenarios): byte-identical output to Jinja for text,
  expressions, attribute interpolation, fragment conditionals, escape.
- tasks_list (2 scenarios): one normalized-match (Jinja {% if %} blocks
  leave whitespace that element-level :if doesn't), one real spec-
  intentional diff (HTML-aware boolean-attribute rendering vs Jinja's
  Python-stringification). Both differences catalogued in
  parity_allowlist.yml with rationale.

Out of scope for Phase 0: <template :include>, slots, :as scoped slots,
URL-attr scheme validation, script/style refusal, AOT compile-to-Python,
loader-based template resolution, plain html check. Captured as
follow-ups in the implementation plan.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
- Drop plain-html/plain/html/escape.py; use plain.utils.html.conditional_escape
  for HTML-safe escaping. Trades byte-for-byte Jinja parity on quote-style
  entities for consistency with the rest of Plain — recorded as the
  `escape-entity-style` allowlist entry.
- Switch frontmatter.py to python-frontmatter (already used by plain.pages)
  instead of hand-rolling YAML split. Preserve trailing newline since the
  library strips it.
- Promote attribute representation from tuple[str, list | None, bool] to
  named Attribute + AttrText/AttrExpr dataclasses across tokenizer / parser /
  engine. Drops the redundant `is_expr` derived flag and stringly-typed
  segment kind discriminators.
- Collapse parallel ElementNode.for_target / for_iter Optionals into a single
  ForClause that pre-parses target names at parse-time (so the per-iteration
  binding doesn't re-split the target string).
- Drop ElementNode.is_template_fragment; the renderer checks tag == "template"
  directly.
- Rewrite isinstance chains in parse() and _render_node() as match
  statements.
- Move tests from tests/internal/ to tests/public/ — assertions are
  user-visible contract per the project's public-vs-internal convention.
- Read template files with encoding="utf-8" rather than locale default.

Parity results after refactor: 2/5 byte-identical, 1 cosmetic entity-style
diff, 1 whitespace-only diff, 1 spec-intentional boolean-attribute diff.
Each remaining diff is documented in parity_allowlist.yml.

19 unit tests still pass.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
…e renders

Engine extensions:
- <template :include="path"> with attrs and slot composition
- slot="name" routes content; <template slot="name"> is a transparent wrapper
- Default slot exposed as `children` per spec
- Filesystem loader walks Plain's get_template_dirs(); relative ./ and ../
  resolve against the calling template
- Globals registry seeded from Plain's Jinja default_globals (url, asset,
  reverse, etc.) plus mark_safe/Markup
- View-level context (request, DEBUG, ...) flows down into included templates
  via root_ctx threading; explicit attrs always override

Integration:
- plain.templates.Template routes .plain files through plain.html.render;
  .html still goes to Jinja so both engines coexist during migration
- _shims module bridges tailwind_css/pageviews_js/toolbar helpers to their
  existing Jinja-rendered package templates until those are ported

Example app:
- base.html ported to base.plain (extends/block → :include/slots)
- index.html ported to index.plain; row macro became components/row.plain
- IndexView/ErrorView template_name switched to .plain

Verified: `uv run plain request /` returns 200 with the index page rendered
end-to-end through plain.html (with the toolbar bridge falling back to
Jinja for plain.toolbar's templates).

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
Every template in example/app/templates/ now has a .plain counterpart and
the corresponding view.template_name is switched. End-to-end smoke test
(uv run plain request <path> --user 1) is 200 across every route:

  / /notes/ /notes/new/ /notes/1/ /notes/1/edit/ /notes/1/delete/
  /tasks/ /tasks/1/ /tasks/1/edit/ /tasks/1/delete/ /tasks/new/
  /contacts/ /contacts/archive/ /login/ /sse/ /admin/ /observer/

Engine additions to support this:
- <script>/<style> bodies tokenize as opaque text per spec (no {expr})
- class={list} flattens, drops falsy, joins with spaces
- Declared attrs default to None when caller doesn't pass (so :if={x}
  works without raising NameError)
- View-level context (request, DEBUG, etc.) flows down into included
  templates via root_ctx threading; explicit attrs override
- Added Markup/mark_safe to engine globals so templates can `:include`
  a value pre-marked as Markup

plain.forms additions:
- Form.__getattr__ falls back to self[name] for declared fields, so
  `form.email` works the same as `form["email"]` (matches Jinja
  attribute-then-subscript semantics)

Components introduced (replacing _macros.html):
- components/back_link, field, field_errors, nonfield_errors
- components/row (was an inline {% macro %} in index.html)
- components/htmxfragment (lightweight wrapper; partial-fragment
  rendering still goes through plain-htmx's Jinja extension)

The toolbar/tailwind_css/pageviews_js helpers continue to bridge to the
Jinja-rendered package templates via plain.html._shims. Porting those
package templates to .plain is the next phase.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
Package templates ported to .plain:
- plain.tailwind/templates/tailwind/css.plain
- plain.pageviews/templates/pageviews/js.plain
- plain.htmx/templates/htmx/js.plain
- plain.toolbar/templates/toolbar/toolbar.plain (the main dev-toolbar)
- plain.toolbar/templates/toolbar/request.plain (Request panel)
- plain.toolbar/templates/toolbar/exception_button.plain (Exception button)

Shims switched: tailwind_css, pageviews_js, toolbar all render through
plain.html now. The Exception panel and other toolbar panels (sessions,
email, observer) still resolve to their .html counterparts via Jinja —
plain.templates.Template dispatches per-file, so they coexist cleanly.

Engine: applies frontmatter `attrs:` defaults at the top-level
render_source call too, not just at include sites. This is what the
toolbar panels need — render_panel() builds its context manually and
the template's declared attrs need None defaults when the caller
omits them.

All example routes still return 200 with the toolbar rendering through
plain.html.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
- plain.sessions/templates/toolbar/session.plain (uses `get_request_session`
  imported via frontmatter; original used Jinja `pass_context` global that
  doesn't translate cleanly)
- plain.email/templates/toolbar/email.plain (accesses email['from'] via
  subscript since `from` is a Python keyword)

Both panel-template-names updated to .plain. All example routes still
return 200 with these panels rendering through plain.html.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
The observer toolbar panel was inlining Jinja-rendered values into a
`<script>` body, which doesn't work under plain.html's spec (script
bodies are opaque). Restructured per the spec's "use data-* attributes
on a host element and read them from JS" pattern:

- data-observer-mode and data-traces-url set on the container div
- JS reads them via container.dataset

Functionally equivalent, but follows plain.html's CSP-safe stance and
works through the new engine. Example routes still 200.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
…ates

Engine:
- :include now accepts either a literal string (`:include="path"`) or
  a {expression} (`:include={path_var}`); the spec's open question
  about dynamic dispatch is resolved as 'yes' for v1
- ElementNode carries both include_path (literal) and include_path_code
  (expression); renderer evaluates whichever is set

Package templates ported:
- plain.loginlink: sent.plain, failed.plain
- plain.oauth: error.plain
- plain.pages: page.plain (uses mark_safe(page.content))
- plain.support: iframe.plain, page.plain, forms/default.plain,
  success/default.plain (the layouts use the new dynamic include
  to dispatch to success_template_name or form_template_name)

19 unit tests pass; example routes all 200.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
….html

plain.templates.Template now tries `.plain` before falling back to the
named `.html`, so the dozens of admin value templates that admin's
field-template-resolution machinery generates (`admin/values/<type>.html`)
get routed to plain.html the moment a `.plain` version lands. Callers
don't have to change.

Admin value templates ported:
- admin/values/UUID, setting_name, bool, default, Avatar, Img
- admin/values/dict, list (use stdlib json instead of jinja's tojson filter)
- admin/values/EncryptedTextField, EncryptedJSONField
- admin/values/setting_source, setting_value
- admin/values/Model, QuerySet
- admin/values/datetime (uses plain.utils.timezone.localtime + strftime)

Also:
- plain.jobs/jobs/values/job_status.plain
- plain.passwords/admin/values/PasswordField.plain

Engine still through both example/ and /admin/ end-to-end (200s).

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
- plain.passwords/email/password_reset.plain
- plain.loginlink/email/loginlink.plain
- plain.support/email/support_form_entry.plain (keeps inline `style=`
  since email clients require it — out-of-band of the CSP-safe stance
  that applies to browser-rendered pages)
- plain.observer/observer/values/span_kind.plain, span_status.plain
- plain.admin/elements/admin/Help.plain

Tried elements/admin/Icon.plain too, but admin's Jinja callers pass
`class="..."` and `class` is a Python keyword, so plain.html can't
reference it as a name in expressions. Leaving Icon as Jinja for now
(the dot-syntax admin.Icon callers go through plain.elements anyway).

All example routes still 200.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
…ress

- globals._load_defaults now reads from `environment.globals` (where
  `register_template_global` writes) in addition to the static
  `default_globals` dict. This means helpers like `is_package_installed`
  and `get_current_session` registered by packages are available in
  plain.html scopes too.
- plain-html-implementation-plan.md updated: Phase 0 status reflects
  the actual surface implemented (Phase 0-7 engine + integration + most
  of Phases 10-14's package porting, modulo admin's extends-chain
  templates which have to migrate together).

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
… .plain exists

plain.elements:
- Element() now calls plain.templates.Template instead of going through
  Jinja's environment.get_template. This means the .html → .plain
  fallback applies to elements too, so <admin.Submit> can find
  elements/admin/Submit.plain transparently when called from a Jinja
  template.

Admin elements ported to .plain (mechanical translation of
<admin.X> → <template :include="elements/admin/X">):
- elements/admin/Label.plain
- elements/admin/Submit.plain
- elements/admin/FieldErrors.plain
- elements/admin/Input.plain, Select.plain, Textarea.plain, Checkbox.plain
- elements/admin/InputField.plain, SelectField.plain, TextareaField.plain,
  CheckboxField.plain (wrappers that compose Label + Input + Help +
  FieldErrors)

When a Jinja admin template writes <admin.InputField field={f} label="X">
the call now flows through plain.html via the new fallback. Admin model
detail/list/edit pages continue to return 200.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
Card templates ported to .plain (admin's cards/base.py loads card.html,
key_value.html, chart.html, table.html via Template(name) — all picked
up by the .html → .plain fallback now):

- admin/cards/base.plain (the wrapper aside; takes slug/title/filters
  and a default slot for content)
- admin/cards/card.plain (metric + text + optional link)
- admin/cards/key_value.plain (dl of label/value pairs)
- admin/cards/chart.plain (Chart.js canvas with json_script payload)
- admin/cards/table.plain (admin-table with headers/rows/footers)

Each .plain replaces the Jinja `{% extends "admin/cards/base.html" %}`
with `<template :include="admin/cards/base">` so the chain stays on
plain.html when the entry-point dispatches there.

Engine shim:
- plain.html.globals exposes `htmx_js(request, extensions=[])` that
  renders htmx/js.html. admin/base.html uses `{% htmx_js %}` and the
  plain.html port (when it lands) will need this helper.

All example routes + /admin/ still return 200.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
…e alias for Python-keyword attrs

Engine:
- `_build_scope` now exposes any Python-keyword scope name under
  `name_` (e.g. `class` → `class_`). Lets templates that receive
  `class="..."` from a caller access the value as `{class_}`.

Admin elements:
- elements/admin/Icon.plain (uses class_ alias)
- elements/admin/SearchInput.plain (uses class_ alias for input class
  and wrapper_class for the surrounding div)

Admin layout skeleton:
- admin/base.plain — port of admin/base.html. Inert until a child
  page (admin/list/detail/etc.) is ported and explicitly uses
  `<template :include="admin/base">`. References admin/_header.plain
  which is still TODO.

All routes still 200; admin's chrome/sidebar continues to render
through Jinja until the full layout chain ports.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
…index

Skeleton layout chain ported to .plain. These templates remain inert
until a child page (admin/list.plain, admin/detail.plain, etc.) is
ported and explicitly invokes <template :include="admin/base">.

- admin/_header.plain — top bar + tabs + Menu popover (uses
  elements/admin/Icon, elements/admin/SearchInput, header_branding,
  _menu_section)
- admin/_menu_section.plain — packages-nav menu items
- admin/header_branding.plain — app name + Admin link
- admin/page.plain — empty wrapper for AdminView default template
- admin/index.plain — empty wrapper for AdminIndexView

Engine already exposes `class_` alias for `class="..."` attrs so the
Icon/SearchInput ports can read the value without colliding with the
Python keyword.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
- admin/detail.plain renders the object's fields by composing
  admin/base.plain + a per-field render via `render_value_template()`
  (new global in plain.admin.templates that mirrors Jinja's
  `{% include [list_of_names] %}` semantics — tries each candidate
  template in order, returns the first match)
- Fixed admin/_header.plain to use `preflight_counts["errors"]`
  (it's a dict, not an object — Jinja's attribute-fallback-to-subscript
  hid this)

Verified: `/admin/p/flag/1/` renders end-to-end through plain.html
(detail.plain → base.plain → _header.plain → menu/branding/Icon/
SearchInput .plain). Output includes the `<details class="admin-card">`
and `<h3>Flag Details</h3>` from detail.plain.

Other admin URLs (/admin/, /admin/p/<model>/, /admin/settings/,
/admin/preflight/) continue to render through Jinja since their
templates (list.html, search.html, setting_detail.html, preflight.html)
haven't been ported yet.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
…outes through plain.html

- admin/list.plain — 200+ line port. Renders htmxfragment wrapper,
  per-column sortable headers (uses enumerate for loop.index), per-row
  actions dropdown, pagination footer, search/filter form. Heavy use
  of render_value_template() for per-field value rendering.
- admin/delete.plain — danger-card confirmation
- admin/setting_detail.plain — uses subscript access for setting dict

Subscript-access fixes for dict-typed admin context values that Jinja
silently fell back from `.attr` to `[key]`:
- _header.plain: tab["view"], tab["pinned"]
- _header.plain: preflight_counts["errors"|"warnings"]
- values/setting_source.plain: object["source"]
- values/setting_value.plain: object["is_secret"]

Verified: every admin route in the smoke set returns 200 — / /notes/
/tasks/ /admin/ /admin/p/{model}/{,1/} /admin/settings/{,SECRET_KEY/}
/admin/preflight/ — and the list/detail/setting-detail pages render
end-to-end through plain.html (detail.plain → base.plain → _header.plain
→ menu_section.plain / Icon.plain / SearchInput.plain).

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
- admin/search.plain — search input + per-view htmx fragment loaders
- admin/preflight.plain — passed/warning/error summary + per-check
  status list; inlines the `tiered_split(text)` macro logic as a
  mark_safe expression since plain.html doesn't have macros
- admin/toolbar/button.plain — dev-toolbar button (Admin link with
  impersonate badge + preflight warning indicator)

Subscript fixes for dict-typed context values:
- preflight check dicts: check["status"], check["name"], check["issues"]
- preflight issue dicts: issue["warning"], issue["id"], issue["fix"]
- preflight_counts dict: ["errors"], ["warnings"]

All 16 routes in the smoke set still return 200 (example pages + all
exercised admin pages + sse). Admin's list/detail/setting/search/
preflight chains all render end-to-end through plain.html now.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
- observer/trace_detail.plain — standalone HTML page wrapper for a
  single-trace view (tailwind_css + htmx_js + the asset script tags),
  then includes observer/trace via `<template :include="observer/trace">`
- observer/partials/log.plain — single log event row with level
  color-coding and localtime timestamp

observer/trace itself still goes through Jinja for now; this just
ports the outermost layout. /observer/ continues to render via the
existing Jinja chain.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
Per-package summary of what's on plain.html now:
- example/app: 100% (every template ported)
- plain.admin: ~47/48 (everything except admin/ui.html)
- plain.toolbar: outer + request + session/email/observer panels
- plain.tailwind/pageviews/htmx/sessions/email: fully covered
- plain.loginlink/oauth/pages/support/passwords: primary templates
- plain.observer: trace_detail/partials/log/toolbar + values (main
  UI still Jinja)

16 routes (full example + main admin + observer) all return 200,
mostly rendering through plain.html end-to-end with Jinja fallback
for the un-ported tail.

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
All 101 template files renamed via `git mv` from `*.plain` to
`*.plain.html`. The double extension keeps the engine-routing signal
(`.plain` discriminator) while letting most editors pick up the
trailing `.html` for syntax highlighting.

Code updates:
- plain.html.loader: looks for `<name>.plain.html`
- plain.templates.core: `.plain.html` extension routes to plain.html;
  the bare `.html` fallback still tries a same-named `.plain.html`
  first before deferring to Jinja
- ToolbarItem subclasses (toolbar, sessions, email, observer): panel/
  button template names updated
- example/app views: template_names updated to `.plain.html`
- plain-html/tests/parity/run_parity.py: fixture lookup uses
  `<stem>.plain.html`

Smoke: all 12 routes (example + admin + observer) still 200; 19 unit
tests pass; parity harness unchanged (1 byte-identical, 2 normalized-
match, 1 entity-style cosmetic diff, 1 bool-attr spec-intentional diff).

https://claude.ai/code/session_01M3ACFdWig8pX8jKc2hewid
Brings in the 0.143–0.145 url-routing arc (segment resolver, trailing-slash
convention, catchall semantics), plain.templates and plain.assets package
extractions, and assorted release commits.

Mechanical merge — git rename detection replanted the branch's .html →
.plain.html fallback patch onto the new plain-templates/plain/templates/core.py
location. plain-html and plain-templates cohabit as workspace members during
the migration; rip-out happens at the end of the branch's work per the
option-3 plan.

Note: leaves a handful of ty errors from the new interaction surface
(BoundField __getattr__ vs FormView, htmx fragment renderer, plus
pre-existing plain-html engine errors). Fixed in follow-up commits.
ty errors:
- plain-templates/views.py: BaseForm.__getattr__ now resolves to BoundField,
  so the pre-existing form.save() ignore comments need [call-non-callable]
  instead of [unresolved-attribute].
- plain-toolbar: widen Toolbar/ToolbarItem context param from jinja2 Context
  to Mapping[str, Any] so plain.html's plain-dict context also satisfies it.
  Cast the Jinja-side call in templates.py since Context isn't a declared
  Mapping subclass even though it walks like one.
- plain-htmx/views.py: render_template_fragment is Jinja-specific; raise a
  clear NotImplementedError when an HTMX view's template is a .plain.html,
  since the {% htmxfragment %} mechanism hasn't been ported yet.
- plain-html/engine.py + tokenizer.py: type-narrow _eval results and the
  attribute-segment list to silence pre-existing ty errors; annotate
  _flatten return type.

Test regressions from the isolated-test harness:
- plain-templates/core.py: make the plain.html import in _find_plain
  optional via try/except. plain-templates can ship without plain.html, so
  the in-process test harness (which installs only the package under test)
  was tripping a hard ModuleNotFoundError on every .html template lookup.
- plain-elements/templates.py: tier the Element() resolution — try
  plain.html first if installed and a .plain.html element exists, otherwise
  fall back to ctx.environment.get_template. Restores compatibility with
  tests that use a custom DictLoader Jinja environment, while keeping the
  production benefit of picking up <admin.X> .plain.html ports during
  migration.
Reverses two recent decisions and pins the end state:

- Extension is .html (not .plain.html) — editors highlight automatically.
- Directory is html/ (not templates/).
- No per-package templates.py / html.py registration files —
  templates declare what they use via frontmatter imports:.

Lists the 8-step sequence: loader standalone → per-package migration
recipe → port the 14 Jinja-only holdouts → move TemplateView family →
switch direct callers → build the plain.html fragment story → delete
plain-templates → /plain-upgrade rewrite rules.
The loader no longer imports `get_template_dirs` from
plain.templates.jinja.environments. plain.html.loader now owns:

- `get_html_dirs()` — walks the app's `html/` dir + every installed
  package's `html/` dir.
- `find_template()` — resolves `<name>` to `<name>.html` inside an
  `html/` directory.

Transitional fallback: if `<name>.html` is not found under `html/`,
probe `<name>.plain.html` under the old `templates/` directories so
in-flight templates that haven't moved yet keep rendering. The
fallback comes out when every package has migrated.

This unblocks the per-package `templates/` → `html/` moves and is the
first step of the rip-out plan documented in
plain-html-implementation-plan.md.
`templates/tailwind/css.plain.html` → `html/tailwind/css.html`. The
plain.html loader finds the new location via `get_html_dirs()`; example
app and admin routes still render the tailwind <link> tag.

The old `templates/tailwind/css.html` (Jinja source) and `templates.py`
(InclusionTagExtension) stay until the last `{% tailwind_css %}`
caller is ported — currently plain-admin/admin/_base.html and the
observer holdouts (traces, trace).
Per the option-3 rip-out plan, every package's plain.html templates
move from `<pkg>/templates/<...>.plain.html` to
`<pkg>/html/<...>.html`. Python string references (`template_name`,
`panel_template_name`, etc.) lose the `.plain` infix.

Loader fix: the bulk sed accidentally collapsed two distinct
extension checks in plain.html/loader.py — restore the
`.plain.html`-aware legacy fallback so the transitional lookup of
unported `templates/*.plain.html` files still works. (No such files
exist post-move, but the fallback remains correct in case anyone
runs the loader before completing the per-package move.)

plain-templates/core.py: simplify `Template.__init__` — the
`.plain.html`-specific branch was a sed-introduced duplicate; merge
into the single `.html` path that checks plain.html first then
falls back to Jinja.

Parity harness: restore the `.plain.html` fixture path on the plain
side of the parity comparison (the fixtures are paired by stem and
their extensions are intentional).

Smoke verified: /, /notes/, /tasks/, /contacts/, /admin/,
/admin/preflight/, /admin/search/, /observer/, /login/ all return 200.
- plain-flags/admin/plainflags/flagresult_form.html
- plain-jobs/admin/plainqueue/jobresult_detail.html
- plain-redirection/admin/plainredirection/redirect_form.html
- plain-pageviews/pageviews/card.html
- plain-support/support/card.html

The two model forms (flags, redirection) follow the existing admin/list
and admin/delete pattern: :include "admin/base", call elements via
:include "elements/admin/InputField" / "CheckboxField" / "Submit".

jobresult_detail.html only overrides the actions block, so admin/detail
gains a slot { actions: optional, default: optional } that pass through
to admin/base.

The two cards (pageviews, support) :include "admin/cards/base" with
slug/title/etc. passed through, iterate via :for and bring in
get_admin_model_detail_url + timesince via the new template `imports:`
mechanism instead of relying on Jinja-registered globals/filters.

plain-html/engine.py: declared slots now default to empty SafeString at
root render too (paralleling the existing :include behavior). Without
this, admin/detail rendered as the root template raises NameError when
the caller doesn't fill the actions slot — surfaced by porting
jobresult_detail.

Smoke: /, /admin/, /admin/p/user/, /admin/p/user/1/, /admin/preflight/,
/admin/search/, /observer/, /login/, /notes/, /tasks/, /contacts/ all 200.
163 → ~163 lines, renders only on errors. Translation notes from the agent
that did the work:
- `{% for frame in ... %}` → `:for={frame in ...}` on the wrapper
- `{% if frame.source_lines %}` → `:if={frame.source_lines}` inline
- `frame.locals` dicts use `var["name"]` for item access
- `frame.is_error_line|lower` → `"true" if line["is_error_line"] else "false"`
  (Jinja's `|lower` on a bool was a Jinja-ism)
- vscode:// link concatenates the URL in Python rather than mixing
  literal text with `{expr}` in one attribute value
davegaeddert and others added 30 commits May 15, 2026 14:54
Delete the three root-level design docs (superseded by the shipped
engine) and the stale plain-templates Jinja rule. Capture the deferred
lint/formatter roadmap in plain-html/ROADMAP.md. Fix admin and package
READMEs that still showed Jinja syntax or dotted element tags.
check_source now swallows TokenizeError from synthesis — the structural
check pass already reports tokenize errors, so typecheck must not crash.
Repoint the corpus test repo-checkout marker to CLAUDE.md (the old
marker file was a deleted design doc).
Review fixes:
- render_source(source_path=...) bypasses the process cache on both
  read and write, so an in-memory source override never returns a
  stale compile.
- Frontmatter `attrs:` defaults are emitted into the compiled
  render() signature. Declared defaults were honored by --typecheck
  but discarded at runtime, so components rendered None.

Markdown text mode:
- New render_text_source() and text-mode compile path: recognizes
  only {{ }}, {% raw %}, and {# #}; all other text (placeholder
  <tags>, autolinks, code fences) stays literal, and expressions are
  emitted unescaped.
- plain.pages renders Markdown bodies through it. Markdown is not
  balanced HTML, so routing it through the HTML-aware engine crashed
  ~35% of real pages; text mode renders them cleanly.

Formatter:
- A single-{{ }} attribute value now formats quoted
  (href="{{ href }}") so HTML highlighters see attr="...". The
  tokenizer already accepted both forms identically. All repo
  templates reformatted to the quoted form.

plain-email:
- TemplateEmail renders email/{template}.txt through text mode when
  the file exists, otherwise strips tags from the HTML body. The
  subject is passed as the subject= kwarg.

Docs:
- Fixed stale Jinja references (Jinja is no longer shipped) and the
  admin template-customization section, which documented a
  {% extends %}/{% block %} mechanism plain.html does not have.
Brings the rebuilt forms API onto the branch so we can see how it lands
alongside plain.html before forms-rebuild merges to master.

Notable cross-cutting changes the merge forced:

- Plain.html components consume FormDisplay/FieldDisplay fields directly.
  Same render surface (.value/.errors/.required/.choices/.html_id/.name) as
  the old BoundField, so the `<Field bf="{{ form.x }}" />` pattern keeps
  working — the underlying object just changed.
- Removed FormView/CreateView/UpdateView/DeleteView from plain.html.views;
  loginlink/passwords/support/admin and the example app now use explicit
  get/post + FormDisplay.
- `.subject.txt` removal applied at the moved locations: explicit subject=
  kwargs in links.py:send_login_link and core.py:send_password_reset.
- plain-templates dependency replaced with plain.html everywhere
  (INSTALLED_PACKAGES, package deps, imports of TemplateView).
- plain-connect's `{% connect_pageviews %}` Jinja extension rewritten as a
  Markup-returning Python helper invoked from a plain.html template.
- plain-support's dynamic panel dispatch (`Template(name).render(...)`)
  restructured so `form` is in scope at panel render time.
- plain-admin's detail.html `{% if actions %}{% slot %}{% endif %}` was
  invalid plain.html (slot must be direct child of component) — now the
  conditional lives inside the slot.
- aria-invalid emits lowercase "true" in the admin element components.
- All forms-rebuild test apps that shipped Jinja-style {% extends %} /
  {% block %} templates (plain-passwords × 6, plain-loginlink × 2,
  plain-admin × 1) converted to plain.html component syntax.
- plain-pageviews removal re-applied (forms-rebuild predated it).
- plain/debug.py off markupsafe (transitive dep that vanished with
  plain.templates), on plain.utils.html.escape + SafeString.

All package test suites and the example app pass.
Picks up three new forms-rebuild commits on top of the earlier merge:
guarded API surface, preserved validator codes in get_password_errors,
and JSON-API rendering example for Invalid. Auto-merge only — no manual
conflict resolution needed. All tests pass.
…helpers

Picks up the two follow-ups requested in the design feedback:

- FormDisplay[F]/FieldDisplay[T] generic typing — types now survive the
  template boundary.
- TemplateView.render_form() / validate_form() — absorbs the validate-then-
  re-render-on-failure boilerplate that was three lines in every form view.

The helpers landed in plain.html.views.TemplateView on this branch (forms-
rebuild put them in plain.templates.views, which doesn't exist here). The
auto-merge picked the right home; only the caller import blocks needed
hand-resolving (FormDisplay no longer imported in 5 of 6 view files).
…pers

Bigger swing than the prompt — forms-rebuild took "ship typed helpers
alongside FormDisplay" as "the helpers make the wrapper classes
unnecessary, kill them." Three free helpers (field_value, field_errors,
form_errors) dispatch typing through the Field[T] reference, so a
ContactForm.email reference rides into the template as Field[str]. Field
metadata (.required, .choices, .name, .html_id) lives on the field
descriptor itself.

Templates now receive both form_class (for metadata access) and form (a
Form | Invalid for value/errors via the helpers). The blank/initial case
is handled by constructing form_class(**defaults) — a real Form instance,
not a special wrapper.

This branch's work:
- Ported the new render_form/validate_form shape on plain.html.views.
- Reworked plain-admin element components (Checkbox/Input/Select/Textarea
  + FieldErrors) to take both form and field and call the helpers.
  Wrapper *Field components (InputField/CheckboxField/SelectField/
  TextareaField) thread form through.
- Updated example app's Field/FieldErrors/NonfieldErrors components and
  every form-bearing template to the new shape.
- Updated plain-loginlink/plain-passwords test templates similarly.
- Updated plain-flags and plain-redirection admin form templates that
  plain.html --typecheck caught as missing the new form parameter.
- Dropped the auto-merged plain.templates.templates global-registration
  module — plain.html templates import helpers per-template via
  frontmatter rather than registering globals.
- plain-support kept its dynamic panel dispatch (going away anyway,
  preserved to compile).

All package test suites and the example app pass.
Addresses the gap I flagged on the last merge — the helpers were
typed-by-default but nothing pinned that statically. New test
(test_form_helpers_typing.py) uses assert_type to lock the contract: a
regression that collapses field_value's return to Any would fail under
ty rather than passing silently.

Clean auto-merge — single new test file, no conflicts.
Bring master's changes (via forms-rebuild) into the plain.html branch,
reconciling them with plain-html's template-engine rewrite:

- Deleted plain-observer and plain-support to follow master (observer's
  trace/optimize functionality moved into plain core); plain-elements and
  plain-templates stay deleted (plain-html subsumed them into plain.html)
- plain-connect: ported master's evolved connect (secret-based identity,
  current_trace(), route tracking, support-fields/connect_support_url) onto
  plain-html's plain.html Markup-helper style, since master's Jinja
  InclusionTagExtension approach no longer exists; dropped the dead
  pageviews.html/support_fields.html templates
- plain-toolbar: ported master's redesigned toolbar (drag/dock pill,
  collapse states) from Jinja to plain.html syntax to match master's
  already-merged toolbar.js; admin list footer now keys off the new
  data-toolbar-fullbar body signal
- example app + rules + scripts/test: kept plain-html's plain.html/component
  style, dropped deleted-package references, regenerated uv.lock
Bring the latest master (via forms-rebuild) into plain-html. master's new
work is postgres-focused (FK scaffolding collapse, db_constraint removal,
preflight split) and merged cleanly. Conflicts were only deleted-package
leftovers (plain-elements, plain-templates, and the Jinja-based
plain-htmx/test_templates.py — all removed by plain-html) plus uv.lock.

Bumped plain-html's dev `ty` pin from ==0.0.34 to >=0.0.45 to match master's
workspace bump; `plain html check --typecheck` verified clean against ty 0.0.45.
Brings ~330 commits of master in through the already-merged plain-forms
branch. 49 conflicts:

Kept deleted (the branch's engine replaces them) — plain-templates/*,
plain-elements/*, the per-package `templates.py` Jinja registrations in
plain-auth / plain-htmx / plain-sessions / plain-tailwind / plain-toolbar,
`.claude/rules/plain-templates.md`, and plain-htmx's fragment tests.
Master's only changes to those files were import formatting, a RUF012
noqa and an `__all__` sort, so nothing was lost; the one real change
(plain-elements masking Jinja comments before scanning for capitalized
tags) has no analogue — plain.html parses comments as nodes.

Master's deletions honored — plain-redirection and plain-esbuild are
gone, including the branch's ported `redirect_form.html`.

Branch wins on engine surface — plain-connect's `connect_pageviews` /
`connect_support_fields` stay Markup-returning functions rather than
Jinja `InclusionTagExtension`s; `plain.html.Template` in the toolbar;
`render_source`/`render_text_source` in plain.pages; `plain.utils`
escape/SafeString instead of markupsafe in `plain/debug.py`;
`plain.html.views` imports everywhere. HTMX fragment rendering keeps the
branch's `NotImplementedError` — master's `render_template_fragment`
walks a Jinja AST and has no plain.html equivalent.

Master wins elsewhere — `Response(..., status_code=)` construction,
`status_for_exception`, ListView pagination (`page_size` / `page_obj`,
which git carried into `plain-html/plain/html/views.py` via the
plain-templates rename), admin `AdminListView` on `ListView`, the
loginlink `send_login_link` / `LoginLinkFormView.post()` docs, and the
plain-oauth dependency bumps (requests dropped, postgres 0.113.0).

Templates master touched since June were ported rather than reverted:
admin list actions menu + 3-state sort + `popover="manual"`, the menu
`group_label` restructure and empty state, the ui.html popover/hovercard
docs, and example pagination (master's `_macros.html` `pagination` macro
became `components/Pagination.html`). Master's new `plain-templates`
ListView tests moved to `plain-html/tests/` with a local conftest.
The package landed on master written for Jinja. Swap the dependency and
INSTALLED_PACKAGES entry to plain.html, import Template from plain.html,
and give authorize.html frontmatter attrs. Two Jinja-isms in the template
needed real Python: `params.response_type` (attribute access on a dict)
becomes `params["response_type"]`, and the strict-mode `error: None`
sentinel becomes `""` so the declared attr type stays `str`.
Master's baseline and reset tests build a temp migrations root for the
`plaintemplates` package label. plain.templates is gone; plain.html has
no config.py, so its label is `html`. Renaming the fixture label fixes
16 failures (nonexistent-parent-node / no-migrations-to-reset).
Master widened the lint set while this branch was away, and plain.html
was the only package it had never been run over. Mechanical: startswith
tuples, enumerate, list.extend, a flattened nested if, a negated-return
collapse, late-binding closures in the bench, and `# noqa: S102` on the
four `exec()` calls that run the engine's own generated module — that
call is the whole point of a compiler.
The port left a Jinja `is false` test and no frontmatter, so
`plain html check --typecheck` reported `trace_url`, `sampled` and
`false` as undefined names. Declare the attrs the toolbar item passes
and use Python's `is False`.
Also tightens the empty-list assertion in test_list_view to match the
newline the formatter introduces after `items:`, so it still proves the
loop emitted nothing.
Master's forms README still imported TemplateView from
plain.templates.views and reached for Jinja macros; its field example
put `{% if %}` inside a start tag, which plain.html rejects — that's an
expression-valued attribute now. Package listings drop plain.elements
and the core `templates` module and gain plain.html; agent-guidance and
the rules' Django-differences cross-reference point at plain-html, which
now carries its own `## Differences from Django` section. CLAUDE.md
regains the `./scripts/html-check` row, and type-validate stops looking
for the deleted plain-elements.
`plain-code check .` type-checks the whole repo, and plain-html's own dev
deps aren't in the root environment — html5lib and jinja2 (what the bench
measures against) get hoisted the same way plain-api's
openapi-spec-validator already is. jinja2 previously resolved by accident,
via the plain-templates wheel that plain-oauthserver was still pulling
from PyPI. The newer ty no longer needs three `too-many-positional-arguments`
suppressions, and master's `test_edit_task` needed `task.project` narrowed
before reading `.id`.
`{% fragment expr %}…{% endfragment %}` names a region of a template.
A normal render emits it inline; `render(..., fragment="name")` renders
the whole template as usual and returns only that region.

Render-everything-keep-one is the point, not a shortcut: a fragment
inside a `{% for %}` sees its own iteration, a fragment inside an
`{% if %}` only exists when the branch ran, and a fragment inside a
component works because `_frag_capture` threads down include call sites
the way `_root_ctx` already does. Nesting falls out for free — the outer
capture contains the inner one's output, and the inner one is still
addressable on its own.

The name is an expression matched as `str(...)`, since it arrives over
HTTP; first occurrence wins when one name is produced twice. An unknown
name raises `FragmentNotFound` naming the template, what the render
actually produced, and whether a declared fragment simply wasn't reached.

`HTMXView.render()` uses it for `Plain-HX-Fragment` requests again,
forwarding `status_code=`. `HtmxFragment` now ships with plain-htmx
instead of being copied into the example app, and puts the block inside
its wrapper div so the response is the contents — what `hx-swap=innerHTML`
expects.

`lazy=True` is not restored: it relied on the fragment body being a
deferred callable, and component slots render eagerly in the caller's
scope, so it could not actually defer work. Noted in the README and
folded into the ROADMAP's static/dynamic section, which is now the
optimization path for fragments rather than the enabling one.
Which arrow to show, where the link goes next, and how to describe it to
a screen reader are three answers to one question — what state is this
column in. `get_column_headers` answers it once per column and hands the
template a `ColumnHeader`; the template renders it. Before, each of the
three was its own nested conditional expression in the markup, each
recomputing the label.

This also fixes the header: the ported template gated sorting on
`{% if order_by_field %}`, which is empty until something is already
sorted, so an unsorted list rendered no sort links at all. The Jinja
original tested `is defined` — "does this view do sorting" — which for
`admin/list.html` is always true.

Declared attrs in the admin templates now match what the views pass:
`cards`, `fields`, `actions`, `search_fields` and `filters` are tuples.
`plain html check --typecheck` caught the two downstream templates
(plain-flags, plain-jobs) that had to move with them, which is the
component call-site check doing its job.
`plain html check` / `format` take `--template-dir DIR` (repeatable):
the directories to work on, and the roots component paths resolve
against. With them nothing is read from the app — no settings, no
package registry, no database — so the standalone `plain-html` console
script (same pattern plain-code already uses) can check a checkout that
has no app to load.

That closes a real hole. The old script ran from `example/` with
`--include-installed-packages`, which only ever saw packages the example
app installs — plain.oauthserver's template was never checked by
anything. The new walk is every `*/plain/*/templates` plus the example
app: 98 templates, up from 97.

`./scripts/html-check` and the new `./scripts/html-format` now run from
the repo root, which is also where `[tool.ty.environment] extra-paths`
lets ty resolve `app.*` in the example templates' `attrs:`. Both are in
the Postgres-less lint job.
…s/fix

Seven templates had never been through `plain html format`. Now they
have, and `./scripts/fix` runs the formatter alongside ruff — writing by
default, `--check` in the pre-commit and CI paths — so the corpus stays
canonical instead of drifting one file at a time.
The `jinja-test` page and the tests around it are about `{{ }}` being
interpolated into a markdown body before it's served — which `plain.html`
does through `render_text_source`. Nothing Jinja about it any more, and
the name sent readers looking for an engine that isn't there.
One conflict, in plain-postgres's baseline test: master renamed
`since` to `shipped_in`, this branch renamed the fixture package label
`plaintemplates` to `html`. Both.

Master's new ModelForm round-trip suite arrived written for Jinja —
`plain.templates.views` imports and two widget templates with no
frontmatter. Ported. Nothing would have caught that, which is the
argument for the fixture-directory coverage in the next commit.

Master's rule restructure re-added `plain-templates.md` at the package
level. Deleted; its one piece of guidance without a home in the
plain-html rule ("never call `.query` from a template") moved there. The
CSRF and multi-line-attribute bullets were already covered, by the
Django-differences section and by `plain html format` respectively.

Also drops a comment in plain/internal/reloader.py crediting Jinja for
reloading modified templates.
Master's ModelForm suite landed two Jinja templates under
plain-postgres/tests/app/ and nothing noticed, because html-check only
ever walked shipped packages and the example app. It now makes one pass
per app: shipped packages plus the example app together (a few package
templates declare `components: - base`, so they're only checkable inside
a host app that supplies one), then each `*/tests/app/templates` on its
own, with the package dirs as resolution roots but only the app's own
files checked. 120 templates, up from 98.

Two things that found:

`plain html format` was not idempotent on a root-level text run followed
by a block. `_format_root` joins top-level nodes with a newline and
emitted each text node verbatim, so the newline it wrote last time came
back as part of the node — one more blank line on every pass. Root-level
whitespace is the formatter's to decide; the text node's own is dropped
now, matching how whitespace-only nodes were already handled.

The formatter's corpus test never saw that, because it globbed
`templates/*` and kept the directories — which skips every template
sitting directly in a `templates/` dir. That's `base.html`,
`index.html`, `404.html`, 22 files in all. Now covered.
The htmx fixture went through `plain html format` once the fixture
directories joined the corpus, so the slot content is indented now. What
the test means is "the wrapper's contents and nothing else", not "this
exact byte run".
# Conflicts:
#	plain-postgres/tests/internal/test_migrations_reset.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants