Skip to content

feat(skills): add the PolyCSS agent skill package and evaluation - #84

Merged
apresmoi merged 37 commits into
mainfrom
feat/agent-skill-package
Aug 16, 2026
Merged

feat(skills): add the PolyCSS agent skill package and evaluation#84
apresmoi merged 37 commits into
mainfrom
feat/agent-skill-package

Conversation

@apresmoi

@apresmoi apresmoi commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add @layoutit/polycss-skills, a zero-dependency npx installer that drops the PolyCSS agent skill into .claude/skills or .agents/skills
  • own the skill content in packages/skills/skill/ and mirror it to website/public/skill/ with pnpm sync:skill; pnpm check:skill fails on drift in CI
  • split the single skill.md into an entry point plus 13 per-topic reference docs (authoring invariants, scenes, shapes, loading, lighting, shadows, textures, controls, animation, performance, three parity, troubleshooting, api index)
  • track installs with a content-hash manifest so an upgrade refreshes shipped docs, drops removed ones, and refuses to overwrite files the user edited
  • fix the receiver-shadow lift default: it was 0.001 against a documented 0.05, so castShadow + receiveShadow emitted paths that darkened nothing in all three renderers
  • add eval/skill/, which runs real agent CLIs in isolated workspaces outside the repo and grades what the scene paints in Chromium, across three tracks: polycss (skill installed), polycss-noskill (the control — same library and contract, skill withheld), and three (an external baseline, not a control)

Results

Withdrawn pending a re-run. The earlier skill-vs-no-skill table was measured
with a harness that gave the intervention an extra "you have the PolyCSS skill
installed — use it" line the control never received, so it compared the skill
plus an activation cue against nothing. Agents were also the outer loop, so
ordering skill-less tracks first only isolated the first agent — later controls
could read an earlier agent's installed skill through a sibling path. Both are
fixed here; the numbers they produced are not trustworthy and have been removed
rather than restated.

The harness itself is verified independently of any paid run: reference
solutions score 46/46 on polycss, 29/29 on polycss-noskill and 29/29 on
three, and pnpm eval:selftest catches 13/13 injected mistakes, so the
graders both pass correct work and fail broken work.

shadow.lift remains a known PolyCSS limitation, unrelated to any score: it is
expressed in world units while the depth conflict it must win is resolved in
device pixels, so below zoom ≈ 1 a cast shadow collapses into its receiver and
disappears. The default camera zoom is 0.65. Documented in shadows.md and
troubleshooting.md with the workaround; a projection-aware fix changes
documented semantics across three renderers and is left for a separate decision.

Scores from this harness are evidence that the skill is sufficient, not proof it
was the only source used — workspaces sit outside the repo but are not
sandboxed, and track order is fixed for isolation, which confounds wall-clock
comparisons. eval/skill/README.md states both limits. Gemini is not covered;
its CLI blocks on an interactive login.

Verification

  • pnpm test
  • pnpm build:packages
  • pnpm build:website
  • pnpm test:scripts
  • pnpm check:readmes
  • pnpm check:skill
  • pnpm eval:skill --agent oracle --track all — reference solutions at 46/46 (polycss), 29/29 (polycss-noskill), 29/29 (three)
  • pnpm eval:selftest — 14/14 mutations caught

@apresmoi
apresmoi merged commit 8576085 into main Aug 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant