feat(skills): add the PolyCSS agent skill package and evaluation - #84
Merged
Conversation
…holds from the control
…ount tiles by region
…and shadow zoom limit
…IFT to React and Vue
… scope and gizmo example
…omise the temp name
…optional gizmo field
…s, fail on missing cases
…rols before interventions
…on the oracle, measure framing on the subject
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@layoutit/polycss-skills, a zero-dependencynpxinstaller that drops the PolyCSS agent skill into.claude/skillsor.agents/skillspackages/skills/skill/and mirror it towebsite/public/skill/withpnpm sync:skill;pnpm check:skillfails on drift in CIskill.mdinto an entry point plus 13 per-topic reference docs (authoring invariants, scenes, shapes, loading, lighting, shadows, textures, controls, animation, performance, three parity, troubleshooting, api index)liftdefault: it was0.001against a documented0.05, socastShadow+receiveShadowemitted paths that darkened nothing in all three rendererseval/skill/, which runs real agent CLIs in isolated workspaces outside the repo and grades what the scene paints in Chromium, across three tracks:polycss(skill installed),polycss-noskill(the control — same library and contract, skill withheld), andthree(an external baseline, not a control)Results
Withdrawn pending a re-run. The earlier skill-vs-no-skill table was measured
with a harness that gave the intervention an extra "you have the PolyCSS skill
installed — use it" line the control never received, so it compared the skill
plus an activation cue against nothing. Agents were also the outer loop, so
ordering skill-less tracks first only isolated the first agent — later controls
could read an earlier agent's installed skill through a sibling path. Both are
fixed here; the numbers they produced are not trustworthy and have been removed
rather than restated.
The harness itself is verified independently of any paid run: reference
solutions score 46/46 on
polycss, 29/29 onpolycss-noskilland 29/29 onthree, andpnpm eval:selftestcatches 13/13 injected mistakes, so thegraders both pass correct work and fail broken work.
shadow.liftremains a known PolyCSS limitation, unrelated to any score: it isexpressed in world units while the depth conflict it must win is resolved in
device pixels, so below
zoom ≈ 1a cast shadow collapses into its receiver anddisappears. The default camera zoom is
0.65. Documented inshadows.mdandtroubleshooting.mdwith the workaround; a projection-aware fix changesdocumented semantics across three renderers and is left for a separate decision.
Scores from this harness are evidence that the skill is sufficient, not proof it
was the only source used — workspaces sit outside the repo but are not
sandboxed, and track order is fixed for isolation, which confounds wall-clock
comparisons.
eval/skill/README.mdstates both limits. Gemini is not covered;its CLI blocks on an interactive login.
Verification
pnpm testpnpm build:packagespnpm build:websitepnpm test:scriptspnpm check:readmespnpm check:skillpnpm eval:skill --agent oracle --track all— reference solutions at 46/46 (polycss), 29/29 (polycss-noskill), 29/29 (three)pnpm eval:selftest— 14/14 mutations caught