-
Notifications
You must be signed in to change notification settings - Fork 5
ci: add shared visual proof author and shots actions [ED-25578] #55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
1bb093a
ci: Add shared Visual proof author and shots actions [ED-25578]
davseve d18c218
fix: Avoid CodeQL injection on github.action_path [ED-25578]
davseve 43361d6
fix: keep visual-proof playwright out of npm workspaces [ED-25578]
davseve 59f3cee
style: format visual-proof files with prettier [ED-25578]
davseve 322fd7f
style: ignore visual-proof scripts in prettier [ED-25578]
davseve 3b3c15b
docs: Run visual-proof-shots when a draft is marked ready [ED-25578]
davseve d44fced
Fix visual-proof actions: shared CJS parser, strict upsert, security …
cursoragent 7f815a5
Fix: Use folder-level ignores for visual-proof actions
cursoragent File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,3 +2,5 @@ | |
| **/dist/** | ||
| **/__snapshots__/** | ||
| package-lock.json | ||
| actions/visual-proof*/** | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # Visual proof author | ||
|
|
||
| Writes `## Visual proof` on a PR body when that section is still missing. | ||
| Ported from Core so Elementor and Elementor Pro share one implementation. | ||
|
|
||
| If the caller checkout has `.cursor/skills/visual-proof/SKILL.md`, that file | ||
| wins. Otherwise the bundled skill in this action’s `skill.md` is used. | ||
|
|
||
| ## Usage | ||
|
|
||
| ```yaml | ||
| name: Visual proof author | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, ready_for_review] | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| author: | ||
| if: github.event.pull_request.head.repo.full_name == github.repository | ||
| runs-on: ubuntu-22.04 | ||
| timeout-minutes: 8 | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| with: | ||
| fetch-depth: 0 | ||
| ref: ${{ github.event.pull_request.head.sha }} | ||
|
|
||
| - uses: elementor/elementor-editor-github-actions/actions/visual-proof-author@main | ||
| with: | ||
| pr-number: ${{ github.event.pull_request.number }} | ||
| pr-head-sha: ${{ github.event.pull_request.head.sha }} | ||
| pr-base-sha: ${{ github.event.pull_request.base.sha }} | ||
| product-name: 'Elementor' | ||
| cursor-api-key: ${{ secrets.CURSOR_APIKEY }} | ||
| model: ${{ vars.PR_REVIEW_MODEL }} | ||
| ``` | ||
|
|
||
| Use `product-name: 'Elementor Pro'` in Pro. | ||
|
|
||
| ## Intentional non-blocking behavior | ||
|
|
||
| The action runs with `continue-on-error: true` because the **Visual proof** | ||
| section is optional for contributors. Author failures (timeout, missing skill, | ||
| API errors) do not block the PR workflow. The shots action will fall back to | ||
| a generic Playground walk when the section is missing or marked `#skip_proof`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| name: 'Visual proof author' | ||
| description: 'Write ## Visual proof on a PR body when the section is still missing' | ||
|
|
||
| inputs: | ||
| pr-number: | ||
| description: 'Pull request number' | ||
| required: true | ||
| pr-head-sha: | ||
| description: 'PR head commit SHA' | ||
| required: true | ||
| pr-base-sha: | ||
| description: 'PR base commit SHA' | ||
| required: true | ||
| product-name: | ||
| description: 'Product name (Elementor or Elementor Pro)' | ||
| required: true | ||
| cursor-api-key: | ||
| description: 'Cursor Agent API key' | ||
| required: true | ||
| model: | ||
| description: 'Cursor Agent model' | ||
| required: false | ||
| default: '' | ||
| github-token: | ||
| description: 'Token with pull-requests:write' | ||
| required: false | ||
| default: ${{ github.token }} | ||
| skill-file: | ||
| description: 'Optional path to a skill markdown file in the caller workspace' | ||
| required: false | ||
| default: '' | ||
| examples-file: | ||
| description: 'Optional path to examples markdown in the caller workspace' | ||
| required: false | ||
| default: '' | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v6 | ||
| with: | ||
| node-version: 24.15.0 | ||
| package-manager-cache: false | ||
|
|
||
| - name: Install Cursor CLI | ||
| shell: bash | ||
| run: | | ||
| curl https://cursor.com/install -fsS | bash | ||
| echo "$HOME/.cursor/bin" >> "$GITHUB_PATH" | ||
|
|
||
| - name: Author ## Visual proof if missing | ||
| shell: bash | ||
| continue-on-error: true | ||
| env: | ||
| GH_TOKEN: ${{ inputs.github-token }} | ||
| CURSOR_API_KEY: ${{ inputs.cursor-api-key }} | ||
| GITHUB_REPOSITORY: ${{ github.repository }} | ||
| PR_NUMBER: ${{ inputs.pr-number }} | ||
| PR_HEAD_SHA: ${{ inputs.pr-head-sha }} | ||
| PR_BASE_SHA: ${{ inputs.pr-base-sha }} | ||
| PR_REVIEW_MODEL: ${{ inputs.model }} | ||
| PRODUCT_NAME: ${{ inputs.product-name }} | ||
| VISUAL_PROOF_SKILL_FILE: ${{ inputs.skill-file }} | ||
| VISUAL_PROOF_EXAMPLES_FILE: ${{ inputs.examples-file }} | ||
| ACTION_PATH: ${{ github.action_path }} | ||
| run: bash "$ACTION_PATH/author.sh" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| # Visual proof author (CI) | ||
|
|
||
| You are the only Cursor job that **writes** `## Visual proof` (including **Steps**). You run when that section is still missing on PR **opened** or **ready for review**. You do not review code, capture screenshots, or change git files. | ||
|
|
||
| Follow the Visual proof skill and examples pasted below. The product name is in the runtime context (Elementor or Elementor Pro). | ||
|
|
||
| ## Context (CI fills this) | ||
|
|
||
| The extra message after this prompt has repository, PR number, and SHAs. | ||
|
|
||
| ## Steps | ||
|
|
||
| 1. `gh pr view "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --json title,body,labels` — read title, body, labels. | ||
| 2. If the body already has a line that is exactly `## Visual proof`, stop. Do not rewrite it. | ||
| 3. `git diff --stat "$PR_BASE_SHA"..."$PR_HEAD_SHA"` and skim the diff. Decide: | ||
| - Editor Bug you can show on this repo’s Playground preview → Broken + Where / Steps / Pass / Fail. | ||
| - Anything else (Task, docs, CI, missing plugin on Playground, no editor UI) → heading + `#skip_proof` + one sentence. | ||
| 4. Insert the section **after** `## Test plan` and **before** `## Jira` or `<!--start_gitstream`. If those headings are missing, insert after `## Summary`, else append at the end of the human-written body (still before gitstream). | ||
| 5. Do not edit Summary, Test plan, or Jira except to insert this block. | ||
| 6. Write the new full body to a temp file. Run: | ||
| `gh pr edit "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --body-file "$tmp"` | ||
| 7. Confirm `gh pr view` now contains `## Visual proof`. Print a one-line summary of skip vs filled. Then stop. | ||
|
|
||
| ## Hard rules | ||
|
|
||
| - Visible in-app labels only inside Steps. CI will try to click those labels on Playground. | ||
| - Do not act out the bug. Playground has the fixed zip. Broken is past tense. | ||
| - Never leave placeholders (`TODO`, `TBD`, angle-bracket templates). | ||
| - Never print `GH_TOKEN` or `CURSOR_API_KEY`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| REPO_ROOT="${GITHUB_WORKSPACE:-$(pwd)}" | ||
|
|
||
| log() { | ||
| echo "[visual-proof:author] $*" | ||
| } | ||
|
|
||
| err() { | ||
| echo "[visual-proof:author] $*" >&2 | ||
| } | ||
|
|
||
| require_env() { | ||
| local name="$1" | ||
| if [[ -z "${!name:-}" ]]; then | ||
| err "${name} is required" | ||
| exit 1 | ||
| fi | ||
| } | ||
|
|
||
| require_env GH_TOKEN | ||
| require_env GITHUB_REPOSITORY | ||
| require_env PR_NUMBER | ||
| require_env PR_HEAD_SHA | ||
| require_env PR_BASE_SHA | ||
| require_env CURSOR_API_KEY | ||
|
|
||
| log "step=start PR_NUMBER=${PR_NUMBER} HEAD_SHA=${PR_HEAD_SHA} BASE_SHA=${PR_BASE_SHA}" | ||
|
|
||
| log "step=parse-section reading PR body" | ||
| DECISION=$( | ||
| gh pr view "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --json body --jq .body | node -e ' | ||
| const { shouldCaptureVisualProof } = require(process.argv[1]); | ||
|
Ntnelbaba marked this conversation as resolved.
|
||
| const fs = require("fs"); | ||
| const body = fs.readFileSync(0, "utf8"); | ||
| process.stdout.write(JSON.stringify(shouldCaptureVisualProof(body))); | ||
| ' "${SCRIPT_DIR}/../visual-proof/parse-section.cjs" | ||
| ) | ||
|
|
||
| REASON=$(echo "$DECISION" | jq -r .reason) | ||
| log "step=parse-section reason=${REASON}" | ||
|
|
||
| if [[ "$REASON" != "missing-section" ]]; then | ||
| log "step=skip section already present (${REASON})" | ||
| exit 0 | ||
| fi | ||
|
|
||
| PROMPT_FILE="${SCRIPT_DIR}/author-prompt.md" | ||
| if [[ -n "${VISUAL_PROOF_SKILL_FILE:-}" && -f "$VISUAL_PROOF_SKILL_FILE" ]]; then | ||
| SKILL_FILE="$VISUAL_PROOF_SKILL_FILE" | ||
| elif [[ -f "${REPO_ROOT}/.cursor/skills/visual-proof/SKILL.md" ]]; then | ||
| SKILL_FILE="${REPO_ROOT}/.cursor/skills/visual-proof/SKILL.md" | ||
| else | ||
| SKILL_FILE="${SCRIPT_DIR}/skill.md" | ||
| fi | ||
|
|
||
| if [[ -n "${VISUAL_PROOF_EXAMPLES_FILE:-}" && -f "$VISUAL_PROOF_EXAMPLES_FILE" ]]; then | ||
| EXAMPLES_FILE="$VISUAL_PROOF_EXAMPLES_FILE" | ||
| elif [[ -f "${REPO_ROOT}/.cursor/skills/visual-proof/examples.md" ]]; then | ||
| EXAMPLES_FILE="${REPO_ROOT}/.cursor/skills/visual-proof/examples.md" | ||
| else | ||
| EXAMPLES_FILE="${SCRIPT_DIR}/examples.md" | ||
| fi | ||
|
|
||
| if [[ ! -f "$PROMPT_FILE" || ! -f "$SKILL_FILE" ]]; then | ||
| err "author prompt or visual-proof skill is missing" | ||
| exit 1 | ||
| fi | ||
|
|
||
| PROMPT=$(cat "$PROMPT_FILE") | ||
| SKILL=$(cat "$SKILL_FILE") | ||
| EXAMPLES="" | ||
| if [[ -f "$EXAMPLES_FILE" ]]; then | ||
| EXAMPLES=$(cat "$EXAMPLES_FILE") | ||
| fi | ||
|
|
||
| DYNAMIC_CONTEXT=$(cat << EOF | ||
| ## Runtime | ||
| - Product: ${PRODUCT_NAME:-Elementor} | ||
| - Repository: ${GITHUB_REPOSITORY} | ||
| - PR Number: ${PR_NUMBER} | ||
| - PR Head SHA: ${PR_HEAD_SHA} | ||
| - PR Base SHA: ${PR_BASE_SHA} | ||
|
|
||
| ## Skill | ||
| ${SKILL} | ||
|
|
||
| ## Examples | ||
| ${EXAMPLES} | ||
| EOF | ||
| ) | ||
|
|
||
| MODEL="${PR_REVIEW_MODEL:-}" | ||
| log "step=cursor-agent about to author Visual proof model=${MODEL:-default}" | ||
|
|
||
| cd "$REPO_ROOT" | ||
|
|
||
| set +e | ||
| if [[ -n "$MODEL" ]]; then | ||
| timeout 180 cursor-agent --force --model "$MODEL" --output-format=text --print "$PROMPT" "$DYNAMIC_CONTEXT" | ||
| else | ||
| timeout 180 cursor-agent --force --output-format=text --print "$PROMPT" "$DYNAMIC_CONTEXT" | ||
| fi | ||
| AGENT_EXIT=$? | ||
| set -e | ||
|
|
||
| if [[ "$AGENT_EXIT" -eq 124 ]]; then | ||
| err "step=cursor-agent timed out after 3 minutes" | ||
| exit 1 | ||
| fi | ||
| if [[ "$AGENT_EXIT" -ne 0 ]]; then | ||
| err "step=cursor-agent failed exit=${AGENT_EXIT}" | ||
| exit "$AGENT_EXIT" | ||
| fi | ||
|
|
||
| AFTER=$( | ||
| gh pr view "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --json body --jq .body | node -e ' | ||
| const { shouldCaptureVisualProof } = require(process.argv[1]); | ||
| const fs = require("fs"); | ||
| const body = fs.readFileSync(0, "utf8"); | ||
| process.stdout.write(JSON.stringify(shouldCaptureVisualProof(body))); | ||
| ' "${SCRIPT_DIR}/../visual-proof/parse-section.cjs" | ||
| ) | ||
| AFTER_REASON=$(echo "$AFTER" | jq -r .reason) | ||
| log "step=verify reason=${AFTER_REASON}" | ||
|
|
||
| if [[ "$AFTER_REASON" == "missing-section" ]]; then | ||
| err "cursor-agent finished but ## Visual proof is still missing" | ||
| exit 1 | ||
| fi | ||
|
|
||
| log "step=done" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # Visual proof — examples | ||
|
|
||
| ## Editor bug | ||
|
|
||
| ```markdown | ||
| ## Visual proof | ||
|
|
||
| **Broken:** Z-index stayed disabled on Tablet when Position was inherited from Desktop. | ||
| **Where:** Edit with Elementor → select a container → Style → Position → device switcher | ||
| **Steps:** on Desktop set Position to Absolute → switch to Tablet → open Style → Position → set Z-index to 5 | ||
| **Pass:** Z-index is enabled on Tablet and value 5 is accepted | ||
| **Fail:** Z-index remains disabled on Tablet | ||
| ``` | ||
|
|
||
| ## Skip — not an editor bug | ||
|
|
||
| ```markdown | ||
| ## Visual proof | ||
|
|
||
| #skip_proof | ||
| Task adds a new Style control; visual proof is editor-bugs only. | ||
| ``` | ||
|
|
||
| ## Skip — cannot run on this Playground | ||
|
|
||
| ```markdown | ||
| ## Visual proof | ||
|
|
||
| #skip_proof | ||
| Fix needs a plugin that playground-preview does not install. | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Visual proof | ||
|
|
||
| This skill is for the **CI author** action. Local Cursor agents should not add | ||
| `## Visual proof` on `gh pr create` or `gh pr edit`. Capturing screenshots is | ||
| not this skill. | ||
|
|
||
| ## What triggers it | ||
|
|
||
| The caller’s `visual-proof-author` workflow runs this action when a same-repo | ||
| PR is **opened** or marked **ready for review**, and only if `## Visual proof` | ||
| is still missing. Capture is a separate `visual-proof-shots` action after | ||
| `playground-preview`. | ||
|
|
||
| ## Environment | ||
|
|
||
| The proof target is the PR’s **playground-preview** deployment in the calling | ||
| repo (Elementor or Elementor Pro), not a local site. | ||
|
|
||
| Login: `admin` / `password`. Landing page is `/wp-admin`. | ||
|
|
||
| If the bug cannot be shown on that Playground, use `#skip_proof`. | ||
|
|
||
| ## When to fill vs skip | ||
|
|
||
| **Fill** (Broken + Where / Steps / Pass / Fail): | ||
|
|
||
| - Jira type `Bug` or `Editor Bug` (or a clear regression fix) | ||
| - User-visible **editor** UI (panel, canvas, navigator, Style / Content) | ||
| - Can be shown on this repo’s Playground | ||
|
|
||
| **`#skip_proof`** + one sentence: | ||
|
|
||
| - Story / Task / feature | ||
| - No editor UI | ||
| - Needs a plugin Playground does not install | ||
| - Docs or CI | ||
|
|
||
| ## Rules | ||
|
|
||
| 1. Write **Steps** the actor can click: short, visible labels, stay in the editor. | ||
| 2. Visible in-app labels only. No file paths, no GitHub, no workflow names. | ||
| 3. Do not act out the bug. Playground has the **fixed** zip. Put the old | ||
| behaviour in `**Broken:**`, then show the fixed path. | ||
|
|
||
| ## PR body | ||
|
|
||
| Order: Summary → Test plan → Visual proof → Jira. | ||
|
|
||
| ### Editor bug | ||
|
|
||
| ```markdown | ||
| ## Visual proof | ||
|
|
||
| **Broken:** <one sentence: what used to happen in the editor> | ||
| **Where:** <editor path using visible labels> | ||
| **Steps:** <happy path on this PR’s Playground> | ||
| **Pass:** <fixed editor behaviour visible> | ||
| **Fail:** <bug still visible> | ||
| ``` | ||
|
|
||
| ### Skip | ||
|
|
||
| ```markdown | ||
| ## Visual proof | ||
|
|
||
| #skip_proof | ||
| <One sentence why this cannot be shown on this repo’s Playground editor.> | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| node_modules/ | ||
| .npm-playwright/ | ||
| video-raw/ |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.