Skip to content

feat(capture): file a link card when a page cannot be read - #96

Merged
arthware-dev merged 1 commit into
mainfrom
feat/web-capture-gate
Sep 16, 2026
Merged

arthware-dev merged 1 commit into
mainfrom
feat/web-capture-gate

Conversation

@arthware-dev

Copy link
Copy Markdown
Contributor

A pasted link that a site refuses to serve used to become either a silent failure or a wiki entry summarising a cookie banner. Every fetched page is now classified before anything is filed, and a blocked link still files as a link card carrying whatever the sender wrote around it.

The chat reply names the obstacle (bot check, sign-in, consent wall, paywall, no text) instead of "couldn't read that link". Recipes come from the page's own JSON-LD, so quantities are the site's rather than a summary's.

Phase 1 of docs/design/web/plan.md. Re-validation of the spike findings, including one reversal, is recorded in that doc.

@arthware-dev

Copy link
Copy Markdown
Contributor Author

Rig verification, live against the local model (stacktests ai local, Qwen3.6-35B-A3B).

Blocked page - https://www.decathlon.de/ posted with the text "Gear list for the camping trip":

🔒 www.decathlon.de checks for bots, so I couldn't read the page. Saved the link.
✅ Captured: Camping trip gear list (Decathlon)
   camping | gear-list | decathlon | shopping | Homer

The entry is built from the sender's own words. Previously this was a silent ❌ or an entry summarising the Cloudflare challenge page.

Recipe - tier 1, JSON-LD, 968 chars, no LLM guessing at the page:

✅ Captured: Greek Salad Recipe (Griechischer Salat)
   - Servings: 4
   - Total time: 35 min
   - Ingredients: 500 g red and yellow peppers, 300 g tomatoes, 400 g cucumber, ...

Shell page - maps/place/Brandenburger+Tor/ filed as "Brandenburger Tor" via tier 0, instead of a summary of "Find local businesses, view maps".

Log lines:

[extractor] https://www.decathlon.de/ → challenge: bot protection served 'just a moment...' instead of the page
[extractor] https://www.essen-und-trinken.de/rezepte/48816-... → tier 1 (default), 968 chars
[extractor] https://www.google.com/maps/place/Brandenburger+Tor/ → tier 0 (google-maps), 71 chars

Tests: 648 framework, 2165 stacklets, both green. The 5 fixtures under tests/fixtures/web/ are real captures with inline script/style bodies stripped; provenance and the strip rules are in their README.

A pasted link that a site refuses to serve produced either a silent
failure or an entry summarising a consent banner. Fetched pages are now
classified before anything is filed, and a page that cannot be read
still files as a link card carrying the text the sender wrote with it.

The chat reply names the obstacle: bot protection, sign-in, consent
wall, paywall, or a page with no text.

Recipes are read from the page's own structured data, so quantities and
steps come from the publisher rather than from a summary. Application
shell pages whose subject appears in the URL are answered from the URL.

Refs docs/design/web/plan.md (phase 1).
@arthware-dev arthware-dev changed the title feat(capture): keep the link when a page cannot be read feat(capture): file a link card when a page cannot be read Sep 16, 2026
@arthware-dev
arthware-dev merged commit 42d06b6 into main Sep 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant