Skip to content

The fetcher's timeouts call a healthy adopter feed unreachable #70

Description

@hhkaos

validator.opentechevents.org cannot check https://eventos.wiki/events.json — a registered adopter feed that is valid OTE 0.4 — because the fetcher gives up long before the origin answers.

GET /fetch?url=https%3A%2F%2Feventos.wiki%2Fevents.json
{"ok":false,"code":"timeout",
 "message":"That server did not answer within 5 s (10 s for the whole chain, redirects included). …"}

GET /badge?doc=https%3A%2F%2Feventos.wiki%2Fevents.json   →   "OTE feed: unreachable"

Measured directly, the same feed answers 200 with 551,915 bytes in 26–51 s (three runs today: 51.0 s, 50.9 s, 26.6 s — it is consistently slow, not occasionally). Current limits, from /health:

{"maxBytes":5242880,"maxRedirects":3,"hopTimeoutMs":5000,"totalTimeoutMs":10000}

The document is well under maxBytes; only the clock is the problem.

Why this matters beyond one feed

The same false positive was just removed from the two other tools in the ecosystem, and this one now contradicts them:

"Slow origin" is not "broken feed". A community feed generated on demand, on a small host, is exactly the population OTE is for.

Options

  1. Raise the limits — the cheap fix. Something like 20–30 s per hop and 45–60 s total, aligned with the other tools. A Worker can stream and wait far longer than 10 s; the risk is holding a request open, not CPU.
  2. Keep a short budget but report it honestly. If the deadline stays low, "unreachable" is the wrong word for "did not finish in 10 s": both the message and the badge should say slow, and the badge should not present it as the feed's failure.
  3. Upstream: paginate large feeds. The real fix for a 550 KB feed may be not to serve it in one response. The spec today says the opposite on purpose — "The feed is an interchange format, not an API: no pagination, no filtering, no authentication, no federation" (spec/v0.4/README.md) — so this is a spec-level question, not something to solve here. Tracked separately in Should large feeds have a way to be served in parts? opentechevents-spec#48.

Options 1 and 2 are not exclusive, and neither waits for 3.

Suggested acceptance

  • /fetch and /badge return a verdict for https://eventos.wiki/events.json.
  • A genuinely dead host still fails fast — do not raise the connect deadline to the same value as the download one.
  • Whatever budget is chosen, /health keeps publishing it, and the page's message states it in the same terms the badge uses.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions