Skip to content

A Pathfinder outage is indistinguishable from an empty result set #244

Description

@NathanTarbert

Found by the review of PR #242 (Phase 2, code search).

PathfinderClient.search() collapses every failure to [] with a console.error, so the pipeline cannot tell "the code index is down" from "there are no code hits for this question". Both arrive as an empty array.

The consequence is the silent-no-retrieval mode that #242 exists to remove, one level up: during a code-index outage the answer is built from docs alone, at full confidence, with nothing marking it degraded. Nobody finds out except by reading worker logs.

This matters more now than it did with one retrieval source, because with two the failure is partial and therefore invisible — a docs-only answer still looks like a normal answer.

Shape

Returning { results, degraded } from the search path, or otherwise surfacing the failure to the confidence signal, would let the pipeline mark an answer as degraded when a source it expected was unavailable. ConfidenceAssessment already carries a degraded flag for the scorer-unavailable case, so there is a precedent to follow.

Related: #231 covers the same class of problem in the generator (an error and a guard trip landing on identical output), and #197 is why console.error is the only signal today.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: aiAI response pipeline, groundedness, confidenceroadmapTracked on the Outpost roadmaproadmap: nextRoadmap horizon: after launch path clears

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions