Skip to content

[Backport release/0.0.14] fix(zenoh): wait for a session's links only once - #4166

Open
github-actions[bot] wants to merge 1 commit into
release/0.0.14from
backport/4144-to-release/0.0.14
Open

github-actions[bot] wants to merge 1 commit into
release/0.0.14from
backport/4144-to-release/0.0.14

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Description

Backport of #4144 to release/0.0.14.

Co-authored-by: Paul Nechifor <paul@nechifor.net>
(cherry picked from commit 4c0a91c)
@greptile-apps

greptile-apps Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 3/5

Not safe to merge until the Zenoh session shutdown and failed-initialization recovery paths are fixed.

Findings

  1. P1 Prevent closed-session returns
  2. P1 Retry failed initialization
  3. P2 Avoid scheduler-sensitive timing

Summary

The pooled Zenoh session lifecycle has two reproduced shutdown and recovery failures: a caller can receive a session after teardown has closed it, and a transient initial connection failure prevents later same-configuration acquisitions from retrying. The new timing-based test can also fail solely because the caller is descheduled after a successful pooled acquisition.

The lifecycle failures must be resolved before merging. The timing assertion is non-blocking but should be made deterministic to avoid intermittent CI failures.

Reviews (1) · Last reviewed commit: "fix(zenoh): wait for a session's links o..."

else:
entry.ready.set_result(None)
entry.ready.result()
return entry.session

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Prevent closed-session returns

After readiness completes, acquire() returns entry.session without coordinating with close_all(). Teardown can close and remove that entry in the interval, while the caller still receives the now-closed session. The service can finish starting and then fail on its first Zenoh operation. Synchronize the ready-to-return handoff with teardown and retry acquisition when the entry was removed.

Knowledge Base Used: RPC, pubsub, and transforms

Artifacts

Evidence from the check

  • The executed focused pytest source compiles the checked-in pool classes, gates readiness, and forces close_all to close the session before acquire returns; it demonstrates the race without requiring Zenoh to be installed.

Command output from the check

  • The captured pytest command output shows the deterministic assertion failure after close_all closes the session and acquire returns it; the takeaway is that the reported race reproduces.

View artifacts

T-Rex Ran code and verified through T-Rex

Comment on lines +321 to +324
except BaseException as e:
# Release every waiter even if initialization is interrupted.
entry.ready.set_exception(e)
raise

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Retry failed initialization

When the initial link check fails, the session remains in _sessions with a failed readiness future. Every later acquisition for that configuration retrieves the same entry and re-raises the original error instead of opening a replacement session. A transient connection failure therefore leaves the transport unavailable until another caller explicitly runs close_all(). Remove and close the failed entry before propagating the initialization error so later starts can retry.

Knowledge Base Used: RPC, pubsub, and transforms

Artifacts

Evidence from the check

  • The exact focused pytest source executed against the checked pool definitions with deterministic mocks; it expects the second acquire to reopen and exposes the retained-failure behavior.

Command output from the check

  • Pytest executed the first failed initialization followed by a same-config acquire; the second call raised the same error and performed only one open, proving no retry occurred.

Command output from the check

  • The command copied the executed test source into the artifact location and recorded its SHA-256 digest; this confirms the uploaded source matches the executed test.

Command output from the check

  • The captured source lines show the entry is inserted before initialization and its failure path only sets the Future exception; this identifies why the failed entry remains.

View artifacts

T-Rex Ran code and verified through T-Rex

return time.monotonic() - started

assert acquire_seconds() >= 0.2
assert acquire_seconds() < 0.1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Avoid scheduler-sensitive timing

This non-blocking assertion requires the second acquisition's wall-clock measurement to be below 100 ms. The caller can be descheduled after acquire() has already returned the ready pooled session, causing the assertion to fail even though no second link check occurred. This creates intermittent CI failures; assert that the link-wait operation is not invoked instead of relying on a tight elapsed-time threshold.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Artifacts

Evidence from the check

  • This exact authored script loads the production pool implementation, controls its clock, and proves the assertion can fail after pooling has completed; the takeaway is that timing alone is not deterministic proof of a second wait.

Command output from the check

  • The executed script reports a 0.200-second second acquisition measurement, identical pooled session, and unchanged link-call count; the takeaway is that post-acquire descheduling alone fails the original threshold.

Command output from the check

  • The direct pytest command was executed but test collection stopped because the available Python environment lacks `dotenv`; the takeaway is that the focused production-method harness was required for runnable verification.

View artifacts

T-Rex Ran code and verified through T-Rex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants