[eval][1/3] Introduce EvalDispatcher (towards async evals)

[kyuds]

This is the first out of a series of PRs for async evals. We create a new EvalDispatcher class that orchestrates evals and provides a framework for various evaluation execution strategies.

The dispatcher design is simple. For every step of the training loop:

1. submit evaluations for the current step.
2. query whether there are some completed evals and log their metrics via get_completed
3. and when training finishes, we drain all pending evals and log them.

Misc changes:

• We now define custom step metric for wandb as wandb doesn't allow out-of-order logging with the default step metric.
• Callback for on_log no longer contains eval metrics as most async eval strategies don't support it.

---

Note

Medium Risk
Touches core training/eval loops and experiment logging semantics (eval callback timing, per-step tracker writes, wandb step axis); behavior should match blocking eval today but logging and callbacks differ slightly from direct eval() + commit batching.

Overview
Introduces an eval dispatcher layer (BaseEvalDispatcher / BlockingEvalDispatcher) so training loops submit, poll completed, and drain evals instead of calling eval() inline. The dispatcher owns on_eval_start / on_eval_end; the loop logs via _log_eval_results at each result’s global_step (needed for future async eval). RayPPOTrainer and FullyAsyncRayPPOTrainer are wired through this path, with drain + close on exit.

W&B tracking gets a _WandbAdapter that logs a custom global_step axis so metrics can be written out of order (e.g. late eval rows). Tracking.log(commit=...) is deprecated (one committed row per call). eval() gains optional generator / eval_dataloader / global_step overrides for dispatchers. Tests and GPU e2e checks updated for the new step field.

^{Reviewed by Cursor Bugbot for commit 36f4bd9. Bugbot is set up for automated code reviews on this repo. Configure here.}

[gemini-code-assist]

Code Review

This pull request introduces an evaluation dispatcher abstraction (BaseEvalDispatcher, BlockingEvalDispatcher) to decouple training loops from direct evaluation calls, and refactors the W&B tracking adapter to support out-of-order step logging using a custom step axis. Review feedback suggests ensuring accurate timing of baseline evaluations by calling drain() inside the timed blocks in both FullyAsyncTrainer and RayPPOTrainer. Additionally, it is recommended to improve robustness in _WandbAdapter by guarding against self.run being None and using run-specific logging methods, with corresponding updates to the test assertions.

[greptile-apps]

Confidence Score: 4/5

The blocking dispatcher behavior appears functionally safe, but the repository-mandated test-command documentation must be corrected before merging, and exceptional dispatcher cleanup should be hardened for the asynchronous strategies this interface enables.

Findings

1.  Cleanup Skipped On Errors ▶
2.  Test Output Is Not Persisted ▶

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Training step] --> B[submit evaluated step]
    B --> C[Dispatcher runs or schedules evaluation]
    C --> D[get_completed]
    D --> E[EvalResult with evaluated global_step]
    E --> F[Log evaluation metrics at evaluated step]
    A --> G[Log training metrics at current step]
    H[Training finishes] --> I[drain outstanding evaluations]
    I --> F
    F --> J[close dispatcher]

Loading

_{Reviews (1) · Last reviewed commit: "eval dispatcher design"}