Stop worker processes even when finalization rethrows a task failure

[giordano]

No description provided.

[giordano]

#158 and #159 were relatively simple and hopefully uncontroversial, this one is more convoluted and probably warrants a more careful review (ignoring whitespace changes helps). At least I verified that the newly added tests fail on main, without the corresponding source code fix.

[christiangnrd]

Sorry for the Claude review. These seem like unlikely issues that may not be worth the churn to fix unless someone actually encounters them?

🤖
1. Unguarded Malt.stop in cleanup finally — PLAUSIBLE.
The finalization finally (src/ParallelTestRunner.jl:1600-1611) now runs on teardown-after-crash paths, where workers are likely dead or half-dead — exactly when Malt.stop can throw (TerminatedWorkerException race after the isrunning check, socket IOError, or error() on a SIGKILL-resistant worker). A throw aborts the remaining stops (leaking workers) and, being in a finally, replaces the real propagating exception. The serial-worker stop at line 1603 is worst: it precedes close(worker_pool), so one throw leaks the whole pool. Fix: wrap each stop in try/catch.

2. Printer death mid-run hangs before the new cleanup — PLAUSIBLE, pre-existing.
The new comment claims cleanup runs "even when a … printer task failed", but only covers failure surfacing at wait(printer_task). If the printer dies mid-run (non-Interrupt exception, e.g. IOError on a closed output pipe → rethrow() at line 1340), nothing closes printer_channel; its 100-slot buffer fills, worker tasks block forever in put! (lines 1426/1453/1472), the phase @sync never returns, and the run hangs upstream of the new try/finally — workers leak and the run never ends. Fix: on printer failure, close printer_channel or call stop_work() before rethrowing, so the failure reaches finalization instead of deadlocking.

3. The stranded serial worker is compensated for downstream instead of fixed at the source — CONFIRMED (cleanup).
The leak originates in run_test_phase: the hand-back at lines 1510–1514 is skipped when the @sync throws, leaving the pool a slot short — a state the PR compensates for in finalization (lines 1600–1604) and that would deadlock drain_pool_leaving_n_workers! if ever reached. Fix: move the hand-back into a try/finally around the @sync (safe: @sync waits for all tasks, whose own finallys have already refilled the Ref; the pool is never closed mid-phase), then delete the finalization special case.