Fail the parallel run when a worker returns no result - #3012
Merged
Conversation
Four files went into a Run.Parallel run on CI and three came back. The missing file was not failed, skipped or errored, it was not reported at all, and the run looked green with one test fewer than it should have had. Invoke-TestInParallel filters worker results with a Where-Object that keeps only well formed result objects, to guard against stray pipeline output. That filter cannot tell stray output from a worker that died before returning its result, and nothing afterwards compared the number of results to the number of files sent. So a lost file was dropped as quietly as a stray string, and Sort-Object put the survivors in discovery order, which made the output look like a normal shorter run. Compare the counts after the filter and throw, naming the files that went missing. The worker's own error is already surfaced by Invoke-InRunspacePool, this says which file it cost us. Losing results silently is worse than failing. This is independent of why the worker died. Any worker that dies for any reason costs a whole test file, and #3004 only stopped one failing worker from aborting the results of the others. Added a test that hands Invoke-TestInParallel a runspace pool runner which drops one file's result, and asserts the run throws and names that file. Fix #3011 🤖
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four files went into a
Run.Parallelrun on CI and three came back. The missing file was not failed, skipped or errored, it was not reported at all, and the run looked green with one test fewer than it should have had. Details and the CI log are in #3011.Invoke-TestInParallelfilters worker results to guard against stray pipeline output:That line cannot tell stray output from a worker that died before it returned its result object, and nothing afterwards compared the number of results to the number of files sent. A lost file was dropped as quietly as a stray string, and
Sort-Objectthen put the survivors in discovery order, so the output looked like a normal shorter run.Now the counts are compared after the filter, and a short result set throws and names the files that went missing. The worker's own error is already surfaced by
Invoke-InRunspacePool, this says which file it cost us.This does not depend on why the worker died, any worker that dies for any reason costs a whole test file. #3004 stopped one failing worker from aborting the results of the others, it did not check that every file came back.
The import race that killed the worker in the first place is the other half of #3011 and is not fixed here. I could not reproduce it locally in 288 staggered imports, 1600 worker imports across 200 real parallel runs, or 400 imports timed onto the five second
validVerbsremoval.Tests
Added a test that hands
Invoke-TestInParallela runspace pool runner which drops one file's result, and asserts the run throws and names that file. Full local suite: 2915 passed, 0 failed, 3 skipped.Fix #3011
🤖