Call the bandit with no cache lock held - #14
Merged
Conversation
Three properties the epoch does not have yet. All three fail against this
commit's parent, which is the point of committing them first.
SlowBanditDoesNotBlockReaders a bandit parked in SelectPolicy must not
stall Get. Fails: "Get blocked behind the
bandit: a slow bandit is a cache outage".
StaleSelectionIsDiscarded a decision superseded by a later epoch must
be dropped. Fails by deadlock -- the second
epoch cannot start while the first holds the
write lock, so the package times out.
TickerAndRequestEpochsDoNotLose every counted request reaches the bandit or
Counts is still live in its arm, under both epoch
clocks at once.
Go's RWMutex queues new readers behind a waiting writer, so a bandit called
while the cache's write lock is held stalls every Get in the process for as
long as it runs: a store timeout becomes a cache outage. The godoc on Bandit
currently manages this by forbidding it -- "an implementation must not block"
-- which pushes the problem onto every implementer and is why the distributed
bandit had to buffer and serve selections from an atomic.
An epoch used to run entirely under the cache's write lock, bandit call
included. Go's RWMutex queues new readers behind a waiting writer, so for as
long as SelectPolicy ran, every Get and Add in the process waited on it: a
bandit that read a network store turned a store timeout into a cache outage.
The interface managed this by forbidding it -- "an implementation must not
block" -- which pushed the problem onto every implementer, and is why the
distributed bandit had to buffer its reports and serve selections from an
atomic.
The epoch now runs in three phases:
1. under the write lock: close any gradual window, read and reset every
arm's counters into an epochSnapshot;
2. holding no cache lock: deliver that snapshot and ask for a selection;
3. under the write lock again: check the selection is still current, apply.
Phases 1 and 3 are each atomic, so no caller can observe a half-applied
switch. Between them the cache is fully usable.
Two things the split needs that holding one lock gave for free.
Calls into the bandit are serialised by a new banditMu, taken only while no
cache lock is held. A Bandit is caller-supplied code with no stated
concurrency contract and it used to be entered under the write lock; that
guarantee is kept, and the cost of a slow bandit is now paid by the next
epoch rather than by every reader.
A selection is dropped if another epoch collected while it was being made.
Such a decision was computed from evidence two epochs old, and the stability
gates would check it against c.epochStats the newer epoch has since
overwritten -- it would be admitted or rejected on numbers that are not its
own.
epochsCollected is a second counter, and the reason is not decorative: epochID
advances when an epoch is applied and the stability gates are specified
against it, so reusing it for staleness moved the cooldown arithmetic by one
epoch. TestSwitchStability_CooldownRearmsAfterSwitch caught that.
The Bandit godoc and docs/design.md no longer forbid blocking. They state what
it costs instead: the switch is delayed and may be dropped, concurrent Get and
Add are not affected, and under EpochRequests the caller completing the epoch
still pays -- as it already did for the switch and any migration.
Verified: go test -race -short -count=5 on the root module, make all across
nine modules, make evidence regenerated. Hit rates move within run-to-run
noise (zipf 68.14% against a recorded 67.88%, with otter moving 73.36% against
73.25% in the same run) and no table is rewritten. Serial hot-path benchmarks
are unchanged -- Get 102/153 ns, sampled 41/44, Add 98/144, sampled 53/55 --
which is expected: the new lock is on the epoch path, and Get's path is
untouched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An epoch used to run entirely under the cache's write lock, bandit call
included. Go's
RWMutexqueues new readers behind a waiting writer, so for aslong as
SelectPolicyran, everyGetandAddin the process waited onit — a bandit reading a network store turned a store timeout into a cache
outage. The interface managed this by forbidding it ("an implementation must
not block"), which pushed the problem onto every implementer and is why the
distributed bandit had to buffer its reports and serve selections from an
atomic.
The split
Phases 1 and 3 are each atomic, so no caller observes a half-applied switch.
Between them the cache is fully usable.
Two things holding one lock gave for free
Serialisation. Calls into the bandit are serialised by a new
banditMu,taken only while no cache lock is held. A
Banditis caller-supplied code withno stated concurrency contract, and it used to be entered under the write lock;
that guarantee is kept. A slow bandit is now paid for by the next epoch instead
of by every reader.
Freshness. A selection is dropped if another epoch collected while it was
being made: it was computed from evidence two epochs old, and the stability
gates would check it against
c.epochStatsthe newer epoch has alreadyoverwritten.
epochsCollectedis a second counter and the reason is not decorative.epochIDadvances when an epoch is applied and the stability gates arespecified against it, so reusing it for staleness moved the cooldown arithmetic
by one epoch.
TestSwitchStability_CooldownRearmsAfterSwitchcaught that.Red → green
The first commit (
798f9f0) is the failing tests. Against its parent:SlowBanditDoesNotBlockReadersGet blocked behind the bandit: a slow bandit is a cache outageStaleSelectionIsDiscardedCondition never satisfied— the second epoch cannot collect while the first holds the write lockTickerAndRequestEpochsDoNotLoseCountsThe tests were retargeted after that commit — the first version used
EpochRequests: 1, under which everyGetruns an epoch, so there was nonon-triggering reader to measure. The final versions were re-run against the
pre-refactor implementation and still fail there, which is the property that
matters.
Contract change
The
Banditgodoc anddocs/design.mdno longer forbid blocking. They statewhat it costs: the switch is delayed and may be dropped, concurrent
GetandAddare unaffected, calls are serialised, and underEpochRequeststhecaller completing the epoch still pays — as it already did for the switch and
any migration it triggers.
Verification
Hit rates move within run-to-run noise (zipf 68.14% against a recorded 67.88%,
with otter moving 73.36% against 73.25% in the same run); no table is
rewritten. Serial hot-path benchmarks unchanged —
Get102/153 ns, sampled41/44;
Add98/144, sampled 53/55 — as expected, since the new lock is on theepoch path and
Get's path is untouched.Class H — please read
epoch.goin full, plus the newdocs/design.mdparagraph. This is a concurrency core; the reviewer is the weak link here.