Avoid bucket DB race during content node cluster state transition

It was possible for a distributor bucket fetch request to be processed
_after_ a cluster state was enabled (and internally propagated) on the content
node, but _before_ all side effects of this enabling were complete and fully
visible. This could cause inconsistent information to be returned to the
distributor, causing nodes to get out of sync bucket metadata.

This commit handles such transition periods by introducing an implicit
barrier between observing the incoming command and outgoing reply for a
particular cluster state version. Upon observing the reply for a version,
all side effects must already be visible since the reply is only sent
once internal state processing is complete (both above and below the SPI).
Until initiated and completed versions converge, requests are rejected and
will be transparently retried by the distributors.

0 files changed, 0 insertions, 0 deletions