| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
that require it.
- Clean up some old members and code not used any more.
|
|
|
|
|
|
|
|
|
|
| |
Instead of sending `REJECTED` when lock token does not match, send
`TEST_AND_SET_CONDITION_FAILED` from distributors. The reindexing visitor
will detect this and remap the failure code to `ABORTED` while
simultaneously failing the backend visitor.
`ABORTED` will be considered transient by the visiting client, allowing
it to retry towards another distributor without the whole thing
having to fail out.
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/use-random-bucket-lock-passthrough-token-for-reindexing-visitors
Only let reindexing puts through locked bucket if their TaS token matches that of the lock
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
of the lock
Avoids race condition edge case where reindexing puts from an outdated
visitor may pass through a bucket lock intended for a newly created
visitor operation
Tokens are 128-bit values derived from a CSPRNG, so uniqueness is for
all intents and purposes guaranteed.
|
|\ \
| | |
| | |
| | |
| | | |
vespa-engine/balder/wire-cluster-name-as-reference-all-the-way
- Wire the clustername through as a reference to avoid referencing te…
|
| |/ |
|
|\ \
| |/
|/|
| |
| | |
vespa-engine/geirst/optimize-two-phase-update-operation
Avoid lookup bucket database entries two times when handling fast pat…
|
| |
| |
| |
| | |
update operation.
|
|\ \
| | |
| | | |
Extend use of bucket ownership hash.
|
| |/ |
|
|\ \
| |/
|/|
| |
| | |
vespa-engine/toregge/remove-internal-bucket-join-command
Remove InternalBucketJoinCommand and InternalBucketJoinReply.
|
| | |
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Since reindexing visitors take a bucket lock when they arrive and
wait for pending ops to drain before they start, doing so when there's
a pending merge risks starving the bucket for a long time. This is
because merges may linger for a long time in the merge throttling
queues in the cluster. By not starting such visitors if there is a
pending merge, we avoid this edge case. Functionality is already in
place to inhibit merges from starting if there's an active bucket lock
present.
|
|/ |
|
| |
|
| |
|
| |
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/block-ideal-state-ops-towards-locked-buckets
Block ideal state operations towards buckets that are locked
|
| |
| |
| |
| |
| |
| |
| | |
Prevents ideal state ops from modifying buckets that are being used
in a read-for-write context.
Move `OperationSequencer` to main `Distributor` to more easily
facilitate sharing of it across components.
|
|/ |
|
| |
|
|\
| |
| |
| |
| | |
vespa-engine/geirst/decouple-external-operation-handler
Decouple external operation handler
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
| |
| |
| | |
command.
|
|/ |
|
|\
| |
| |
| |
| | |
vespa-engine/toregge/faster-distributor-bucket-db-update
Add process_update member function to BucketDatabase.
|
| |
| |
| |
| | |
updating an entry with a single lookup.
|
| |
| |
| |
| | |
ha…"
|
|\ \
| | |
| | |
| | |
| | | |
vespa-engine/vekterli/allow-starting-deferred-tasks-concurrently-with-reads
Allow starting deferred tasks concurrently with reads
|
| | |
| | |
| | |
| | |
| | |
| | | |
Since read ops aren't blocked by active bucket locks, a constant
stream of read ops towards a bucket could otherwise starve deferred
tasks from starting.
|
| |/ |
|
| | |
|
|/
|
|
| |
command.
|
|\
| |
| |
| |
| | |
vespa-engine/toregge/remove-distributor-component-trampoline-member-function
Remove DistributorComponent trampoline member function.
|
| | |
|
| | |
|
| | |
|
|\ \
| |/
|/|
| |
| | |
vespa-engine/vekterli/initial-backend-reindexing-visitor-support
Initial support for backend reindexing visitor functionality
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Introduces the concept of a read-for-write visitor operation which
blocks all mutating operations from starting for a bucket being
visited. This read-for-write mode is used if (and only if) the visitor
library being specified by the client is "reindexingvisitor".
Since read-for-write visitors cannot race with concurrent write
operations, starting such visitors are deferred until no further
mutations are pending.
Also adds a basic reindexingvisitor implementation to the content node
which sends all documents as Puts containing a special TaS token
that will let the operation through even if a bucket is locked. This
token is cleared by the distributor before it is passed on to the
content nodes.
Note: this feature is not yet production ready. For now the following
caveats apply:
* Mutating vs non-mutating pending ops to a bucket are not tracked
separately, so it’s possible to starve a reindexing visitor by
sending constant pending read load, as read load is not blocked by
the operation sequencer.
* Ideal state operations towards locked buckets are not blocked, so
it's possible for e.g. a split to be sent for a bucket that is being
visited.
|
| | |
|
| | |
|
| | |
|
| | |
|