| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
Made side effects of replica resurrection bug more likely to be observed,
but not in itself a believed root cause of any issues.
|
|
|
|
|
|
|
|
|
|
|
| |
We previously only checked for node availability in the _active_ state
without looking at the pending state. This opened up for a race condition
where a reply for a previously DB-pruned node could bring a replica back
in the DB iff received during a pending state window.
Consider Maintenance as unavailable for this case, not just Down.
Also move all `PutOperation` tests to GTest.
|
|
|
|
|
|
|
|
|
|
| |
Buckets are sorted in increasing key order before being merged into
the database. Instead of repeatedly calling `toKey()` on entries, just
store the key verbatim.
Add a (by default disabled) benchmarking test for this case.
With the bucket key change, running this locally brings total merge time
for 917K buckets down from 2 seconds to 1.3 seconds.
|
|
|
|
|
|
| |
There may be a currently unknown edge case where some valid edges do not
trigger pruning as they should, so disable optimization entirely until
we know for sure.
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/add-distributor-btree-bucket-database-foundations
Add initial B+tree distributor bucket database
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Still uses legacy `BucketDatabase` API, which is not optimized for
bulk loading or updating. Focus for this iteration is functional
correctness rather than API redesign. Legacy DB is still the one
wired in for all production logic.
Unit tests have been expanded to cover discovered edge cases that
were not properly tested for.
Also move distributor bucket DB tests to GTest. Use value-
parameterized test fixture instead of ad-hoc CppUnit approach.
|
|/ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Only do a (potentially expensive) pruning pass over the bucket DB
when the cluster state transition indicates that one or more nodes
are not in the same availability-state as the currently active state.
For instance, if a cluster state change is for a content node going
from Maintenance to Down, no pruning action is required since that
shall already have taken place during the processing of the initial
Down edge (and no buckets shall have been created for it in the meantime).
We do not currently attempt to elide distribution config changes, as
these happen much more rarely than cluster state changes.
|
|
|
|
|
|
|
|
| |
Distributor bucket ownership is assigned on a per superbucket basis,
so all buckets with the same superbucket can use the same decision.
The bucket DB is explicitly ordered in such a way that all buckets belonging
to the same superbucket are ordered after each other, so we need only
maintain O(1) extra state for this.
|
| |
|
|
|
|
|
|
| |
Avoids starving other operations when there is a lot of merge activity
taking place.
For now, 1/2 of the total persistence thread pool may process merges.
|
| |
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/add-read-only-support-during-cluster-state-transitions
Add read-only support during cluster state transitions
|
| | |
|
| | |
|
| |
| |
| |
| |
| | |
Otherwise we'd miss activation commands sent for a cluster state in
which our own node is marked down.
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| | |
Bonus: no more spurious "we have removed buckets" log messages caused
by ownership changes.
Also ensure that we BUSY-bounce operations in `ExternalOperationHandler`
when there is no actual state to send back in a `WrongDistributionReply`.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
| |
| |
| | |
JsonStream will get overloads for the relevant fundamental types.
|
| | |
|
|/ |
|
| |
|
| |
|
| |
|
| |
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/log-bucket-info-before-and-after-on-update-inconsistency
Log before/after bucket info for when update operation inconsistency is discovered
|
| |
| |
| |
| |
| |
| | |
discovered
Makes it more obvious if the inconsistency is likely due to e.g. a checksum collision.
|
|/ |
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/do-not-bruteforce-abort-client-ops-during-orchestrated-down
Fail client ops gracefully when distributor is marked down
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously, clients would only receive `ABORTED` when the distributor was
marked down by orchestration. This would simply cause the client to resend
until either the `StoragePolicy` would discard the cluster state entirely
and retry against a working distributor, or the operations would time out.
Now they will receive a `WrongDistributionReply` that shall immediately update
the `StoragePolicy` to avoid sending to the distributor that has been
marked down.
Also add a separate metric for number of operations aborted by `Bouncer`.
This fixes #8448.
|
|/ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This addresses a regression introduced as part of #8479, which in
turn was intended to serve as a fix for issue #8475. This regression
would stall cluster state convergence when a subset of nodes contained
the fix and another subset did not.
With the workarounds present, nodes gracefully handle the case where
different distribution hashes are expected for the global bucket space.
`BucketManager` will now fall back to comparing the new incoming hash
to that of the legacy derived distribution config if it mismatches.
`PendingClusterState` will try to send a subset of bucket info requests
with legacy hash format for the global bucket space iff there has been
at least 1 rejected request.
All these workarounds will be removed on Vespa 8.
|
|
|
|
| |
with each other, cf. PersistenceTestUtils::setupDisks().
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
Simplify code by emitting wildcards for all groups instead of using
explicit leaf counts. Distribution code will distribute replicas
evenly across all wildcarded groups.
This fixes #8475
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/add-per-bucket-space-data-metrics-on-content-node
Expose data metrics per bucket space on content node
|
| | |
|
| |
| |
| |
| | |
Legacy metrics that cover all bucket spaces remain unchanged.
|
|/ |
|
| |
|
|
|
|
|
| |
- Use dashes instead of underscores
- Explicitly separate client/server metrics in metric path
|
| |
|