| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bucket DB updating happened unconditionally anyway; this was
only used for failing operations in an overly pessimistic way.
Removing this lookup has two benefits:
- Less CPU spent in DB
- Less impact expected during feeding during node state
transitions since fewer operations will have to be needlessly
retried by the client.
Rationale: an operation towards a given bucket completes (i.e.
is ACKed by all its replica nodes) at time t and the bucket is
removed from the DB at time T. There is no fundamental change
in correctness or behavior from the client's perspective if
the order of events is tT or Tt. Both are equally valid, as
the state transition edge happens independently of any reply
processing.
|
|
|
|
|
| |
B-tree/datastore stats can be expensive to sample, so don't do
this after every full DB iteration. For now, wait at least 30s.
|
|\
| |
| |
| | |
balder/move-sequenced-task-executors-to-staging_vespalib
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
|/ |
|
| |
|
| |
|
| |
|
|\
| |
| |
| |
| | |
vespa-engine/balder/control-net-and-worker-threads-independent
Control mbus worker threads and network threads separately.
|
| | |
|
| | |
|
|\ \
| |/
|/|
| |
| | |
vespa-engine/vekterli/add-distributor-bucket-db-memory-usage-metrics
Add distributor bucket db memory usage metrics
|
| | |
|
| | |
|
| | |
|
| | |
|
|/ |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
Use optimize_for to select executor type instead.
|
| |
|
| |
|
| |
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/add-metric-coverage-of-new-update-phases
Track metrics for new inconsistent update phases
|
| |
| |
| |
| |
| |
| | |
Reuses the old update-get metric for the single full Get sent
after the initial metadata-only phase. Adds new metric set for
the initial metadata Gets.
|
|/ |
|
| |
|
|
|
|
|
|
|
| |
Just inhibiting MergeCommand itself does not inhibit all message
variants that may cause significant read or write load for any
given bucket. This should lessen the competition between merge
backend activity and client operations.
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/add-cheap-metadata-fetch-phase-for-updates
Add initial metadata-only phase to inconsistent update handling
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If bucket replicas are inconsistent, the common case is that only
a small subset of documents contained in the buckets are actually
mutually out of sync. The added metadata phase optimizes for
such a case by initially sending Get requests to all divergent
replicas that only ask for the timestamps (and no fields). This
is a very cheap and fast operation. If all returned timestamps
are in sync, the update can be restarted in the fast path.
Otherwise, a full Get will _only_ be sent to the newest replica,
and its result will be used for performing the update on the
distributor itself, before pushing out the result as Puts.
This is in contrast to today's behavior where full Gets are
sent to all replicas. For users with large documents this
can be very expensive.
In addition, the metadata Get operations are sent with weak
internal read consistency (as they do not need to read any
previously written, possibly in-flight fields). This lets them
bypass the main commit queue entirely.
|
|/
|
|
|
|
|
|
|
|
|
|
| |
Legacy locking code had a hidden assumption that each disk
had only a single queue associated with it and locking requests
could therefore be deduped at the disk level. Since we now
only have a single logical disk with a number of queues striped
over it, this would introduce race conditions when splits/joins
would cross stripe boundaries for their target/source buckets.
Use a composite disk/stripe multi lock key to ensure we only
dedupe locking requests at the stripe-level.
|
|
|
|
|
|
|
|
|
|
|
| |
With a sufficient, even thread count this will ensure that no
stripes end up completely blocked on processing merges, which can
starve client operations. Having a global limit means that it was
possible for stripes to completely fill up with merges.
As an added bonus, moving the limit tracking to individual stripes
means that we no longer have to track this as an atomic, since all
access already happens under the Stripe lock.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New distributor metric available as:
```
vds.idealstate.garbage_collection.documents_removed
```
Add documents removed statistics to `RemoveLocation` responses,
which is what GC is currently built around. Could technically have
been implemented as a diff of before/after BucketInfo, but GC is
very low priority so many other mutating ops may have changed the
bucket document set in the time span between sending the GC ops
and receiving the replies.
This relates to issue #12139
|
| |
|
| |
|