summaryrefslogtreecommitdiffstats
path: root/storage
Commit message (Collapse)AuthorAgeFilesLines
* Simplify Distributor class to remove now unused functionalityTor Brede Vekterli2021-03-234-258/+25
| | | | | In particular, remove `DistributorStripeInterface` inheritance and subclassed `DistributorComponent` usage.
* Merge pull request #17113 from vespa-engine/geirst/distributor-stripe-refactor-2Tor Brede Vekterli2021-03-2315-151/+111
|\ | | | | Distributor stripe refactor 2
| * Stop exposing DistributorStripeComponent from IdealStateManager.Geir Storli2021-03-2314-102/+109
| |
| * Remove unused functions.Geir Storli2021-03-232-41/+0
| |
| * Remove functions from DistributorStripeComponent that are part of ↵Geir Storli2021-03-222-8/+2
| | | | | | | | DistributorOperationContext interface.
* | Fix forward declarations.Tor Egge2021-03-221-2/+2
|/
* Remove functions from DistributorStripeComponent that are part of ↵Geir Storli2021-03-2217-134/+116
| | | | DistributorOperationContext interface.
* Rename functions that are part of DistributorNodeContext interface.Geir Storli2021-03-222-12/+14
|
* Rename DistributorComponent -> DistributorStripeComponent.Geir Storli2021-03-2217-70/+71
|
* Rename DistributorInterface -> DistributorStripeInterface.Geir Storli2021-03-2211-24/+30
|
* First steps of splitting distributor logic up into disjoint stripesTor Brede Vekterli2021-03-1931-855/+1590
| | | | For now only a single stripe may be used.
* Abandon gcc 8 support.Tor Egge2021-03-181-3/+1
|
* Stop exposing DistributorComponent from BucketDBUpdater.Geir Storli2021-03-173-8/+9
|
* Reduce usage of DistributorComponent by using DistributorInterface instead.Geir Storli2021-03-172-10/+12
|
* Reduce usage of DistributorComponent by using DistributorOperationContext ↵Geir Storli2021-03-174-34/+39
| | | | instead.
* Reduce usage of DistributorComponent by using DistributorNodeContext instead.Geir Storli2021-03-174-15/+20
|
* Instantiate all member variables explicitly.Geir Storli2021-03-174-2/+10
|
* Use make_shared and auto.Geir Storli2021-03-173-16/+13
|
* Guard against processing bucket requests with inconsistent internal state ↵Tor Brede Vekterli2021-03-033-12/+50
| | | | | | | | | | version There's a tiny window of time between when the bucket manager observes a new state version and when the state version actually is visible in the rest of the process. We must ensure that we don't end up processing requests when these two differ, or we might erroneously process requests for version X using a state only valid for version Y < X.
* Remove notion of node-specific reliability from C++ distribution codeTor Brede Vekterli2021-03-021-10/+2
| | | | | I have never seen this in use anywhere, and can find no code that ever sets it. Bonus is that node candidate trimming can be vastly simplified.
* Merge pull request #16658 from ↵Tor Brede Vekterli2021-02-246-97/+102
|\ | | | | | | | | vespa-engine/vekterli/prefer-ready-replicas-with-more-documents Prefer activating ready replicas with more documents
| * Prefer activating ready replicas with more documentsTor Brede Vekterli2021-02-226-97/+102
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This goes contrary to the old behavior where we activated the replica the distributor thinks is "probably" most up to date, but where the information needed to make such a decision disappears upon bucket ownership transfer. In turn, this can cause highly sub-optimal replica activation if the distributors makes the wrong decision. Remove using the "trusted" concept entirely from the bucket activation heuristic in favor of a much more deterministic one based on replica readiness, document count and ideal state order. Also fixes an incidentally discovered old bug where an edge case in the cluster state's handling of node counts had the potential of not clearing pending maintenance operations if the newly unavailable node had a distribution key equal to `node count - 1`.
* | - Change error handling so that both synchonous and asynchronous errors can ↵Henning Baldersheim2021-02-236-33/+39
| | | | | | | | | | | | | | | | be reported back from bucket executor. - Treat remapping as an error. - For lidspace compaction job iterator is reset and will be recreated on next invocation. - For bucketmove th ebucket is rechecked and either discarded or restarted.
* | Remap storageHenning Baldersheim2021-02-231-0/+5
|/
* string.h does not need alloc.hHenning Baldersheim2021-02-213-22/+10
|
* Use vespalib::rmdir()Tor Egge2021-02-211-1/+2
|
* Remove unused class definition.Henning Baldersheim2021-02-191-4/+0
|
* - Reduce visibility of ClusterState and Distribution.Henning Baldersheim2021-02-1966-77/+133
|
* Fix typoTor Brede Vekterli2021-02-181-1/+1
|
* Inhibit activation of replicas out of sync with a replica majorityTor Brede Vekterli2021-02-1712-21/+249
| | | | | | | | | | Adds a configurable max number of groups (default 0) whose replica activation is inhibited if the replica's bucket info is out of sync with a majority of other replicas. Intended to be used for the case where a group comes back up after transient unavailability and where the nodes are out of sync and should preferably not be activated until post-merging.
* Log count of lost documents when all bucket replicas are goneTor Brede Vekterli2021-02-162-2/+6
|
* revert changes inside stringsArne Juul2021-02-151-1/+1
|
* use size literals in storageArne Juul2021-02-158-13/+20
|
* Forward declare MemoryAllocator in vespalib/util/alloc.h.Tor Egge2021-02-131-0/+1
|
* Avoid starving global merges with default space bucket deletionsTor Brede Vekterli2021-02-115-10/+15
| | | | | | | Introduces a new top-most internal maintenance priority level and changes bucket activation to use this level (still considered the most important maintenance operation). Have global merge operations use the old `VERY_HIGH` level. No other operations share this level.
* Make the noise level used when deciding whether to report resource usage ↵Geir Storli2021-02-044-25/+59
| | | | configurable.
* Remove the sync interface from Bucket Executor. Due to the nature of ↵Henning Baldersheim2021-02-022-9/+0
| | | | | | requiring a bucket lock it is very hard to get sync to work in a safe way. Instead the users must do their own accounting as they know their own threading model.
* Merge pull request #16355 from vespa-engine/balder/shutdown-in-closeHenning Baldersheim2021-02-021-1/+5
|\ | | | | The StateManager is destructed before the FileStorManager leading to …
| * The StateManager is destructed before the FileStorManager leading to ↵Henning Baldersheim2021-02-021-1/+5
| | | | | | | | possibly accesing invalid memory. Disconnecting upcalls must happen during close.
* | Ensure that task not executed are returned.Henning Baldersheim2021-02-024-8/+16
|/
* Merge pull request #16348 from vespa-engine/balder/refactor-bucketmove-jobHenning Baldersheim2021-02-021-1/+1
|\ | | | | Factor out the scan iterator from the BucketMoveJob.
| * Factor out the scan iterator from the BucketMoveJob.Henning Baldersheim2021-02-021-1/+1
| |
* | Add noexcept and some other minor code health.Henning Baldersheim2021-02-024-40/+26
|/
* Revert "Properly track execution of BucketTasks and provide sync() and ↵Henning Baldersheim2021-02-0210-161/+54
| | | | order… "
* No need for a guardHenning Baldersheim2021-02-021-3/+1
|
* Properly track execution of BucketTasks and provide sync() and orderly shutdown.Henning Baldersheim2021-02-0210-54/+163
|
* Revert "Implement BucketExecutor::sync."Henning Baldersheim2021-02-0210-150/+54
|
* Revert "Balder/disconnect upcalls early"Henning Baldersheim2021-02-022-26/+6
|
* No need this guard.Henning Baldersheim2021-02-011-1/+0
|
* Disconnect upcalls early during shutdown in order to let task drain out.Henning Baldersheim2021-02-012-6/+27
|