summaryrefslogtreecommitdiffstats
path: root/storage
Commit message (Collapse)AuthorAgeFilesLines
* Bounce reindexing visitor with BUSY if merge is pending for bucketTor Brede Vekterli2020-12-095-0/+62
| | | | | | | | | | | Since reindexing visitors take a bucket lock when they arrive and wait for pending ops to drain before they start, doing so when there's a pending merge risks starving the bucket for a long time. This is because merges may linger for a long time in the merge throttling queues in the cluster. By not starting such visitors if there is a pending merge, we avoid this edge case. Functionality is already in place to inhibit merges from starting if there's an active bucket lock present.
* Remove BucketOperationLogger.Tor Egge2020-12-0913-512/+0
|
* Remove outdated comment.Tor Egge2020-12-091-2/+0
|
* Cleanup remains of disk concept.Tor Egge2020-12-081-29/+24
|
* Remove leftovers from removed disk concept.Tor Egge2020-12-0813-141/+2
|
* Merge pull request #15725 from ↵Tor Brede Vekterli2020-12-0828-57/+199
|\ | | | | | | | | vespa-engine/vekterli/block-ideal-state-ops-towards-locked-buckets Block ideal state operations towards buckets that are locked
| * Block ideal state operations towards buckets that are lockedTor Brede Vekterli2020-12-0728-57/+199
| | | | | | | | | | | | | | Prevents ideal state ops from modifying buckets that are being used in a read-for-write context. Move `OperationSequencer` to main `Distributor` to more easily facilitate sharing of it across components.
* | Annotate config requiring restart, and properly handle the ones that do notHenning Baldersheim2020-12-073-19/+21
|/
* Add noexcept specifiers to non-throwing constructors and operators.Tor Egge2020-12-058-11/+11
|
* Merge pull request #15664 from ↵Geir Storli2020-12-0530-175/+292
|\ | | | | | | | | vespa-engine/geirst/decouple-external-operation-handler Decouple external operation handler
| * Remove explicit.Geir Storli2020-12-041-2/+2
| |
| * Use noexcept for functions that return a reference.Geir Storli2020-12-043-13/+13
| |
| * Remove dependency between ExternalOperationHandler and Distributor.Geir Storli2020-12-045-16/+29
| |
| * Remove dependency between ExternalOperationHandler and DistributorComponent.Geir Storli2020-12-0329-161/+265
| |
* | Use the super bucket key when resolving which RPC target to use to handle a ↵Geir Storli2020-12-045-10/+68
| | | | | | | | command.
* | Remove unused member variable.Tor Egge2020-12-031-1/+0
|/
* Merge pull request #15632 from ↵Henning Baldersheim2020-12-037-32/+159
|\ | | | | | | | | vespa-engine/toregge/faster-distributor-bucket-db-update Add process_update member function to BucketDatabase.
| * Add process_update member function to BucketDatabase. It is used forTor Egge2020-12-037-32/+159
| | | | | | | | updating an entry with a single lookup.
* | Revert "Use the super bucket key when resolving which RPC target to use to ↵Jon Marius Venstad2020-12-035-62/+11
| | | | | | | | ha…"
* | Merge pull request #15611 from ↵Tor Brede Vekterli2020-12-035-42/+153
|\ \ | | | | | | | | | | | | vespa-engine/vekterli/allow-starting-deferred-tasks-concurrently-with-reads Allow starting deferred tasks concurrently with reads
| * | Allow starting deferred tasks concurrently with pending read opsTor Brede Vekterli2020-12-023-16/+77
| | | | | | | | | | | | | | | | | | Since read ops aren't blocked by active bucket locks, a constant stream of read ops towards a bucket could otherwise starve deferred tasks from starting.
| * | Test edge case where a bucket is removed before starting deferred visitorTor Brede Vekterli2020-12-022-26/+76
| |/
* | Make function inline and noexcept.Geir Storli2020-12-031-1/+1
| |
* | Use the super bucket key when resolving which RPC target to use to handle a ↵Geir Storli2020-12-025-11/+62
|/ | | | command.
* Merge pull request #15594 from ↵Henning Baldersheim2020-12-029-46/+27
|\ | | | | | | | | vespa-engine/toregge/remove-distributor-component-trampoline-member-function Remove DistributorComponent trampoline member function.
| * Remove getIdealNodes() trampoline member function from DistributorComponent.Tor Egge2020-12-024-13/+5
| |
| * Use mutable hash maps in DistributorBucketSpace.Tor Egge2020-12-024-8/+8
| |
| * Remove DistributorComponent trampoline member function.Tor Egge2020-12-027-29/+18
| |
* | Merge pull request #15572 from ↵Tor Brede Vekterli2020-12-0226-146/+941
|\ \ | |/ |/| | | | | vespa-engine/vekterli/initial-backend-reindexing-visitor-support Initial support for backend reindexing visitor functionality
| * Add top-level commentsTor Brede Vekterli2020-12-022-3/+21
| |
| * Simplify code interacting with blocked sequencing handlesTor Brede Vekterli2020-12-023-9/+18
| |
| * Initial support for backend reindexing visitor functionalityTor Brede Vekterli2020-12-0126-146/+914
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduces the concept of a read-for-write visitor operation which blocks all mutating operations from starting for a bucket being visited. This read-for-write mode is used if (and only if) the visitor library being specified by the client is "reindexingvisitor". Since read-for-write visitors cannot race with concurrent write operations, starting such visitors are deferred until no further mutations are pending. Also adds a basic reindexingvisitor implementation to the content node which sends all documents as Puts containing a special TaS token that will let the operation through even if a bucket is locked. This token is cleared by the distributor before it is passed on to the content nodes. Note: this feature is not yet production ready. For now the following caveats apply: * Mutating vs non-mutating pending ops to a bucket are not tracked separately, so it’s possible to starve a reindexing visitor by sending constant pending read load, as read load is not blocked by the operation sequencer. * Ideal state operations towards locked buckets are not blocked, so it's possible for e.g. a split to be sent for a bucket that is being visited.
* | Remove BucketDBUpdater trampoline member function.Tor Egge2020-12-029-25/+9
| |
* | Trim down DistributorComponent member functions.Tor Egge2020-12-027-59/+25
| |
* | Move config setters directly onto main TransportConfig objectHenning Baldersheim2020-12-011-1/+1
| |
* | provide the config in the constructor only.Henning Baldersheim2020-12-011-4/+2
| |
* | Merge pull request #15558 from vespa-engine/balder/fnet-cleanupHenning Baldersheim2020-12-011-1/+3
|\ \ | |/ |/| Misc cleanup of includes and code health
| * Misc cleanup of includes and code healthHenning Baldersheim2020-12-011-1/+3
| |
* | Revert "Revert "Move bucket ownership methods to DistributorBucketSpace.""Tor Egge2020-12-0112-87/+471
|/
* Merge pull request #15548 from vespa-engine/balder/wakeup-every-timeHenning Baldersheim2020-11-301-1/+1
|\ | | | | Go back to default
| * Go back to defaultHenning Baldersheim2020-11-301-1/+1
| |
* | Revert "Move bucket ownership methods to DistributorBucketSpace."Tor Egge2020-11-3012-471/+87
|/
* Merge pull request #15541 from ↵Geir Storli2020-11-3034-109/+321
|\ | | | | | | | | vespa-engine/geirst/remove-coupling-between-external-operations-and-distributor-component Remove coupling between external operations and distributor component
| * Add operation context interface to remove coupling between external ↵Geir Storli2020-11-3030-81/+213
| | | | | | | | operations and DistributorComponent.
| * Add interface to parse a document selection to reduce coupling between ↵Geir Storli2020-11-308-13/+50
| | | | | | | | external operations and DistributorComponent.
| * Add node context interface to reduce coupling between external operations ↵Geir Storli2020-11-3014-27/+70
| | | | | | | | and DistributorComponent.
* | Wakeup every 10 eventsHenning Baldersheim2020-11-301-1/+1
|/
* Merge pull request #15506 from ↵Tor Brede Vekterli2020-11-3012-87/+471
|\ | | | | | | | | vespa-engine/toregge/move-bucket-ownership-methods-to-distributor-bucket-space Move bucket ownership methods to DistributorBucketSpace.
| * Add comments about ideal nodes changing when bucket used bits > 33.Tor Egge2020-11-301-0/+6
| |
| * Add explicit keyword.Tor Egge2020-11-301-1/+1
| |