Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Bounce reindexing visitor with BUSY if merge is pending for bucket | Tor Brede Vekterli | 2020-12-09 | 5 | -0/+62 |
| | | | | | | | | | | | Since reindexing visitors take a bucket lock when they arrive and wait for pending ops to drain before they start, doing so when there's a pending merge risks starving the bucket for a long time. This is because merges may linger for a long time in the merge throttling queues in the cluster. By not starting such visitors if there is a pending merge, we avoid this edge case. Functionality is already in place to inhibit merges from starting if there's an active bucket lock present. | ||||
* | Remove BucketOperationLogger. | Tor Egge | 2020-12-09 | 13 | -512/+0 |
| | |||||
* | Remove outdated comment. | Tor Egge | 2020-12-09 | 1 | -2/+0 |
| | |||||
* | Cleanup remains of disk concept. | Tor Egge | 2020-12-08 | 1 | -29/+24 |
| | |||||
* | Remove leftovers from removed disk concept. | Tor Egge | 2020-12-08 | 13 | -141/+2 |
| | |||||
* | Merge pull request #15725 from ↵ | Tor Brede Vekterli | 2020-12-08 | 28 | -57/+199 |
|\ | | | | | | | | | vespa-engine/vekterli/block-ideal-state-ops-towards-locked-buckets Block ideal state operations towards buckets that are locked | ||||
| * | Block ideal state operations towards buckets that are locked | Tor Brede Vekterli | 2020-12-07 | 28 | -57/+199 |
| | | | | | | | | | | | | | | Prevents ideal state ops from modifying buckets that are being used in a read-for-write context. Move `OperationSequencer` to main `Distributor` to more easily facilitate sharing of it across components. | ||||
* | | Annotate config requiring restart, and properly handle the ones that do not | Henning Baldersheim | 2020-12-07 | 3 | -19/+21 |
|/ | |||||
* | Add noexcept specifiers to non-throwing constructors and operators. | Tor Egge | 2020-12-05 | 8 | -11/+11 |
| | |||||
* | Merge pull request #15664 from ↵ | Geir Storli | 2020-12-05 | 30 | -175/+292 |
|\ | | | | | | | | | vespa-engine/geirst/decouple-external-operation-handler Decouple external operation handler | ||||
| * | Remove explicit. | Geir Storli | 2020-12-04 | 1 | -2/+2 |
| | | |||||
| * | Use noexcept for functions that return a reference. | Geir Storli | 2020-12-04 | 3 | -13/+13 |
| | | |||||
| * | Remove dependency between ExternalOperationHandler and Distributor. | Geir Storli | 2020-12-04 | 5 | -16/+29 |
| | | |||||
| * | Remove dependency between ExternalOperationHandler and DistributorComponent. | Geir Storli | 2020-12-03 | 29 | -161/+265 |
| | | |||||
* | | Use the super bucket key when resolving which RPC target to use to handle a ↵ | Geir Storli | 2020-12-04 | 5 | -10/+68 |
| | | | | | | | | command. | ||||
* | | Remove unused member variable. | Tor Egge | 2020-12-03 | 1 | -1/+0 |
|/ | |||||
* | Merge pull request #15632 from ↵ | Henning Baldersheim | 2020-12-03 | 7 | -32/+159 |
|\ | | | | | | | | | vespa-engine/toregge/faster-distributor-bucket-db-update Add process_update member function to BucketDatabase. | ||||
| * | Add process_update member function to BucketDatabase. It is used for | Tor Egge | 2020-12-03 | 7 | -32/+159 |
| | | | | | | | | updating an entry with a single lookup. | ||||
* | | Revert "Use the super bucket key when resolving which RPC target to use to ↵ | Jon Marius Venstad | 2020-12-03 | 5 | -62/+11 |
| | | | | | | | | ha…" | ||||
* | | Merge pull request #15611 from ↵ | Tor Brede Vekterli | 2020-12-03 | 5 | -42/+153 |
|\ \ | | | | | | | | | | | | | vespa-engine/vekterli/allow-starting-deferred-tasks-concurrently-with-reads Allow starting deferred tasks concurrently with reads | ||||
| * | | Allow starting deferred tasks concurrently with pending read ops | Tor Brede Vekterli | 2020-12-02 | 3 | -16/+77 |
| | | | | | | | | | | | | | | | | | | Since read ops aren't blocked by active bucket locks, a constant stream of read ops towards a bucket could otherwise starve deferred tasks from starting. | ||||
| * | | Test edge case where a bucket is removed before starting deferred visitor | Tor Brede Vekterli | 2020-12-02 | 2 | -26/+76 |
| |/ | |||||
* | | Make function inline and noexcept. | Geir Storli | 2020-12-03 | 1 | -1/+1 |
| | | |||||
* | | Use the super bucket key when resolving which RPC target to use to handle a ↵ | Geir Storli | 2020-12-02 | 5 | -11/+62 |
|/ | | | | command. | ||||
* | Merge pull request #15594 from ↵ | Henning Baldersheim | 2020-12-02 | 9 | -46/+27 |
|\ | | | | | | | | | vespa-engine/toregge/remove-distributor-component-trampoline-member-function Remove DistributorComponent trampoline member function. | ||||
| * | Remove getIdealNodes() trampoline member function from DistributorComponent. | Tor Egge | 2020-12-02 | 4 | -13/+5 |
| | | |||||
| * | Use mutable hash maps in DistributorBucketSpace. | Tor Egge | 2020-12-02 | 4 | -8/+8 |
| | | |||||
| * | Remove DistributorComponent trampoline member function. | Tor Egge | 2020-12-02 | 7 | -29/+18 |
| | | |||||
* | | Merge pull request #15572 from ↵ | Tor Brede Vekterli | 2020-12-02 | 26 | -146/+941 |
|\ \ | |/ |/| | | | | | vespa-engine/vekterli/initial-backend-reindexing-visitor-support Initial support for backend reindexing visitor functionality | ||||
| * | Add top-level comments | Tor Brede Vekterli | 2020-12-02 | 2 | -3/+21 |
| | | |||||
| * | Simplify code interacting with blocked sequencing handles | Tor Brede Vekterli | 2020-12-02 | 3 | -9/+18 |
| | | |||||
| * | Initial support for backend reindexing visitor functionality | Tor Brede Vekterli | 2020-12-01 | 26 | -146/+914 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduces the concept of a read-for-write visitor operation which blocks all mutating operations from starting for a bucket being visited. This read-for-write mode is used if (and only if) the visitor library being specified by the client is "reindexingvisitor". Since read-for-write visitors cannot race with concurrent write operations, starting such visitors are deferred until no further mutations are pending. Also adds a basic reindexingvisitor implementation to the content node which sends all documents as Puts containing a special TaS token that will let the operation through even if a bucket is locked. This token is cleared by the distributor before it is passed on to the content nodes. Note: this feature is not yet production ready. For now the following caveats apply: * Mutating vs non-mutating pending ops to a bucket are not tracked separately, so it’s possible to starve a reindexing visitor by sending constant pending read load, as read load is not blocked by the operation sequencer. * Ideal state operations towards locked buckets are not blocked, so it's possible for e.g. a split to be sent for a bucket that is being visited. | ||||
* | | Remove BucketDBUpdater trampoline member function. | Tor Egge | 2020-12-02 | 9 | -25/+9 |
| | | |||||
* | | Trim down DistributorComponent member functions. | Tor Egge | 2020-12-02 | 7 | -59/+25 |
| | | |||||
* | | Move config setters directly onto main TransportConfig object | Henning Baldersheim | 2020-12-01 | 1 | -1/+1 |
| | | |||||
* | | provide the config in the constructor only. | Henning Baldersheim | 2020-12-01 | 1 | -4/+2 |
| | | |||||
* | | Merge pull request #15558 from vespa-engine/balder/fnet-cleanup | Henning Baldersheim | 2020-12-01 | 1 | -1/+3 |
|\ \ | |/ |/| | Misc cleanup of includes and code health | ||||
| * | Misc cleanup of includes and code health | Henning Baldersheim | 2020-12-01 | 1 | -1/+3 |
| | | |||||
* | | Revert "Revert "Move bucket ownership methods to DistributorBucketSpace."" | Tor Egge | 2020-12-01 | 12 | -87/+471 |
|/ | |||||
* | Merge pull request #15548 from vespa-engine/balder/wakeup-every-time | Henning Baldersheim | 2020-11-30 | 1 | -1/+1 |
|\ | | | | | Go back to default | ||||
| * | Go back to default | Henning Baldersheim | 2020-11-30 | 1 | -1/+1 |
| | | |||||
* | | Revert "Move bucket ownership methods to DistributorBucketSpace." | Tor Egge | 2020-11-30 | 12 | -471/+87 |
|/ | |||||
* | Merge pull request #15541 from ↵ | Geir Storli | 2020-11-30 | 34 | -109/+321 |
|\ | | | | | | | | | vespa-engine/geirst/remove-coupling-between-external-operations-and-distributor-component Remove coupling between external operations and distributor component | ||||
| * | Add operation context interface to remove coupling between external ↵ | Geir Storli | 2020-11-30 | 30 | -81/+213 |
| | | | | | | | | operations and DistributorComponent. | ||||
| * | Add interface to parse a document selection to reduce coupling between ↵ | Geir Storli | 2020-11-30 | 8 | -13/+50 |
| | | | | | | | | external operations and DistributorComponent. | ||||
| * | Add node context interface to reduce coupling between external operations ↵ | Geir Storli | 2020-11-30 | 14 | -27/+70 |
| | | | | | | | | and DistributorComponent. | ||||
* | | Wakeup every 10 events | Henning Baldersheim | 2020-11-30 | 1 | -1/+1 |
|/ | |||||
* | Merge pull request #15506 from ↵ | Tor Brede Vekterli | 2020-11-30 | 12 | -87/+471 |
|\ | | | | | | | | | vespa-engine/toregge/move-bucket-ownership-methods-to-distributor-bucket-space Move bucket ownership methods to DistributorBucketSpace. | ||||
| * | Add comments about ideal nodes changing when bucket used bits > 33. | Tor Egge | 2020-11-30 | 1 | -0/+6 |
| | | |||||
| * | Add explicit keyword. | Tor Egge | 2020-11-30 | 1 | -1/+1 |
| | |