Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
* | Merge pull request #15795 from ↵ | Tor Brede Vekterli | 2020-12-14 | 25 | -63/+268 | |
|\ | | | | | | | | | vespa-engine/vekterli/use-random-bucket-lock-passthrough-token-for-reindexing-visitors Only let reindexing puts through locked bucket if their TaS token matches that of the lock | |||||
| * | Only let reindexing puts through locked bucket if their token matches that ↵ | Tor Brede Vekterli | 2020-12-11 | 25 | -63/+268 | |
| | | | | | | | | | | | | | | | | | | | | | | of the lock Avoids race condition edge case where reindexing puts from an outdated visitor may pass through a bucket lock intended for a newly created visitor operation Tokens are 128-bit values derived from a CSPRNG, so uniqueness is for all intents and purposes guaranteed. | |||||
* | | Merge pull request #15798 from ↵ | Henning Baldersheim | 2020-12-11 | 20 | -98/+100 | |
|\ \ | | | | | | | | | | | | | vespa-engine/balder/wire-cluster-name-as-reference-all-the-way - Wire the clustername through as a reference to avoid referencing te… | |||||
| * | | - Wire the clustername through as a reference to avoid referencing temporaries. | Henning Baldersheim | 2020-12-11 | 20 | -98/+100 | |
| |/ | ||||||
* | | Merge pull request #15797 from ↵ | Geir Storli | 2020-12-11 | 5 | -25/+35 | |
|\ \ | |/ |/| | | | | | vespa-engine/geirst/optimize-two-phase-update-operation Avoid lookup bucket database entries two times when handling fast pat… | |||||
| * | Avoid lookup bucket database entries two times when handling fast path ↵ | Geir Storli | 2020-12-11 | 5 | -25/+35 | |
| | | | | | | | | update operation. | |||||
* | | Merge pull request #15788 from vespa-engine/toregge/extend-use-of-ownership-hash | Tor Egge | 2020-12-11 | 7 | -96/+86 | |
|\ \ | | | | | | | Extend use of bucket ownership hash. | |||||
| * | | Extend use of bucket ownership hash. | Tor Egge | 2020-12-11 | 7 | -96/+86 | |
| |/ | ||||||
* | | Merge pull request #15753 from ↵ | Tor Egge | 2020-12-11 | 8 | -147/+0 | |
|\ \ | |/ |/| | | | | | vespa-engine/toregge/remove-internal-bucket-join-command Remove InternalBucketJoinCommand and InternalBucketJoinReply. | |||||
| * | Remove InternalBucketJoinCommand and InternalBucketJoinReply. | Tor Egge | 2020-12-09 | 8 | -147/+0 | |
| | | ||||||
* | | Avoid constructing a TraceNode (via ensureRoot) when tracing is disabled. | Geir Storli | 2020-12-10 | 1 | -1/+1 | |
| | | ||||||
* | | Make "events before wakeup" configurable for SharedRpcResources. | Geir Storli | 2020-12-10 | 6 | -6/+13 | |
| | | ||||||
* | | Bounce reindexing visitor with BUSY if merge is pending for bucket | Tor Brede Vekterli | 2020-12-09 | 5 | -0/+62 | |
| | | | | | | | | | | | | | | | | | | | | | | Since reindexing visitors take a bucket lock when they arrive and wait for pending ops to drain before they start, doing so when there's a pending merge risks starving the bucket for a long time. This is because merges may linger for a long time in the merge throttling queues in the cluster. By not starting such visitors if there is a pending merge, we avoid this edge case. Functionality is already in place to inhibit merges from starting if there's an active bucket lock present. | |||||
* | | Remove BucketOperationLogger. | Tor Egge | 2020-12-09 | 13 | -512/+0 | |
|/ | ||||||
* | Remove outdated comment. | Tor Egge | 2020-12-09 | 1 | -2/+0 | |
| | ||||||
* | Cleanup remains of disk concept. | Tor Egge | 2020-12-08 | 1 | -29/+24 | |
| | ||||||
* | Remove leftovers from removed disk concept. | Tor Egge | 2020-12-08 | 13 | -141/+2 | |
| | ||||||
* | Merge pull request #15725 from ↵ | Tor Brede Vekterli | 2020-12-08 | 28 | -57/+199 | |
|\ | | | | | | | | | vespa-engine/vekterli/block-ideal-state-ops-towards-locked-buckets Block ideal state operations towards buckets that are locked | |||||
| * | Block ideal state operations towards buckets that are locked | Tor Brede Vekterli | 2020-12-07 | 28 | -57/+199 | |
| | | | | | | | | | | | | | | Prevents ideal state ops from modifying buckets that are being used in a read-for-write context. Move `OperationSequencer` to main `Distributor` to more easily facilitate sharing of it across components. | |||||
* | | Annotate config requiring restart, and properly handle the ones that do not | Henning Baldersheim | 2020-12-07 | 3 | -19/+21 | |
|/ | ||||||
* | Add noexcept specifiers to non-throwing constructors and operators. | Tor Egge | 2020-12-05 | 8 | -11/+11 | |
| | ||||||
* | Merge pull request #15664 from ↵ | Geir Storli | 2020-12-05 | 30 | -175/+292 | |
|\ | | | | | | | | | vespa-engine/geirst/decouple-external-operation-handler Decouple external operation handler | |||||
| * | Remove explicit. | Geir Storli | 2020-12-04 | 1 | -2/+2 | |
| | | ||||||
| * | Use noexcept for functions that return a reference. | Geir Storli | 2020-12-04 | 3 | -13/+13 | |
| | | ||||||
| * | Remove dependency between ExternalOperationHandler and Distributor. | Geir Storli | 2020-12-04 | 5 | -16/+29 | |
| | | ||||||
| * | Remove dependency between ExternalOperationHandler and DistributorComponent. | Geir Storli | 2020-12-03 | 29 | -161/+265 | |
| | | ||||||
* | | Use the super bucket key when resolving which RPC target to use to handle a ↵ | Geir Storli | 2020-12-04 | 5 | -10/+68 | |
| | | | | | | | | command. | |||||
* | | Remove unused member variable. | Tor Egge | 2020-12-03 | 1 | -1/+0 | |
|/ | ||||||
* | Merge pull request #15632 from ↵ | Henning Baldersheim | 2020-12-03 | 7 | -32/+159 | |
|\ | | | | | | | | | vespa-engine/toregge/faster-distributor-bucket-db-update Add process_update member function to BucketDatabase. | |||||
| * | Add process_update member function to BucketDatabase. It is used for | Tor Egge | 2020-12-03 | 7 | -32/+159 | |
| | | | | | | | | updating an entry with a single lookup. | |||||
* | | Revert "Use the super bucket key when resolving which RPC target to use to ↵ | Jon Marius Venstad | 2020-12-03 | 5 | -62/+11 | |
| | | | | | | | | ha…" | |||||
* | | Merge pull request #15611 from ↵ | Tor Brede Vekterli | 2020-12-03 | 5 | -42/+153 | |
|\ \ | | | | | | | | | | | | | vespa-engine/vekterli/allow-starting-deferred-tasks-concurrently-with-reads Allow starting deferred tasks concurrently with reads | |||||
| * | | Allow starting deferred tasks concurrently with pending read ops | Tor Brede Vekterli | 2020-12-02 | 3 | -16/+77 | |
| | | | | | | | | | | | | | | | | | | Since read ops aren't blocked by active bucket locks, a constant stream of read ops towards a bucket could otherwise starve deferred tasks from starting. | |||||
| * | | Test edge case where a bucket is removed before starting deferred visitor | Tor Brede Vekterli | 2020-12-02 | 2 | -26/+76 | |
| |/ | ||||||
* | | Make function inline and noexcept. | Geir Storli | 2020-12-03 | 1 | -1/+1 | |
| | | ||||||
* | | Use the super bucket key when resolving which RPC target to use to handle a ↵ | Geir Storli | 2020-12-02 | 5 | -11/+62 | |
|/ | | | | command. | |||||
* | Merge pull request #15594 from ↵ | Henning Baldersheim | 2020-12-02 | 9 | -46/+27 | |
|\ | | | | | | | | | vespa-engine/toregge/remove-distributor-component-trampoline-member-function Remove DistributorComponent trampoline member function. | |||||
| * | Remove getIdealNodes() trampoline member function from DistributorComponent. | Tor Egge | 2020-12-02 | 4 | -13/+5 | |
| | | ||||||
| * | Use mutable hash maps in DistributorBucketSpace. | Tor Egge | 2020-12-02 | 4 | -8/+8 | |
| | | ||||||
| * | Remove DistributorComponent trampoline member function. | Tor Egge | 2020-12-02 | 7 | -29/+18 | |
| | | ||||||
* | | Merge pull request #15572 from ↵ | Tor Brede Vekterli | 2020-12-02 | 26 | -146/+941 | |
|\ \ | |/ |/| | | | | | vespa-engine/vekterli/initial-backend-reindexing-visitor-support Initial support for backend reindexing visitor functionality | |||||
| * | Add top-level comments | Tor Brede Vekterli | 2020-12-02 | 2 | -3/+21 | |
| | | ||||||
| * | Simplify code interacting with blocked sequencing handles | Tor Brede Vekterli | 2020-12-02 | 3 | -9/+18 | |
| | | ||||||
| * | Initial support for backend reindexing visitor functionality | Tor Brede Vekterli | 2020-12-01 | 26 | -146/+914 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduces the concept of a read-for-write visitor operation which blocks all mutating operations from starting for a bucket being visited. This read-for-write mode is used if (and only if) the visitor library being specified by the client is "reindexingvisitor". Since read-for-write visitors cannot race with concurrent write operations, starting such visitors are deferred until no further mutations are pending. Also adds a basic reindexingvisitor implementation to the content node which sends all documents as Puts containing a special TaS token that will let the operation through even if a bucket is locked. This token is cleared by the distributor before it is passed on to the content nodes. Note: this feature is not yet production ready. For now the following caveats apply: * Mutating vs non-mutating pending ops to a bucket are not tracked separately, so it’s possible to starve a reindexing visitor by sending constant pending read load, as read load is not blocked by the operation sequencer. * Ideal state operations towards locked buckets are not blocked, so it's possible for e.g. a split to be sent for a bucket that is being visited. | |||||
* | | Remove BucketDBUpdater trampoline member function. | Tor Egge | 2020-12-02 | 9 | -25/+9 | |
| | | ||||||
* | | Trim down DistributorComponent member functions. | Tor Egge | 2020-12-02 | 7 | -59/+25 | |
| | | ||||||
* | | Move config setters directly onto main TransportConfig object | Henning Baldersheim | 2020-12-01 | 1 | -1/+1 | |
| | | ||||||
* | | provide the config in the constructor only. | Henning Baldersheim | 2020-12-01 | 1 | -4/+2 | |
| | | ||||||
* | | Merge pull request #15558 from vespa-engine/balder/fnet-cleanup | Henning Baldersheim | 2020-12-01 | 1 | -1/+3 | |
|\ \ | |/ |/| | Misc cleanup of includes and code health | |||||
| * | Misc cleanup of includes and code health | Henning Baldersheim | 2020-12-01 | 1 | -1/+3 | |
| | |