summaryrefslogtreecommitdiffstats
path: root/storage
Commit message (Collapse)AuthorAgeFilesLines
* Improve handling of exceptions during distributor startupTor Brede Vekterli2021-01-052-20/+12
| | | | | | | | | | | | Remove call to requestShutdown which could try to use components that weren't properly set up yet. Only used for _maybe_ being able to scream an error to the cluster controller, but this has very limited usefulness in practice. Since exceptions are categorized and logged with backtrace at the root application main level, remove redundant logging. Also enforce component shutdown for network setup exceptions (not sure why this wasn't there to start with).
* Merge pull request #15909 from ↵Henning Baldersheim2021-01-054-15/+16
|\ | | | | | | | | vespa-engine/vekterli/only-fetch-doc-id-when-splitting Only fetch document IDs in bucket when splitting, not whole documents
| * Only fetch document IDs in bucket when splitting, not whole documentsTor Brede Vekterli2021-01-054-15/+16
| | | | | | | | | | | | | | Splitting code only needs to look at the document IDs to figure out the distribution of the target buckets. Remove tracking of document sizes (which did need the whole documents) since it was only used in a warning log message.
* | Delete implicitly deleted move-assignment operator.Tor Egge2021-01-041-1/+1
|/
* Merge pull request #15894 from ↵Henning Baldersheim2021-01-041-1/+1
|\ | | | | | | | | vespa-engine/toregge/use-stricter-version-of-dynamic-cast Use stricter version of dynamic_cast.
| * Use stricter version of dynamic_cast.Tor Egge2021-01-041-1/+1
| |
* | Merge pull request #15878 from vespa-engine/balder/process-queue-when-closedHenning Baldersheim2021-01-044-24/+5
|\ \ | | | | | | - Do not stop persitence thread until it has been interrupted.
| * | - Do not stop persitence thread until it has been interrupted.Henning Baldersheim2021-01-044-24/+5
| | | | | | | | | | | | | | | - fetch messages even if closed to allow draining the Q. - GC unused code.
* | | Merge pull request #15888 from vespa-engine/toregge/avoid-repeating-magic-numberHenning Baldersheim2021-01-041-3/+13
|\ \ \ | |_|/ |/| | Add is_split_group_bucket() utility method.
| * | Add is_split_group_bucket() utility method.Tor Egge2021-01-041-3/+13
| | |
* | | Merge pull request #15877 from vespa-engine/balder/reduce-to-spam-loggingHenning Baldersheim2021-01-041-2/+2
|\ \ \ | |/ / |/| | Reduce loglevel debug -> spam
| * | Reduce loglevel debug -> spamHenning Baldersheim2021-01-041-2/+2
| |/
* | Merge pull request #15881 from ↵Henning Baldersheim2021-01-041-1/+3
|\ \ | | | | | | | | | | | | vespa-engine/toregge/delay-using-std-string-starts-with Don't use method not available on gcc 8 until we drop support for gcc 8.
| * | Don't use method not available on gcc 8 until we drop support for gcc 8.Tor Egge2021-01-041-1/+3
| | |
* | | Merge pull request #15885 from vespa-engine/toregge/add-missing-mutex-includeHenning Baldersheim2021-01-042-0/+2
|\ \ \ | | | | | | | | Include mutex header file when needed.
| * | | Include mutex header file when needed.Tor Egge2021-01-042-0/+2
| |/ /
* / / Use std namespace for dynamic_pointer_cast.Tor Egge2021-01-041-1/+1
|/ /
* | Merge pull request #15879 from ↵Henning Baldersheim2021-01-043-21/+7
|\ \ | | | | | | | | | | | | vespa-engine/balder/gc-remap-queue-after-disk-move GC unused remapQueueAfterDiskMove method.
| * | GC unused remapQueueAfterDiskMove method.Henning Baldersheim2021-01-043-21/+7
| |/
* / GC unused _threadLockCheckInterval and _failDiskOnError membersHenning Baldersheim2021-01-042-6/+0
|/
* Merge pull request #15836 from vespa-engine/arnej/storage-cluster-contextArne H Juul2020-12-1745-129/+182
|\ | | | | Arnej/storage cluster context
| * review follow-upArne Juul2020-12-167-11/+23
| | | | | | | | | | | | | | * add noexcept to new APIs * use "const auto *" * add documentation comments * add copyright
| * assert that cluster name is only set onceArne Juul2020-12-161-0/+2
| |
| * add ClusterContext APIArne Juul2020-12-1645-129/+168
| | | | | | | | | | | | * instead of asking various components for clustername (as reference to string) and then taking the pointer to it, add an API for getting the pointer which has the correct contract documented.
* | Add missing restart annotation.Henning Baldersheim2020-12-161-1/+1
| |
* | Merge pull request #15809 from ↵Tor Brede Vekterli2020-12-1612-127/+195
|\ \ | |/ |/| | | | | vespa-engine/toregge/handle-more-up-states-variants Store more variants of ideal service layer nodes for a given bucket in hash table.
| * Don't store variants of ideal service layer nodes in hash if bucket usedTor Egge2020-12-151-3/+24
| | | | | | | | bits > 33.
| * Store more variants of ideal service layer nodes for a given bucket in hash ↵Tor Egge2020-12-1412-128/+175
| | | | | | | | table.
* | - Wire in the guard to make it evident that we have it when making changes ↵Henning Baldersheim2020-12-152-4/+6
| | | | | | | | | | | | that require it. - Clean up some old members and code not used any more.
* | Gracefully abort outdated reindexing visitorsTor Brede Vekterli2020-12-147-23/+104
|/ | | | | | | | | | Instead of sending `REJECTED` when lock token does not match, send `TEST_AND_SET_CONDITION_FAILED` from distributors. The reindexing visitor will detect this and remap the failure code to `ABORTED` while simultaneously failing the backend visitor. `ABORTED` will be considered transient by the visiting client, allowing it to retry towards another distributor without the whole thing having to fail out.
* Merge pull request #15795 from ↵Tor Brede Vekterli2020-12-1425-63/+268
|\ | | | | | | | | vespa-engine/vekterli/use-random-bucket-lock-passthrough-token-for-reindexing-visitors Only let reindexing puts through locked bucket if their TaS token matches that of the lock
| * Only let reindexing puts through locked bucket if their token matches that ↵Tor Brede Vekterli2020-12-1125-63/+268
| | | | | | | | | | | | | | | | | | | | | | of the lock Avoids race condition edge case where reindexing puts from an outdated visitor may pass through a bucket lock intended for a newly created visitor operation Tokens are 128-bit values derived from a CSPRNG, so uniqueness is for all intents and purposes guaranteed.
* | Merge pull request #15798 from ↵Henning Baldersheim2020-12-1120-98/+100
|\ \ | | | | | | | | | | | | vespa-engine/balder/wire-cluster-name-as-reference-all-the-way - Wire the clustername through as a reference to avoid referencing te…
| * | - Wire the clustername through as a reference to avoid referencing temporaries.Henning Baldersheim2020-12-1120-98/+100
| |/
* | Merge pull request #15797 from ↵Geir Storli2020-12-115-25/+35
|\ \ | |/ |/| | | | | vespa-engine/geirst/optimize-two-phase-update-operation Avoid lookup bucket database entries two times when handling fast pat…
| * Avoid lookup bucket database entries two times when handling fast path ↵Geir Storli2020-12-115-25/+35
| | | | | | | | update operation.
* | Merge pull request #15788 from vespa-engine/toregge/extend-use-of-ownership-hashTor Egge2020-12-117-96/+86
|\ \ | | | | | | Extend use of bucket ownership hash.
| * | Extend use of bucket ownership hash.Tor Egge2020-12-117-96/+86
| |/
* | Merge pull request #15753 from ↵Tor Egge2020-12-118-147/+0
|\ \ | |/ |/| | | | | vespa-engine/toregge/remove-internal-bucket-join-command Remove InternalBucketJoinCommand and InternalBucketJoinReply.
| * Remove InternalBucketJoinCommand and InternalBucketJoinReply.Tor Egge2020-12-098-147/+0
| |
* | Avoid constructing a TraceNode (via ensureRoot) when tracing is disabled.Geir Storli2020-12-101-1/+1
| |
* | Make "events before wakeup" configurable for SharedRpcResources.Geir Storli2020-12-106-6/+13
| |
* | Bounce reindexing visitor with BUSY if merge is pending for bucketTor Brede Vekterli2020-12-095-0/+62
| | | | | | | | | | | | | | | | | | | | | | Since reindexing visitors take a bucket lock when they arrive and wait for pending ops to drain before they start, doing so when there's a pending merge risks starving the bucket for a long time. This is because merges may linger for a long time in the merge throttling queues in the cluster. By not starting such visitors if there is a pending merge, we avoid this edge case. Functionality is already in place to inhibit merges from starting if there's an active bucket lock present.
* | Remove BucketOperationLogger.Tor Egge2020-12-0913-512/+0
|/
* Remove outdated comment.Tor Egge2020-12-091-2/+0
|
* Cleanup remains of disk concept.Tor Egge2020-12-081-29/+24
|
* Remove leftovers from removed disk concept.Tor Egge2020-12-0813-141/+2
|
* Merge pull request #15725 from ↵Tor Brede Vekterli2020-12-0828-57/+199
|\ | | | | | | | | vespa-engine/vekterli/block-ideal-state-ops-towards-locked-buckets Block ideal state operations towards buckets that are locked
| * Block ideal state operations towards buckets that are lockedTor Brede Vekterli2020-12-0728-57/+199
| | | | | | | | | | | | | | Prevents ideal state ops from modifying buckets that are being used in a read-for-write context. Move `OperationSequencer` to main `Distributor` to more easily facilitate sharing of it across components.
* | Annotate config requiring restart, and properly handle the ones that do notHenning Baldersheim2020-12-073-19/+21
|/