summaryrefslogtreecommitdiffstats
path: root/storage/src
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #22105 from vespa-engine/arnej/just-new-config-filesHenning Baldersheim2022-04-122-216/+259
|\ | | | | switch to new-style config
| * switch to new-style configArne H Juul2022-04-122-216/+259
| |
* | remove FastOS_ApplicationHåvard Pettersen2022-04-081-17/+16
|/ | | | | | | | fixup (per application): - maybe ignore SIGPIPE - wire argc/argv untangle Vespa Test Framework strip down deprecated TestApp
* Add missing include statements. Check for atomic wait support.Tor Egge2022-04-052-0/+2
|
* GC unused Context parameterHenning Baldersheim2022-03-3118-241/+161
|
* Minor simplifications after PR feedback.Henning Baldersheim2022-03-291-1/+1
|
* Remove copy constructors.Henning Baldersheim2022-03-284-71/+36
|
* Use both lvalue and rvalue specifier to avoid explicit std::move()Henning Baldersheim2022-03-284-12/+9
|
* Avoid the need for clone by using unique_ptr.Henning Baldersheim2022-03-284-8/+6
|
* Avoid need to copy/clone FieldUpdateHenning Baldersheim2022-03-274-14/+6
|
* Use vespalib::string in TestAndSetConditionTor Brede Vekterli2022-03-241-1/+1
| | | | | Avoids some implicit conversions. Add `starts_with` to `vespalib::string` and `vespalib::stringref` to allow drop-in replacement for Document API code.
* Revert "Use vespalib::string in TestAndSetCondition"Tor Brede Vekterli2022-03-241-1/+1
|
* Use vespalib::string in TestAndSetConditionTor Brede Vekterli2022-03-241-1/+1
| | | | | Avoids some implicit conversions. Add `starts_with` to `vespalib::string` and `vespalib::stringref` to allow drop-in replacement for Document API code.
* Clean up and refactor visitor management codeTor Brede Vekterli2022-03-185-118/+101
|
* Fix visitor manager test TSan mutex inversion warning and refactor deadline ↵Tor Brede Vekterli2022-03-186-60/+76
| | | | | | | | | | | | | handling Defer starting main message dispatch thread until test has enqueued a message that will be immediately timed out. Avoids having to depend on taking an internal mutex in the test to prevent racing with queue handoffs. Taking said mutex triggered a mutex order inversion warning in TSan. Also refactor visitor queue deadline handling by moving to a strongly typed time point. This removes some manual unit scaling arithmetic that did not appear to be entirely correct in the first place.
* Add comment with rationale for using atomic value updates instead of tree ↵Tor Brede Vekterli2022-03-141-0/+10
| | | | thawing/freezing
* Make B-tree bucket database values atomic to ensure well-defined accessTor Brede Vekterli2022-03-113-24/+98
| | | | | | | | | | | | | Existing implementation already used explicit acquire/release fences to ensure visibility from writer to concurrent readers, but the values written/read were not of an atomic type and thus _technically_ considered a data race. This commit adds an AtomicValueWrapper wrapper to vespalib which looks and acts much like the existing AtomicEntryRef, but for primitive types that are not related to EntryRefs. The bucket DB B-tree primitive u64 value type is replaced with an atomic wrapper and explicit memory fences are replaced with release stores and acquire loads on the atomic values themselves to ensure they form correct pairs between writer and readers.
* Reduce exposure of SymbolTable, Stash and other classes not necessary for ↵Henning Baldersheim2022-03-101-0/+1
| | | | users of Slime.
* Move BucketIdListResultHenning Baldersheim2022-03-092-2/+2
|
* Reduce visibility of document::DocumentHenning Baldersheim2022-03-0714-44/+33
|
* Reduce use of Identifiable for document::DatatTypeHenning Baldersheim2022-03-0310-2/+16
|
* Check cluster-wide state inside lock to match state writeTor Brede Vekterli2022-03-031-1/+3
|
* Merge pull request #21477 from ↵Henning Baldersheim2022-03-014-18/+22
|\ | | | | | | | | vespa-engine/balder/use-wait_until-to-reduce-need-for-sampling-time Use wait_until and a deadline instead of a timeout.
| * timeout_end -> deadlineHenning Baldersheim2022-03-014-10/+10
| |
| * Use wait_until and a deadline instead of a timeout.Henning Baldersheim2022-03-014-18/+22
| | | | | | | | This reduces the need to smaple the time significantly.
* | Make reconfiguring dynamic vs unlimited throttling not require restartingTor Brede Vekterli2022-03-016-35/+52
|/ | | | | | | | | | | | Instead of having one abstract throttler created from bootstrap config, explicitly create one dynamic and one unlimited throttler and allow for atomically switching between the two based on received config. The `MergeHandler` component will now always fetch the current throttler from the `FileStorHandler` instead of caching it at construction time. This commit removes the `restart` annotation on the existing throttler type config enums.
* Add a simple adjustTimeoutByDetectedHz to compute a suitable timeout/naptime ↵Henning Baldersheim2022-02-282-2/+2
| | | | | | based on base VESPA_TIMER_HZ to use for random waiting when doing a poll based appraoch.
* Merge pull request #21408 from ↵Tor Brede Vekterli2022-02-286-2/+55
|\ | | | | | | | | vespa-engine/vekterli/fix-storage-slobrok-threading-edge-cases Gracefully flush pending Slobrok task on content node RPC teardown [run-systemtest]
| * Explicitly wait for cross-node Slobrok visibility in test instead of sleepingTor Brede Vekterli2022-02-285-2/+50
| | | | | | | | | | | | | | Prior code had an ad-hoc (and undocumented) sleep that would hopefully delay the rest of the test until there was mutual Slobrok visibility, but which was not guaranteed and usually over-pessimized. Replace with explicit checks of Slobrok mirrors.
| * Gracefully flush pending Slobrok task on content node RPC teardownTor Brede Vekterli2022-02-251-0/+5
| | | | | | | | | | Avoids race between transport connection destruction in one FNET thread and processing unregistration task in another FNET thread.
* | Remove inlining warnings (storage).Tor Egge2022-02-261-0/+3
|/
* Sync executor to ensure tasks are run prior to stripe teardowns in testsTor Brede Vekterli2022-02-251-2/+11
|
* Revert "Revert "- Create the common transport and threadpool in the main loop.""Henning Baldersheim2022-02-241-1/+1
|
* Revert "- Create the common transport and threadpool in the main loop."Henning Baldersheim2022-02-241-1/+1
|
* Merge pull request #21376 from ↵Tor Brede Vekterli2022-02-248-9/+52
|\ | | | | | | | | vespa-engine/vekterli/configurable-per-merge-feed-op-throttling Make it possible to configure what aspects of merges should be throttled [run-systemtest]
| * Make it possible to configure what aspects of merges should be throttledTor Brede Vekterli2022-02-248-9/+52
| | | | | | | | | | | | | | | | | | | | Add live config for choosing whether merges should be throttled on a per-feed operation (`MergeHandler`) level, or on an `ApplyBucketDiff` persistence thread level. This is intended to be a temporary feature while we do experiments, so some liberties are taken with regards to how holes are punched in the various abstraction layers.
* | - Create the common transport and threadpool in the main loop.Henning Baldersheim2022-02-241-1/+1
|/ | | | | - Also use the common transport for config subscriptions. - Put The TransportConfig in the fnet namespace.
* Add more config for tuning the persistence throttle policyTor Brede Vekterli2022-02-231-6/+10
|
* Add backend metrics for new operation throttler statisticsTor Brede Vekterli2022-02-223-0/+6
|
* Make ConfigUri constructors explicit and use same context where possible in ↵Henning Baldersheim2022-02-2021-43/+34
| | | | proton.
* Add a base vespa timer frequency that is used as a base to control poll ↵Henning Baldersheim2022-02-172-2/+2
| | | | frequency of various tasks.
* Remove possibility to update waitTime, maxProcessTime and maxTicksBeforWait ↵Henning Baldersheim2022-02-173-11/+6
| | | | from config.
* Merge pull request #21227 from ↵Tor Brede Vekterli2022-02-161-2/+8
|\ | | | | | | | | vespa-engine/vekterli/fix-data-race-in-persistence-test-mock Fix data race for persistence provider test mock
| * Fix data race for persistence provider test mockTor Brede Vekterli2022-02-161-2/+8
| |
* | Use std::nullopt for empty optionals.Geir Storli2022-02-161-1/+1
| |
* | Tag persistence threads with cpu category WRITE and relevant operations as READ.Geir Storli2022-02-163-4/+20
| |
* | Tag visitor threads with cpu category READ.Geir Storli2022-02-162-2/+2
| |
* | Support setting cpu category on storageframework threads.Geir Storli2022-02-161-1/+1
|/
* Allow inhibiting all default bucket space merges if global merges pendingTor Brede Vekterli2022-02-1514-10/+205
| | | | | | | | | | | | | | | | | | If the cluster controller publishes a cluster state where at least one node is marked as maintenance in the default space but is up in the global space, global bucket merges are pending in the cluster. We want these merges to complete ASAP, and to avoid starvation caused by distributors that finish their global merges quicker than others (or whose global bucket ops get preempted), allow for _completely_ inhibiting merges to buckets in the default space when global buckets are pending. Requires a live config flag to be set to actually be taken into account in the bucket synchronization state checker. Flag is tracked per distributor bucket space (per stripe) and is automatically cleared once a cluster state is received without pending global merges.
* Merge pull request #21149 from ↵Tor Brede Vekterli2022-02-111-0/+6
|\ | | | | | | | | vespa-engine/vekterli/add-distributor-config-for-default-space-merge-inhibition Add config for default bucket space merge inhibition during global merges