aboutsummaryrefslogtreecommitdiffstats
path: root/storage
Commit message (Collapse)AuthorAgeFilesLines
* Use named constants instead of strings sprinkled all over.Henning Baldersheim2020-08-0413-39/+41
|
* Remove the unused [header] and [body] fieldsets to simplify backend ↵Henning Baldersheim2020-08-044-20/+11
| | | | optimization during get.
* Don't verbose print documents on debug log levelTor Brede Vekterli2020-07-221-2/+2
| | | | | Causes way too much output to be useful. Move to separate logging that is only visible at spam level.
* Fix variable name typoTor Brede Vekterli2020-07-161-3/+3
|
* Fix typo in function nameTor Brede Vekterli2020-07-162-5/+5
|
* Don't pretend process ID is thread ID in log messageTor Brede Vekterli2020-07-161-2/+2
| | | | Logging subsystem includes the correct thread ID either way.
* Merge pull request #13897 from ↵Tor Brede Vekterli2020-07-151-3/+3
|\ | | | | | | | | vespa-engine/vekterli/use-atomic-operation-counting Ensure operation counting is thread-safe
| * Ensure operation counting is thread-safeTor Brede Vekterli2020-07-151-3/+3
| | | | | | | | | | | | | | | | | | With the introduction of async response handling, SPI getBucketInfo() can now be called from multiple threads concurrently. Test was not originally written with that in mind. Use std::atomic instead of unsafe raw increment. For future-proofing, also make other counters atomic.
* | Add per-operation metrics for puts and removes that are part of mergesTor Brede Vekterli2020-07-158-74/+104
|/ | | | Move all merge-related metrics out into a separate wrapper for convenience.
* Emit log warning on repeated content node bucket info fetch failuresTor Brede Vekterli2020-07-133-7/+31
| | | | | | | Currently requires a certain number of repeated failures for a given cluster state transition. Rationale is that problematic nodes usually fail for a prolonged amount of time, so it's wise to reduce log noise from more transient failures. Threshold to be adjusted later as needed.
* Re-add assertsHenning Baldersheim2020-07-091-2/+2
|
* optionalEnqueue -> enque_or_processHenning Baldersheim2020-07-092-5/+5
|
* Merge branch 'master' into balder/skip-communicationmanager-onlyHenning Baldersheim2020-07-092-18/+16
|\
| * Non semantic cleanup of communicationmanager prior to semantic change.Henning Baldersheim2020-07-082-19/+17
| |
* | Config control over what treads to skip.Henning Baldersheim2020-07-093-13/+46
| |
* | Remove assert that i snow possible during testing.Henning Baldersheim2020-07-081-4/+4
| |
* | Since requests are now exeuted by caller storge must be open to send anything.Henning Baldersheim2020-07-081-2/+2
| |
* | Skip communication manager thread for requests.Henning Baldersheim2020-07-081-6/+5
|/
* Reduce log level of bucket DB pruning elision messageTor Brede Vekterli2020-07-081-1/+1
| | | | | | | Was originally set to `info` to help track down the source of an assertion failure that should not be possible to trigger and that only happened once. Many moons later it has yet to reappear, so removing some log noise that could only be interesting to me.
* Merge pull request #13824 from vespa-engine/toregge/use-external-dependsTor Egge2020-07-081-0/+1
|\ | | | | Use EXTERNAL_DEPENDS for external dependencies.
| * Use EXTERNAL_DEPENDS for external dependencies.Tor Egge2020-07-071-0/+1
| |
* | Merge pull request #13819 from ↵Tor Brede Vekterli2020-07-0818-94/+314
|\ \ | |/ |/| | | | | vespa-engine/vekterli/basic-snapshot-support-for-content-node-bucket-db Vekterli/basic snapshot support for content node bucket db
| * Use bucket DB read guards for metric and status aggregationTor Brede Vekterli2020-07-074-17/+34
| |
| * Expose ReadGuard via AbstractLockableMap interfaceTor Brede Vekterli2020-07-0716-78/+281
| | | | | | | | | | | | | | * Add working B-tree snapshot read guard impl * Add placeholder wrapper read guard for legacy DB * Enforce value const-ness of existing for_each_chunked iteration API * Return read guard entries by value instead of modifying ref argument
* | Consolidate search for GTest.Tor Egge2020-07-0710-10/+0
| |
* | Use module source dependencies.Tor Egge2020-07-0717-38/+0
| |
* | Merge pull request #13804 from ↵Henning Baldersheim2020-07-061-1/+14
|\ \ | |/ |/| | | | | vespa-engine/balder/let-sequencer-type-be-configurable Let sequencer type be configurable
| * Do not access object has been moved away.Henning Baldersheim2020-07-061-1/+1
| |
| * Let sequencer type be configurableHenning Baldersheim2020-07-061-1/+14
| |
* | Merge pull request #13798 from ↵Tor Brede Vekterli2020-07-0619-783/+23
|\ \ | |/ |/| | | | | vespa-engine/vekterli/remove-legacy-distributor-bucket-database-implementation Remove legacy distributor bucket DB implementation and config wiring
| * Remove legacy distributor bucket DB implementation and config wiringTor Brede Vekterli2020-07-0319-783/+23
| |
* | Merge pull request #13796 from vespa-engine/arnej/less-exit-calls-2Arne H Juul2020-07-051-1/+1
|\ \ | |/ |/| Arnej/less exit calls 2
| * avoid naked exitArne Juul2020-07-031-1/+1
| |
* | Unify content node and distributor B-tree databasesTor Brede Vekterli2020-07-0217-765/+365
|/ | | | | | * Rewrite distributor B-tree DB in terms of generic implementation * Move out merger and read guard types so they can be used with both content and distributor databases.
* Since there is not a well defined contract for the the sendReply interface,Henning Baldersheim2020-07-0110-3/+36
| | | | | | I add a stricter sendReplyDirectly interface where the caller guarantees that he has no hidden requirements that the calle should be aware of. This will avoid a task switch when propagating the reply.
* Merge pull request #13768 from ↵Henning Baldersheim2020-07-012-14/+7
|\ | | | | | | | | vespa-engine/balder/avoid-dispatch-on-reply-in-bucket-manager Bucket manager also dispatches directly.
| * Bucket manager also dispatches directly.Henning Baldersheim2020-07-012-14/+7
| |
* | Need this dispatching still to avoid deadlocks.Henning Baldersheim2020-07-012-6/+8
| |
* | Skip dispatching on reply in FileStorManagerHenning Baldersheim2020-07-013-10/+6
|/
* Flip distributor B-tree DB config flag to true by defaultTor Brede Vekterli2020-06-302-6/+2
|
* Merge pull request #13706 from ↵Tor Brede Vekterli2020-06-3031-715/+2304
|\ | | | | | | | | vespa-engine/vekterli/btree-bucket-db-support-on-content-node Create generic B-tree bucket DB and content node DB implementation
| * Address review commentsTor Brede Vekterli2020-06-296-23/+25
| | | | | | | | | | Also rewrite some GMock macros that triggered Valgrind warnings due to default test object printers accessing uninitialized memory.
| * Wire config for enabling content node B-tree bucket DBTor Brede Vekterli2020-06-2513-29/+37
| |
| * Create generic B-tree bucket DB and content node DB implementationTor Brede Vekterli2020-06-2521-691/+2270
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the first stage of removing the legacy DB implementation. Support for B-tree specific functionality such as lock-free snapshot reads will be added soon. This commit is just for feature parity. Abstract away actual database implementation to allow it to be chosen dynamically at startup. This abstraction does incur some overhead via call indirections and type erasures of callbacks, so it's likely it will be removed once the transition to the new B-tree DB has been completed. Since the algorithms used for bucket key operations is so similar between the content node and distributor, a generic B-tree backed bucket database has been created. The distributor DB will be rewritten around this code very soon. Due to the strong coupling between bucket locking and actual DB implementation details, the new bucket DB has a fairly significant code overlap with the legacy implementation. This is to avoid spending time abstracting away and factoring out code for a legacy implementation that is to be removed entirely anyway. Remove existing LockableMap functionality not used or that's only used by tests.
* | Use find_package to find gtest library.Tor Egge2020-06-2910-10/+20
| |
* | Precompute node 2 group mapping config time.Henning Baldersheim2020-06-261-9/+13
| |
* | Use std::moveHenning Baldersheim2020-06-261-1/+3
| |
* | Modernize to c++11 style loops.Henning Baldersheim2020-06-261-4/+4
|/
* Clean up some visiting code. No functional changes.Tor Brede Vekterli2020-06-092-116/+76
|
* Use xxhash64 for bucket-to-stripe distributionTor Brede Vekterli2020-06-082-6/+10
| | | | | | | | | Existing naive prime-based solution was susceptible to scheduling operations for the subtree of a superbucket in one strand alone, despite previous attempts to disperse this using prime number multiplication. This would put a serious limiter on parallelism for super bucket locality-sensitive reads such as streaming search visitors.