aboutsummaryrefslogtreecommitdiffstats
path: root/storage
Commit message (Collapse)AuthorAgeFilesLines
* Flip distributor B-tree DB config flag to true by defaultTor Brede Vekterli2020-06-302-6/+2
|
* Merge pull request #13706 from ↵Tor Brede Vekterli2020-06-3031-715/+2304
|\ | | | | | | | | vespa-engine/vekterli/btree-bucket-db-support-on-content-node Create generic B-tree bucket DB and content node DB implementation
| * Address review commentsTor Brede Vekterli2020-06-296-23/+25
| | | | | | | | | | Also rewrite some GMock macros that triggered Valgrind warnings due to default test object printers accessing uninitialized memory.
| * Wire config for enabling content node B-tree bucket DBTor Brede Vekterli2020-06-2513-29/+37
| |
| * Create generic B-tree bucket DB and content node DB implementationTor Brede Vekterli2020-06-2521-691/+2270
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the first stage of removing the legacy DB implementation. Support for B-tree specific functionality such as lock-free snapshot reads will be added soon. This commit is just for feature parity. Abstract away actual database implementation to allow it to be chosen dynamically at startup. This abstraction does incur some overhead via call indirections and type erasures of callbacks, so it's likely it will be removed once the transition to the new B-tree DB has been completed. Since the algorithms used for bucket key operations is so similar between the content node and distributor, a generic B-tree backed bucket database has been created. The distributor DB will be rewritten around this code very soon. Due to the strong coupling between bucket locking and actual DB implementation details, the new bucket DB has a fairly significant code overlap with the legacy implementation. This is to avoid spending time abstracting away and factoring out code for a legacy implementation that is to be removed entirely anyway. Remove existing LockableMap functionality not used or that's only used by tests.
* | Use find_package to find gtest library.Tor Egge2020-06-2910-10/+20
| |
* | Precompute node 2 group mapping config time.Henning Baldersheim2020-06-261-9/+13
| |
* | Use std::moveHenning Baldersheim2020-06-261-1/+3
| |
* | Modernize to c++11 style loops.Henning Baldersheim2020-06-261-4/+4
|/
* Clean up some visiting code. No functional changes.Tor Brede Vekterli2020-06-092-116/+76
|
* Use xxhash64 for bucket-to-stripe distributionTor Brede Vekterli2020-06-082-6/+10
| | | | | | | | | Existing naive prime-based solution was susceptible to scheduling operations for the subtree of a superbucket in one strand alone, despite previous attempts to disperse this using prime number multiplication. This would put a serious limiter on parallelism for super bucket locality-sensitive reads such as streaming search visitors.
* Remove unused legacy bucket DB functionalityTor Brede Vekterli2020-06-033-188/+0
|
* Test that single Get sent by update op works with tombstonesTor Brede Vekterli2020-05-261-0/+57
|
* Handle tombstones in GetOperationTor Brede Vekterli2020-05-265-24/+106
| | | | | | | If the newest document version is a tombstone, behave as if the document was not found at all. Since we still track replica consistency, this should work as expected for multi-phase update operations as well.
* Propagate provider tombstone metadata to internal SPI GetResultTor Brede Vekterli2020-05-261-1/+2
|
* Don't tie message lifetime directly to tracker objectTor Brede Vekterli2020-05-221-1/+3
| | | | | | | The tracker was passed by move down to the handler function, but the surrounding code would try to auto-synthesize a reply from the message (now owned by the tracker) if an exception was thrown from the handler. Fun ensued.
* Ensure that we hold the lock when calling ↵Henning Baldersheim2020-05-132-7/+6
| | | | mark_controller_as_having_observed_explicit_node_state.
* - Update metrics less often by removing the forceEventLogging alltogether.Henning Baldersheim2020-05-133-5/+8
| | | | - Let default bucket iteration work in smaller chunks with shorter waits.
* Avoid filling log with unused metrics events.Henning Baldersheim2020-05-092-17/+4
|
* Remove unused clearResult method, and use std::lock_guardHenning Baldersheim2020-05-081-4/+1
|
* Use a lock to ensure it is thread safe.Henning Baldersheim2020-05-082-6/+18
|
* Update comments.Henning Baldersheim2020-05-051-0/+2
|
* Add async update and followup on PR comments.Henning Baldersheim2020-05-056-72/+79
|
* Only generate reply if it is a command.Henning Baldersheim2020-05-041-7/+9
|
* Implement async putHenning Baldersheim2020-05-0420-201/+394
| | | | Implement async remove.
* Merge branch 'master' into ↵Henning Baldersheim2020-05-0420-325/+161
|\ | | | | | | | | | | | | vekterli/remove-deprecated-bucket-disk-move-functionality Conflicts: storage/src/tests/persistence/diskmoveoperationhandlertest.cpp
| * Revert "- Implement async put"Harald Musum2020-05-0421-326/+162
| |
* | Remove deprecated bucket cross-disk move functionalityTor Brede Vekterli2020-05-0435-1919/+15
|/ | | | | The notion of multiple disks hasn't been supported since we removed VDS, and likely won't be in the future either.
* Merge pull request #13146 from vespa-engine/balder/implement-async-put-4Henning Baldersheim2020-05-0421-162/+326
|\ | | | | - Implement async put
| * Ensure that eval order might cause object to disappear under your feet by ↵Henning Baldersheim2020-05-041-9/+10
| | | | | | | | taking an reference when it is safe.
| * - Implement async putHenning Baldersheim2020-05-0421-157/+320
| | | | | | | | | | | | - Move result processing to MessageTracker - Wire putAsync through provider error wrapper too. - Handle both sync and async replies in tests.
* | Fix forward declaration of PersistenceUtil.Tor Egge2020-05-041-1/+1
|/
* Rename namespace search::btree to vespalib::btree.Tor Egge2020-05-042-8/+8
| | | | Rename namespace search::datastore to vespalib::datastore.
* Remove RepairBucketCommandTor Brede Vekterli2020-04-305-150/+0
|
* Remove deprecated BucketIntegrityCheckerTor Brede Vekterli2020-04-3011-1132/+0
| | | | Not in use after VDS was removed.
* Add class comments and resolve PR comments.Henning Baldersheim2020-04-301-6/+4
|
* Move error checking to tracker.Henning Baldersheim2020-04-294-27/+27
|
* - Add async interface to putHenning Baldersheim2020-04-2922-441/+403
| | | | | - Use MessageTracker for keeping context. - implement putAsync, but still use it synchronously.
* Use rvalue qualifierHenning Baldersheim2020-04-284-8/+8
|
* getReplySP => stealReplySPHenning Baldersheim2020-04-284-8/+8
|
* Implement hasReply avoid copying the shared_ptr just to peak at the result.Henning Baldersheim2020-04-288-70/+64
|
* Merge pull request #13084 from ↵Tor Brede Vekterli2020-04-283-49/+146
|\ | | | | | | | | vespa-engine/vekterli/optimize-btree-find-parents-with-fix Optimize B-tree bucket DB lookup with used-bits aggregation
| * Optimize B-tree bucket DB lookup with used-bits aggregationTor Brede Vekterli2020-04-273-49/+146
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By tracking the minimum used bits count across all buckets in the database we can immediately start seeking at that implicit level in the tree, as we know no parent buckets can exist above that level. Local synthetic benchmarking shows the following results with a DB size of 917504 buckets and performing getParents for all buckets in sequence: Before optimization: - B-tree DB: 0.593321 seconds - Legacy DB: 0.227947 seconds After optimization: - B-tree DB: 0.191971 seconds - Legacy DB: (unchanged)
* | Remove flush from provider interface.Henning Baldersheim2020-04-279-85/+10
| |
* | Update comments, and don't copy the shared ptr when not necessary.Henning Baldersheim2020-04-273-9/+10
| |
* | Remove batching of messages that has no effect in favor of making async ↵Henning Baldersheim2020-04-278-265/+8
| | | | | | | | operations easier to implement.
* | Update comments to reflect reality.Henning Baldersheim2020-04-271-3/+1
| |
* | Prepare for making persistence layer async.Henning Baldersheim2020-04-266-102/+103
|/ | | | Avoid state in the thread.
* Revert "Optimize B-tree bucket DB lookup with used-bits aggregation"Tor Brede Vekterli2020-04-253-79/+12
|
* Optimize B-tree bucket DB lookup with used-bits aggregationTor Brede Vekterli2020-04-243-12/+79
| | | | | | | | | | | | | | | | | | | By tracking the minimum used bits count across all buckets in the database we can immediately start seeking at that implicit level in the tree, as we know no parent buckets can exist above that level. Local synthetic benchmarking shows the following results with a DB size of 917504 buckets and performing getParents for all buckets in sequence: Before optimization: - B-tree DB: 0.593321 seconds - Legacy DB: 0.227947 seconds After optimization: - B-tree DB: 0.213738 seconds - Legacy DB: (unchanged)