aboutsummaryrefslogtreecommitdiffstats
path: root/storage/src/tests/bucketdb
Commit message (Collapse)AuthorAgeFilesLines
* Make ConfigUri constructors explicit and use same context where possible in ↵Henning Baldersheim2022-02-201-3/+3
| | | | proton.
* more descriptive name for header fileArne H Juul2021-12-021-1/+1
|
* track namespace move in documenttypes.defArne H Juul2021-12-021-1/+1
| | | | | | | * For C++ code this introduces a "document::config" namespace, which will sometimes conflict with the global "config" namespace. * Move all forward-declarations of the types DocumenttypesConfig and DocumenttypesConfigBuilder to a common header file.
* Update 2019 Oath copyrights.gjoranv2021-10-271-1/+1
|
* Update 2017 copyright notices.gjoranv2021-10-074-4/+4
|
* Guard against processing bucket requests with inconsistent internal state ↵Tor Brede Vekterli2021-03-031-8/+37
| | | | | | | | | | version There's a tiny window of time between when the bucket manager observes a new state version and when the state version actually is visible in the rest of the process. We must ensure that we don't end up processing requests when these two differ, or we might erroneously process requests for version X using a state only valid for version Y < X.
* - Reduce visibility of ClusterState and Distribution.Henning Baldersheim2021-02-191-0/+2
|
* Inhibit activation of replicas out of sync with a replica majorityTor Brede Vekterli2021-02-171-0/+54
| | | | | | | | | | Adds a configurable max number of groups (default 0) whose replica activation is inhibited if the replica's bucket info is out of sync with a majority of other replicas. Intended to be used for the case where a group comes back up after transient unavailability and where the nodes are out of sync and should preferably not be activated until post-merging.
* Wire in HostInfo to FileStorManager.Tor Egge2021-01-181-1/+1
|
* Start expensive tests earlier.Henning Baldersheim2021-01-131-2/+0
|
* Have the BufferType::_emptyEntry be static.Henning Baldersheim2021-01-101-0/+1
| | | | Use an array of buffer types in the array class.
* - Wire in the guard to make it evident that we have it when making changes ↵Henning Baldersheim2020-12-151-0/+1
| | | | | | that require it. - Clean up some old members and code not used any more.
* Add test for explicit read guard iterator key orderingTor Brede Vekterli2020-10-301-0/+23
|
* Add striped implementation of B-tree content node bucket databaseTor Brede Vekterli2020-10-301-46/+8
| | | | | | | | | | Abstracts away multiple underlying B-tree DBs that each hold a subset of the super bucket space. Offers ordered iteration via a priority-queue based view over the sub DBs. Not yet ready for prime time, as the striping inherently requires an absolute lower bound on the bucket bits used in the system, which is currently not enforced.
* Remove legacy Judy array-backed bucket DB implementationTor Brede Vekterli2020-10-284-359/+2
|
* GC unused disk dimension.Henning Baldersheim2020-10-201-2/+2
|
* Remove legacy bucket DB initializer componentTor Brede Vekterli2020-10-192-597/+0
|
* Greatly simplify bucket DB persistence provider bootstrap procedureTor Brede Vekterli2020-10-163-21/+7
| | | | | | | | | | | | | | | | | | | | | | The legacy bucket DB initialization logic was designed for the case where bucket information was spread across potentially millions of files residing on spinning rust drives. It was therefore async and running in parallel with client operations, adding much complexity in order to deal with a myriad of concurrency edge cases. Replace this with a very simple, synchronous init method that expects the provider to have the required information readily and cheaply available. This effectively removes the concept of a node's "initializing" state, moving directly from reported state Down to Up. Even though a node still technically starts up in Initializing state, we never end up reporting this to the Cluster Controller as the DB init completes before the RPC server stack is set up. Legacy bucket DB initializer code will be removed in a separate pass. Also simplify bucket DB interface contract for mutating iteration, indicating that it is done in an unspecified order.
* GC disk related code.Henning Baldersheim2020-10-152-17/+6
|
* Remove partitions from SPI.Tor Egge2020-10-142-34/+8
|
* Use std::mutex/std::condition_variable over vespalib::MonitorHenning Baldersheim2020-10-131-2/+1
|
* Add noexcept as indicated by -WnoeexceptHenning Baldersheim2020-10-071-1/+1
|
* Add content node bucket DB memory usage metricsTor Brede Vekterli2020-09-041-1/+19
|
* Merge pull request #13819 from ↵Tor Brede Vekterli2020-07-081-14/+68
|\ | | | | | | | | vespa-engine/vekterli/basic-snapshot-support-for-content-node-bucket-db Vekterli/basic snapshot support for content node bucket db
| * Use bucket DB read guards for metric and status aggregationTor Brede Vekterli2020-07-071-0/+16
| |
| * Expose ReadGuard via AbstractLockableMap interfaceTor Brede Vekterli2020-07-071-14/+52
| | | | | | | | | | | | | | * Add working B-tree snapshot read guard impl * Add placeholder wrapper read guard for legacy DB * Enforce value const-ness of existing for_each_chunked iteration API * Return read guard entries by value instead of modifying ref argument
* | Consolidate search for GTest.Tor Egge2020-07-071-1/+0
|/
* Merge pull request #13706 from ↵Tor Brede Vekterli2020-06-302-156/+257
|\ | | | | | | | | vespa-engine/vekterli/btree-bucket-db-support-on-content-node Create generic B-tree bucket DB and content node DB implementation
| * Address review commentsTor Brede Vekterli2020-06-291-7/+6
| | | | | | | | | | Also rewrite some GMock macros that triggered Valgrind warnings due to default test object printers accessing uninitialized memory.
| * Create generic B-tree bucket DB and content node DB implementationTor Brede Vekterli2020-06-252-156/+258
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the first stage of removing the legacy DB implementation. Support for B-tree specific functionality such as lock-free snapshot reads will be added soon. This commit is just for feature parity. Abstract away actual database implementation to allow it to be chosen dynamically at startup. This abstraction does incur some overhead via call indirections and type erasures of callbacks, so it's likely it will be removed once the transition to the new B-tree DB has been completed. Since the algorithms used for bucket key operations is so similar between the content node and distributor, a generic B-tree backed bucket database has been created. The distributor DB will be rewritten around this code very soon. Due to the strong coupling between bucket locking and actual DB implementation details, the new bucket DB has a fairly significant code overlap with the legacy implementation. This is to avoid spending time abstracting away and factoring out code for a legacy implementation that is to be removed entirely anyway. Remove existing LockableMap functionality not used or that's only used by tests.
* | Use find_package to find gtest library.Tor Egge2020-06-291-1/+2
|/
* Remove unused legacy bucket DB functionalityTor Brede Vekterli2020-06-031-149/+0
|
* - Update metrics less often by removing the forceEventLogging alltogether.Henning Baldersheim2020-05-131-3/+3
| | | | - Let default bucket iteration work in smaller chunks with shorter waits.
* Some libraries print "0x0" for a null void ptr,Tor Egge2020-04-221-1/+1
|
* Disable old, non-deterministic testTor Brede Vekterli2019-09-271-1/+1
| | | | Needs to be rewritten or discarded.
* Add config override for simulating bucket info request processing latencyTor Brede Vekterli2019-09-201-1/+1
| | | | | | | | Simulates added request latency caused by the BucketManager computing bucket ownership for a very large number of buckets. Fetched at BucketManager init only, so not a dynamic config. This is only meant for internal testing so should not have any practical consequences.
* Remove DocIdString outside of documentHenning Baldersheim2019-08-191-4/+2
|
* Remove the use and testing of legacy groupdoc/userdoc/orderdoc document ids.Henning Baldersheim2019-08-091-2/+2
|
* Remove CppUnit dependencies in modulesTor Brede Vekterli2019-06-261-2/+2
| | | | Move test config helpers out of cppunit submodule.
* Convert remaining CppUnit tests to GTestTor Brede Vekterli2019-06-251-1/+1
| | | | | | Move base message sender stub out to common test module to avoid artificial dependency from persistence tests to the distributor tests.
* Convert LockableMapTest from CppUnit to GTestTor Brede Vekterli2019-06-072-628/+199
| | | | | Remove convoluted thread stress test which didn't actually _verify_ any kind of correctness (aside from the test not outright crashing).
* Convert JudyMultiMapTest from CppUnit to GtestTor Brede Vekterli2019-06-073-72/+51
|
* Convert JudyArrayTest from CppUnit to GtestTor Brede Vekterli2019-06-073-166/+96
|
* Convert BucketManagerTest and InitializerTest to gtestTor Brede Vekterli2019-06-073-768/+249
| | | | | Still some residual vdstestlib CppUnit traces that will need cleaning up later.
* Convert BucketInfoTest from CppUnit to GTestTor Brede Vekterli2019-06-062-106/+46
|
* Create gtest runner per test sub-module.Geir Storli2019-06-043-0/+25
| | | | This makes it possible to run storage tests in parallel.
* Add workarounds for legacy global distribution hash handlingTor Brede Vekterli2019-02-211-0/+61
| | | | | | | | | | | | | | | | | | | This addresses a regression introduced as part of #8479, which in turn was intended to serve as a fix for issue #8475. This regression would stall cluster state convergence when a subset of nodes contained the fix and another subset did not. With the workarounds present, nodes gracefully handle the case where different distribution hashes are expected for the global bucket space. `BucketManager` will now fall back to comparing the new incoming hash to that of the legacy derived distribution config if it mismatches. `PendingClusterState` will try to send a subset of bucket info requests with legacy hash format for the global bucket space iff there has been at least 1 rejected request. All these workarounds will be removed on Vespa 8.
* Merge pull request #8443 from ↵Tor Brede Vekterli2019-02-111-1/+47
|\ | | | | | | | | vespa-engine/vekterli/add-per-bucket-space-data-metrics-on-content-node Expose data metrics per bucket space on content node
| * Expose data metrics per bucket space on content nodeTor Brede Vekterli2019-02-081-1/+47
| | | | | | | | Legacy metrics that cover all bucket spaces remain unchanged.
* | Eliminate some clang warnings in storage.Tor Egge2019-02-101-27/+0
|/