aboutsummaryrefslogtreecommitdiffstats
path: root/storage/src/tests/bucketdb
Commit message (Collapse)AuthorAgeFilesLines
* Drop support for old gtest.Tor Egge2022-11-181-1/+1
|
* Simplify state version checks by requiring exact version matchTor Brede Vekterli2022-09-221-1/+5
| | | | | | | | | | | The existing state unification logic was likely to help ensure that various distributor availability-states were treated as if they were simply Up, but the distributor has not been able to even _be_ in other available states than Up for many years. So it's effectively pointless. Remove unification entirely and instead require both the distributor and content node to be mutually in sync with the exact cluster state version.
* Avoid bucket DB race during content node cluster state transitionTor Brede Vekterli2022-09-211-4/+56
| | | | | | | | | | | | | | | | It was possible for a distributor bucket fetch request to be processed _after_ a cluster state was enabled (and internally propagated) on the content node, but _before_ all side effects of this enabling were complete and fully visible. This could cause inconsistent information to be returned to the distributor, causing nodes to get out of sync bucket metadata. This commit handles such transition periods by introducing an implicit barrier between observing the incoming command and outgoing reply for a particular cluster state version. Upon observing the reply for a version, all side effects must already be visible since the reply is only sent once internal state processing is complete (both above and below the SPI). Until initiated and completed versions converge, requests are rejected and will be transparently retried by the distributors.
* Remove legacy distribution hash fallbackTor Brede Vekterli2022-06-091-20/+4
| | | | | Was used to handle rolling upgrades between versions with different semantics a long time ago on the 7 branch.
* Make ConfigUri constructors explicit and use same context where possible in ↵Henning Baldersheim2022-02-201-3/+3
| | | | proton.
* more descriptive name for header fileArne H Juul2021-12-021-1/+1
|
* track namespace move in documenttypes.defArne H Juul2021-12-021-1/+1
| | | | | | | * For C++ code this introduces a "document::config" namespace, which will sometimes conflict with the global "config" namespace. * Move all forward-declarations of the types DocumenttypesConfig and DocumenttypesConfigBuilder to a common header file.
* Update 2019 Oath copyrights.gjoranv2021-10-271-1/+1
|
* Update 2017 copyright notices.gjoranv2021-10-074-4/+4
|
* Guard against processing bucket requests with inconsistent internal state ↵Tor Brede Vekterli2021-03-031-8/+37
| | | | | | | | | | version There's a tiny window of time between when the bucket manager observes a new state version and when the state version actually is visible in the rest of the process. We must ensure that we don't end up processing requests when these two differ, or we might erroneously process requests for version X using a state only valid for version Y < X.
* - Reduce visibility of ClusterState and Distribution.Henning Baldersheim2021-02-191-0/+2
|
* Inhibit activation of replicas out of sync with a replica majorityTor Brede Vekterli2021-02-171-0/+54
| | | | | | | | | | Adds a configurable max number of groups (default 0) whose replica activation is inhibited if the replica's bucket info is out of sync with a majority of other replicas. Intended to be used for the case where a group comes back up after transient unavailability and where the nodes are out of sync and should preferably not be activated until post-merging.
* Wire in HostInfo to FileStorManager.Tor Egge2021-01-181-1/+1
|
* Start expensive tests earlier.Henning Baldersheim2021-01-131-2/+0
|
* Have the BufferType::_emptyEntry be static.Henning Baldersheim2021-01-101-0/+1
| | | | Use an array of buffer types in the array class.
* - Wire in the guard to make it evident that we have it when making changes ↵Henning Baldersheim2020-12-151-0/+1
| | | | | | that require it. - Clean up some old members and code not used any more.
* Add test for explicit read guard iterator key orderingTor Brede Vekterli2020-10-301-0/+23
|
* Add striped implementation of B-tree content node bucket databaseTor Brede Vekterli2020-10-301-46/+8
| | | | | | | | | | Abstracts away multiple underlying B-tree DBs that each hold a subset of the super bucket space. Offers ordered iteration via a priority-queue based view over the sub DBs. Not yet ready for prime time, as the striping inherently requires an absolute lower bound on the bucket bits used in the system, which is currently not enforced.
* Remove legacy Judy array-backed bucket DB implementationTor Brede Vekterli2020-10-284-359/+2
|
* GC unused disk dimension.Henning Baldersheim2020-10-201-2/+2
|
* Remove legacy bucket DB initializer componentTor Brede Vekterli2020-10-192-597/+0
|
* Greatly simplify bucket DB persistence provider bootstrap procedureTor Brede Vekterli2020-10-163-21/+7
| | | | | | | | | | | | | | | | | | | | | | The legacy bucket DB initialization logic was designed for the case where bucket information was spread across potentially millions of files residing on spinning rust drives. It was therefore async and running in parallel with client operations, adding much complexity in order to deal with a myriad of concurrency edge cases. Replace this with a very simple, synchronous init method that expects the provider to have the required information readily and cheaply available. This effectively removes the concept of a node's "initializing" state, moving directly from reported state Down to Up. Even though a node still technically starts up in Initializing state, we never end up reporting this to the Cluster Controller as the DB init completes before the RPC server stack is set up. Legacy bucket DB initializer code will be removed in a separate pass. Also simplify bucket DB interface contract for mutating iteration, indicating that it is done in an unspecified order.
* GC disk related code.Henning Baldersheim2020-10-152-17/+6
|
* Remove partitions from SPI.Tor Egge2020-10-142-34/+8
|
* Use std::mutex/std::condition_variable over vespalib::MonitorHenning Baldersheim2020-10-131-2/+1
|
* Add noexcept as indicated by -WnoeexceptHenning Baldersheim2020-10-071-1/+1
|
* Add content node bucket DB memory usage metricsTor Brede Vekterli2020-09-041-1/+19
|
* Merge pull request #13819 from ↵Tor Brede Vekterli2020-07-081-14/+68
|\ | | | | | | | | vespa-engine/vekterli/basic-snapshot-support-for-content-node-bucket-db Vekterli/basic snapshot support for content node bucket db
| * Use bucket DB read guards for metric and status aggregationTor Brede Vekterli2020-07-071-0/+16
| |
| * Expose ReadGuard via AbstractLockableMap interfaceTor Brede Vekterli2020-07-071-14/+52
| | | | | | | | | | | | | | * Add working B-tree snapshot read guard impl * Add placeholder wrapper read guard for legacy DB * Enforce value const-ness of existing for_each_chunked iteration API * Return read guard entries by value instead of modifying ref argument
* | Consolidate search for GTest.Tor Egge2020-07-071-1/+0
|/
* Merge pull request #13706 from ↵Tor Brede Vekterli2020-06-302-156/+257
|\ | | | | | | | | vespa-engine/vekterli/btree-bucket-db-support-on-content-node Create generic B-tree bucket DB and content node DB implementation
| * Address review commentsTor Brede Vekterli2020-06-291-7/+6
| | | | | | | | | | Also rewrite some GMock macros that triggered Valgrind warnings due to default test object printers accessing uninitialized memory.
| * Create generic B-tree bucket DB and content node DB implementationTor Brede Vekterli2020-06-252-156/+258
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the first stage of removing the legacy DB implementation. Support for B-tree specific functionality such as lock-free snapshot reads will be added soon. This commit is just for feature parity. Abstract away actual database implementation to allow it to be chosen dynamically at startup. This abstraction does incur some overhead via call indirections and type erasures of callbacks, so it's likely it will be removed once the transition to the new B-tree DB has been completed. Since the algorithms used for bucket key operations is so similar between the content node and distributor, a generic B-tree backed bucket database has been created. The distributor DB will be rewritten around this code very soon. Due to the strong coupling between bucket locking and actual DB implementation details, the new bucket DB has a fairly significant code overlap with the legacy implementation. This is to avoid spending time abstracting away and factoring out code for a legacy implementation that is to be removed entirely anyway. Remove existing LockableMap functionality not used or that's only used by tests.
* | Use find_package to find gtest library.Tor Egge2020-06-291-1/+2
|/
* Remove unused legacy bucket DB functionalityTor Brede Vekterli2020-06-031-149/+0
|
* - Update metrics less often by removing the forceEventLogging alltogether.Henning Baldersheim2020-05-131-3/+3
| | | | - Let default bucket iteration work in smaller chunks with shorter waits.
* Some libraries print "0x0" for a null void ptr,Tor Egge2020-04-221-1/+1
|
* Disable old, non-deterministic testTor Brede Vekterli2019-09-271-1/+1
| | | | Needs to be rewritten or discarded.
* Add config override for simulating bucket info request processing latencyTor Brede Vekterli2019-09-201-1/+1
| | | | | | | | Simulates added request latency caused by the BucketManager computing bucket ownership for a very large number of buckets. Fetched at BucketManager init only, so not a dynamic config. This is only meant for internal testing so should not have any practical consequences.
* Remove DocIdString outside of documentHenning Baldersheim2019-08-191-4/+2
|
* Remove the use and testing of legacy groupdoc/userdoc/orderdoc document ids.Henning Baldersheim2019-08-091-2/+2
|
* Remove CppUnit dependencies in modulesTor Brede Vekterli2019-06-261-2/+2
| | | | Move test config helpers out of cppunit submodule.
* Convert remaining CppUnit tests to GTestTor Brede Vekterli2019-06-251-1/+1
| | | | | | Move base message sender stub out to common test module to avoid artificial dependency from persistence tests to the distributor tests.
* Convert LockableMapTest from CppUnit to GTestTor Brede Vekterli2019-06-072-628/+199
| | | | | Remove convoluted thread stress test which didn't actually _verify_ any kind of correctness (aside from the test not outright crashing).
* Convert JudyMultiMapTest from CppUnit to GtestTor Brede Vekterli2019-06-073-72/+51
|
* Convert JudyArrayTest from CppUnit to GtestTor Brede Vekterli2019-06-073-166/+96
|
* Convert BucketManagerTest and InitializerTest to gtestTor Brede Vekterli2019-06-073-768/+249
| | | | | Still some residual vdstestlib CppUnit traces that will need cleaning up later.
* Convert BucketInfoTest from CppUnit to GTestTor Brede Vekterli2019-06-062-106/+46
|
* Create gtest runner per test sub-module.Geir Storli2019-06-043-0/+25
| | | | This makes it possible to run storage tests in parallel.