aboutsummaryrefslogtreecommitdiffstats
path: root/storage/src/tests/persistence/filestorage/filestormanagertest.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Avoid using a reserved identifier naming formatTor Brede Vekterli2023-10-251-50/+50
| | | | | | Identifiers of the form `_Uppercased` are considered reserved by the standard. Not likely to cause ambiguity in practice, but it's preferable to stay on the good side of the standard-gods.
* Rewire `FileStorManager` configTor Brede Vekterli2023-10-241-2/+5
|
* Update copyrightJon Bratseth2023-10-091-1/+1
|
* Remove dead code from FileStorManager unit test.Tor Egge2023-08-311-27/+0
|
* GC some unused internal storage message typesTor Brede Vekterli2023-06-161-15/+0
| | | | | Remnants of the "file per bucket on spinning disks" days and no longer used for anything.
* Clean up some todos.Henning Baldersheim2023-06-081-4/+4
|
* Reduce creation of Document instances without DocumentTypeRepo.Geir Storli2023-03-131-6/+6
|
* remove document::RunnableHåvard Pettersen2023-02-201-17/+27
| | | | use std::thread directly instead
* Remove stacksize from the thread pools and thread executors.Henning Baldersheim2022-12-201-1/+1
|
* Force content node-internal bucket DB metric update during startupTor Brede Vekterli2022-08-301-0/+1
| | | | | | | | | | | | | | After initialization, the node will immediately start communicating with the cluster controller, exchanging host info. This host info contains a subset snapshot of the active metrics, which includes the total bucket count, doc count etc. It is critical that we must never report back host info _prior_ to having run at least one full sweep of the bucket database, lest we risk transiently reporting zero buckets held by the content node. Doing so could cause orchestration logic to perform operations based on erroneous assumptions. To avoid this, we explicitly force a full DB sweep and metric update prior to reporting the node as up. Since this function is called prior to the CommunicationManager thread being started, any CC health pings should also always happen after this init step.
* Remove '.sum' form vds sum metrics.Henning Baldersheim2022-06-081-6/+6
| | | | | | | | Remove '.sum' from metric names for storage node and also remove the average metrics for the same. Remove '.sum' from distributor metrics set and remove distributor average metrics. GC '.sum' from distributor metric names. Remove '.alldisks' from metric names and update tests. GC '.alldisks' from filestor metrics.
* GC unused Context parameterHenning Baldersheim2022-03-311-16/+7
|
* Reduce use of Identifiable for document::DatatTypeHenning Baldersheim2022-03-031-0/+4
|
* Sync executor to ensure tasks are run prior to stripe teardowns in testsTor Brede Vekterli2022-02-251-2/+11
|
* Make ConfigUri constructors explicit and use same context where possible in ↵Henning Baldersheim2022-02-201-1/+1
| | | | proton.
* Support dynamic throttling of async persistence operationsTor Brede Vekterli2022-01-101-17/+17
| | | | | | | | | | | | | | | | | | | | | | | Adds an operation throttler that is intended to provide global throttling of async operations across all persistence stripe threads. A throttler wraps a logical max pending window size of in-flight operations. Depending on the throttler implementation, the window size may expand and shrink dynamically. Exactly how and when this happens is unspecified. Commit adds two throttler implementations: * An unlimited throttler that is no-op and never blocks. * A throttler built around the mbus `DynamicThrottlePolicy` and defers all window decisions to it. Current config default is to use the unlimited throttler. Config changes require a process restart. Offers both polling and (timed, non-timed) blocking calls for acquiring a throttle token. If the returned token is valid, the caller may proceed to invoke the asynchronous operation. The window slot taken up by a valid throttle token is implicitly freed up when the token is destroyed.
* Update 2017 copyright notices.gjoranv2021-10-071-1/+1
|
* - Change error handling so that both synchonous and asynchronous errors can ↵Henning Baldersheim2021-02-231-14/+14
| | | | | | | | be reported back from bucket executor. - Treat remapping as an error. - For lidspace compaction job iterator is reset and will be recreated on next invocation. - For bucketmove th ebucket is rechecked and either discarded or restarted.
* - Reduce visibility of ClusterState and Distribution.Henning Baldersheim2021-02-191-0/+2
|
* use size literals in storageArne Juul2021-02-151-6/+7
|
* Revert "Properly track execution of BucketTasks and provide sync() and ↵Henning Baldersheim2021-02-021-34/+0
| | | | order… "
* Properly track execution of BucketTasks and provide sync() and orderly shutdown.Henning Baldersheim2021-02-021-0/+34
|
* Revert "Implement BucketExecutor::sync."Henning Baldersheim2021-02-021-34/+0
|
* Use conditional notify instead of sleep.Henning Baldersheim2021-02-011-0/+1
|
* Implement BucketExecutor::sync.Henning Baldersheim2021-02-011-0/+33
|
* Add debug dumping and other minor followup on PR comments.Henning Baldersheim2021-01-201-3/+3
|
* Implement bucketexecutor interface and.Henning Baldersheim2021-01-191-0/+47
|
* Wire in HostInfo to FileStorManager.Tor Egge2021-01-181-1/+1
|
* Wire in skeleton for implementation of the BucketExecutor interface.Henning Baldersheim2021-01-171-7/+7
|
* GC LoadTypeSetHenning Baldersheim2020-11-271-3/+0
|
* Remove loadmetrics indirection.Henning Baldersheim2020-11-261-7/+7
|
* Use correct storage cluster name.Henning Baldersheim2020-11-261-2/+2
|
* As we have have now removed the expensive Route member we can further ↵Henning Baldersheim2020-11-261-118/+67
| | | | | | | | | | compact the message objects. - Compact StorageMessageAddress to 16 bytes by - using reference to cluster name. - Use small enums for protocol and node type. - Avoid having StorageMessage as separate allocation. - Avoid default values
* Remove load type from ContextHenning Baldersheim2020-11-241-8/+6
|
* Misc cleanup in filestor, no semantic changesTor Brede Vekterli2020-11-131-26/+24
|
* Add support for async message handling when scheduling storage messages in ↵Geir Storli2020-10-261-17/+142
| | | | | | | | | FileStorManager. When turned on, the calling thread (e.g. FNET network thread when using Storage API RPC) gets the next async message to handle (if any) as part of scheduling a storage message. This async message is then handled by the calling thread immediately, instead of going via a persistence thread.
* Use default destructor outside class definition.Geir Storli2020-10-221-2/+6
|
* Reduce code duplication by consolidating setup of PersistenceHandler.Geir Storli2020-10-221-39/+38
|
* Reduce code duplication by consolidating setup of FileStorHandler.Geir Storli2020-10-221-160/+76
|
* Reduce code duplication by using TestFileStorComponents in more tests.Geir Storli2020-10-221-121/+39
|
* GC unused disk dimension.Henning Baldersheim2020-10-201-21/+21
|
* - Add class comments.Henning Baldersheim2020-10-191-1/+1
| | | | | | - Reduce code visibility. - Clean up some unused members. - Some code unification.
* Split the persistence thread and the message handler.Henning Baldersheim2020-10-191-45/+40
| | | | | | - Let FileStorManager own and control the Component and PersistenceHandler separately from the Persistence thread. - Let FileStorManager allocate and control stripe assignment.
* Do not retrieve config once for every thread.Henning Baldersheim2020-10-181-1/+2
|
* We must detect changes in document config.Henning Baldersheim2020-10-181-0/+1
|
* Factor out handling of operations that might change bucket ownership.Henning Baldersheim2020-10-181-8/+21
|
* Change FileStorHandler into an interface that FileStorHandlerImpl implements.Geir Storli2020-10-161-23/+24
|
* Greatly simplify bucket DB persistence provider bootstrap procedureTor Brede Vekterli2020-10-161-20/+80
| | | | | | | | | | | | | | | | | | | | | | The legacy bucket DB initialization logic was designed for the case where bucket information was spread across potentially millions of files residing on spinning rust drives. It was therefore async and running in parallel with client operations, adding much complexity in order to deal with a myriad of concurrency edge cases. Replace this with a very simple, synchronous init method that expects the provider to have the required information readily and cheaply available. This effectively removes the concept of a node's "initializing" state, moving directly from reported state Down to Up. Even though a node still technically starts up in Initializing state, we never end up reporting this to the Cluster Controller as the DB init completes before the RPC server stack is set up. Legacy bucket DB initializer code will be removed in a separate pass. Also simplify bucket DB interface contract for mutating iteration, indicating that it is done in an unspecified order.
* There will always be a sequencer.Henning Baldersheim2020-10-151-6/+7
|
* GC disk related code.Henning Baldersheim2020-10-151-68/+60
|