| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
Identifiers of the form `_Uppercased` are considered reserved by
the standard. Not likely to cause ambiguity in practice, but it's
preferable to stay on the good side of the standard-gods.
|
| |
|
| |
|
| |
|
|
|
|
|
| |
Remnants of the "file per bucket on spinning disks" days and no
longer used for anything.
|
| |
|
| |
|
|
|
|
| |
use std::thread directly instead
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After initialization, the node will immediately start communicating with the cluster
controller, exchanging host info. This host info contains a subset snapshot of the active
metrics, which includes the total bucket count, doc count etc. It is critical that
we must never report back host info _prior_ to having run at least one full sweep of
the bucket database, lest we risk transiently reporting zero buckets held by the
content node. Doing so could cause orchestration logic to perform operations based
on erroneous assumptions.
To avoid this, we explicitly force a full DB sweep and metric update prior to reporting
the node as up. Since this function is called prior to the CommunicationManager thread
being started, any CC health pings should also always happen after this init step.
|
|
|
|
|
|
|
|
| |
Remove '.sum' from metric names for storage node and also remove the average metrics for the same.
Remove '.sum' from distributor metrics set and remove distributor average metrics.
GC '.sum' from distributor metric names.
Remove '.alldisks' from metric names and update tests.
GC '.alldisks' from filestor metrics.
|
| |
|
| |
|
| |
|
|
|
|
| |
proton.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds an operation throttler that is intended to provide global throttling
of async operations across all persistence stripe threads. A throttler
wraps a logical max pending window size of in-flight operations. Depending
on the throttler implementation, the window size may expand and shrink
dynamically. Exactly how and when this happens is unspecified.
Commit adds two throttler implementations:
* An unlimited throttler that is no-op and never blocks.
* A throttler built around the mbus `DynamicThrottlePolicy` and defers
all window decisions to it.
Current config default is to use the unlimited throttler. Config changes
require a process restart.
Offers both polling and (timed, non-timed) blocking calls for acquiring
a throttle token. If the returned token is valid, the caller may proceed
to invoke the asynchronous operation.
The window slot taken up by a valid throttle token is implicitly freed up
when the token is destroyed.
|
| |
|
|
|
|
|
|
|
|
| |
be reported back from bucket executor.
- Treat remapping as an error.
- For lidspace compaction job iterator is reset and will be recreated on next invocation.
- For bucketmove th ebucket is rechecked and either discarded or restarted.
|
| |
|
| |
|
|
|
|
| |
order… "
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
compact the message objects.
- Compact StorageMessageAddress to 16 bytes by
- using reference to cluster name.
- Use small enums for protocol and node type.
- Avoid having StorageMessage as separate allocation.
- Avoid default values
|
| |
|
| |
|
|
|
|
|
|
|
|
|
| |
FileStorManager.
When turned on, the calling thread (e.g. FNET network thread when using Storage API RPC)
gets the next async message to handle (if any) as part of scheduling a storage message.
This async message is then handled by the calling thread immediately,
instead of going via a persistence thread.
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
| |
- Reduce code visibility.
- Clean up some unused members.
- Some code unification.
|
|
|
|
|
|
| |
- Let FileStorManager own and control the Component and PersistenceHandler
separately from the Persistence thread.
- Let FileStorManager allocate and control stripe assignment.
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The legacy bucket DB initialization logic was designed for the case
where bucket information was spread across potentially millions of
files residing on spinning rust drives. It was therefore async and
running in parallel with client operations, adding much complexity
in order to deal with a myriad of concurrency edge cases.
Replace this with a very simple, synchronous init method that expects the
provider to have the required information readily and cheaply available.
This effectively removes the concept of a node's "initializing" state,
moving directly from reported state Down to Up.
Even though a node still technically starts up in Initializing state,
we never end up reporting this to the Cluster Controller as the DB init
completes before the RPC server stack is set up.
Legacy bucket DB initializer code will be removed in a separate pass.
Also simplify bucket DB interface contract for mutating iteration,
indicating that it is done in an unspecified order.
|
| |
|
| |
|