diff options
| author | Tor Brede Vekterli <vekterli@verizonmedia.com> | 2021-02-10 14:36:23 +0100 |
|---|---|---|
| committer | Tor Brede Vekterli <vekterli@verizonmedia.com> | 2021-02-10 14:45:11 +0100 |
| commit | dd230a258ba8896460a1b406ec4271622f2098f4 (patch) | |
| tree | 47f6419c670a948079e78c99ad5d359654b558c2 /configdefinitions | |
| parent | 5f4dcd3002d005ef91ed8721fcf5ca5afa78317c (diff) | |
Support configurable feed block hysteresis on the cluster controller
Adds an absolute number delta that is subtracted from the feed block limit
when a node has a resource already in feed blocked state. This means that
there's a lower watermark threshold that must be crossed before feeding
can be unblocked. Avoids flip-flopping between block states.
Default is currently 0.0, i.e. effectively disabled. To be modified
later for system tests and trial roll-outs.
A couple of caveats with the current implementation:
* The cluster state is not recomputed automatically when just the hysteresis
threshold is crossed, so the description will be out of date on the
content nodes. However, if any other feed block event happens (or the
hysteresis threshold is crossed), the state will be recomputed as expected.
This does not affect correctness, since the feed is still to be blocked.
* A node event remove/add pair is emitted for feed block status when the
hysteresis threshold is crossed and there's a cluster state recomputation.
Diffstat (limited to 'configdefinitions')
| -rw-r--r-- | configdefinitions/src/vespa/fleetcontroller.def | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/configdefinitions/src/vespa/fleetcontroller.def b/configdefinitions/src/vespa/fleetcontroller.def index 3c88639d09d..d2d746363f0 100644 --- a/configdefinitions/src/vespa/fleetcontroller.def +++ b/configdefinitions/src/vespa/fleetcontroller.def @@ -197,3 +197,11 @@ enable_cluster_feed_block bool default=false # The keys used must match the similar keys in the host info JSON structure. # All limits are numbers between 0.0 and 1.0. cluster_feed_block_limit{} double + +# To avoid having the cluster feed block state flip-flop from nodes that are hovering +# just around the feed block limits, this noise threshold implicitly makes the +# feed block limit value _lower_ for a resource that is already exhausted. I.e. the +# node must reach a lower resource usage than the limit for feed to be unblocked. +# This is in absolute numbers, so 0.01 implies that a block limit of 0.8 effectively +# becomes 0.79 for an already blocked node. +cluster_feed_block_noise_level double default=0.0 |