| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
Makes it much easier to reason about which state transitions have been
made visible in the cluster, and which ones have just been internal
state transitions in the controller.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This is done as part of the SAFE REST API call to set the node state of a
storage node to ensure atomicity of the state change, reduce the number of
state changes, and minimize the time to complete the state changes.
The right way to think about the safe-set is then: In order to safely set a
storage node to (e.g.) maintenance, the distributor will also have to be set to
down. And so on for the various permutations of state transitions.
|
| |
|
|\
| |
| |
| |
| | |
vespa-engine/vekterli/re-enable-synchronous-set-node-state
Re-enable synchronous set node state with additional safeguards
|
| |
| |
| |
| |
| |
| | |
Prevents an unstable cluster from potentially holding up all
container request processing threads indefinitely.
Deadline errors are translated into HTTP 504 errors to REST API clients.
|
| |
| |
| |
| |
| |
| | |
We check both for master status and task failure, as we otherwise place
a potentially dangerous silent dependency on the task always failing itself
if the controller is not a master.
|
| |
| |
| |
| |
| |
| | |
Previously, a config-retired node marked as Down by the Orchestrator
would remain as Retired in the cluster state until the node was
actually taken down entirely.
|
|/ |
|
|\
| |
| |
| |
| | |
Conflicts:
vespajlib/src/main/java/com/yahoo/concurrent/lock/Locks.java
|
| |
| |
| |
| |
| |
| | |
Effectively reverts to legacy behavior while some more thinking is
done on how to deal with blocking requests during leader elections
and non-converging clusters.
|
| |
| |
| |
| |
| |
| | |
Avoids edge case where set-node-state requests sent to followers would
have their response delayed indefinitely due to controller not
publishing any versions that the task's ACK barrier could be released by.
|
|/ |
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
Removes dependency on having to invoke broadcastNewState before being
able to observe that all distributors are in sync. Invocations of
broadcastNewState are gated by a grace period between each time,
so unless this is done we get artificial delays before a synchronous
task can be considered complete.
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
| |
Used to enable synchronous operation for set-node-state calls, which
ensure that side-effects of the call are visible when the response
returns.
If controller leadership is lost before state is published, tasks will
be failed back to the client.
|
| |
|
|
|
|
| |
the infamous thread.interrupt.
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|\
| |
| | |
Safely set storage node to DOWN
|
| | |
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
Setting a storage node to DOWN is considered safe if it can be permantenly set
down (e.g. removed from the application):
- The node is RETIRED
- There are no managed buckets
|
| | |
|
| |
| |
| |
| |
| |
| | |
Previously could risk that state transition grace period would elide
write to ZooKeeper if state changes happened within previous grace
period.
|
|\ \
| | |
| | | |
Arnej/remove extra gitignore
|
| |/ |
|
|/
|
|
|
|
|
|
|
|
|
| |
Previously, the controller would not write the version to ZK unless the
version was published to at least one node. This could lead to problems
due to un-written version numbers being visible via the controller's REST
APIs. External observers could see versions that were not present in ZK
and that would not be stable across reelections. As a consequence, invariants
for strictly increasing version numbers would be violated from the
perspective of these external observers (in particular, our system test
framework).
|
| |
|
|
|
|
|
|
| |
- Removes Spec.getLocalHostName
- Removes distinction between listening- and connect- address for Spec
- Makes all usage of connect w/Spec specify hostname
|
| |
|
| |
|
|\
| |
| | |
Bratseth/indexed tensor
|
| | |
|
|/
|
|
|
|
|
|
|
|
|
| |
Using just the versioned cluster state instead can cause the code to
erroneously believe that it is seeing repeated reported state changes
for the first time. This happens when the diffs in the reported node
states are not in and by themselves enough to trigger a new cluster state
version containing the changes.
This can in turn spam the logs and event buffers until a new cluster state
has been versioned.
|
| |
|
|
|
| |
Cluster controller will now generate the new cluster state on-demand in a "pure functional" way instead of conditionally patching a working state over time. This makes understanding (and changing) the state generation logic vastly easier than it previously was.
|
|
|
|
|
|
| |
ip which does not resolve. This works around that problem by finding a resolvable
address (while still falling back to localhost if we only get ipv6 addresses,
as that causes other problems in docker containers).
|
| |
|