| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, entering maintenance state would implicitly deactivate
all buckets on the searchnode and cause empty responses to be returned
for searches.
However, container query dispatch uses async health pings to decide
which nodes to route queries to, so it would be possible for a node to
still be used for queries for a few seconds until the ping discovered
that the node should not be used. In the case of multiple groups without
multiple ready replicas within the group, this would cause transient
coverage loss since the dispatcher would not realize it should route
queries to other groups instead.
With this commit, maintenance edge behavior is changed as follows:
- Buckets are _not_ deactivated when going from an available state
to the maintenance state. However, they _are_ deactivate when going
from maintenance state to an available state in order to avoid transient
query duplicates immediately after the change.
- Searches are executed as normal instead of returning empty replies
when the node is in maintenance state.
The following behavior is intentionally _not_ changed:
- The search interface is still marked as offline when in maintenance
state, as this signals that the node should be taken out of rotation.
In particular, it's critical that the RPC health ping response is
explicitly tagged as having zero active docs when the search interface
is offline, even though many buckets may now actually be active.
Otherwise, queries would not be gracefully drained from the node.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, entering maintenance state would implicitly deactivate
all buckets on the searchnode and cause empty responses to be returned
for searches.
However, container query dispatch uses async health pings to decide
which nodes to route queries to, so it would be possible for a node to
still be used for queries for a few seconds until the ping discovered
that the node should not be used. In the case of multiple groups without
multiple ready replicas within the group, this would cause transient
coverage loss since the dispatcher would not realize it should route
queries to other groups instead.
With this commit, maintenance edge behavior is changed as follows:
- Buckets are _not_ deactivated when going from an available state
to the maintenance state. However, they _are_ deactivate when going
from maintenance state to an available state in order to avoid transient
query duplicates immediately after the change.
- Searches are executed as normal instead of returning empty replies
when the node is in maintenance state.
The following behavior is intentionally _not_ changed:
- The search interface is still marked as offline when in maintenance
state, as this signals that the node should be taken out of rotation.
In particular, it's critical that the RPC health ping response is
explicitly tagged as having zero active docs when the search interface
is offline, even though many buckets may now actually be active.
Otherwise, queries would not be gracefully drained from the node.
|
