| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
| |
|
| |
|
|
|
|
| |
This reverts commit 003f019d7579e49f4ec7609ef8eac26ada6ae753.
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If enabled, garbage collection is performed in two phases (metadata
gathering and deletion) instead of just a single phase. Two-phase GC
allows for ensuring the same set of documents is deleted across all
nodes and explicitly takes write locks on the distributor to prevent
concurrent feed ops to GC'd documents from potentially creating
inconsistencies.
Two-phase GC is only used _iff_ all replica content nodes support
the feature _and_ it's enabled in config. An additional field has
been added to the feature negotiation functionality to communicate
support from content nodes to distributors.
|
|
|
|
|
|
|
| |
This better mirrors how Proton actually works, since it's not a multi
version store. Since only the highest timestamped entry for a document
is the one that is ever considered on a node, there's no point in
storing an explicit tombstone that cannot be referenced.
|
|
|
|
|
| |
Feels more intuitive to have a tuple that implies "document foo at timestamp bar"
rather than the current inverse of "timestamp bar with document foo".
|
| |
|
|
|
|
| |
- Cut dependency to persistencetypes for searchlib.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This prevents the following race condition where the bucket mover logic
fails to notify the content layer that the bucket sub DB status has changed
for a particular bucket:
1. Bucket state is changed over SPI, a mover is created and registered and
a BucketTask is scheduled onto the persistence queues to actually do the
document reads and finalize the move.
2. Before the bucket task is executed, bucket state is changed again over
the SPI. A new mover is created, the old one is cancelled (tagging
mover as not consistent) and another BucketTask is scheduled onto
the persistence queues. Note: the old task still remains.
3. Old bucket task is executed and performs the actual document moving
despite being cancelled. No notification is done towards the content
layer since the mover was tagged as not being consistent.
4. New bucket task is executed and tries to move the same document set
as the old mover. Since the documents are no longer present in the
source document DB, the moves fail. This tags the mover as inconsistent
and no notification is done. Bucket is automatically rechecked, but
since all docs are already moved away there is nothing more to do
and no subsequent mover is created. This means the "should notify?"
edge is not triggered and the content layer remains blissfully unaware
of any sub DB changes.
This commit simply changes cancellation to actually inhibit document moves
from taking place. This lets the preempting mover successfully complete its
moves, thus triggering the notify-edge as expected.
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
| |
that we will be fully synced on the next pull when it happens in the singleCommitter thread.
That allows for further simplification.
|
| |
|
| |
|
|
|
|
| |
- Consistently use DocEntryList as type for std::vector<spi::DocEntry::UP>
|
|
|
|
| |
and gid
|
| |
|
| |
|
|
|
|
|
|
| |
instead of an mutant.
Also add tests for the different variations a DocEntry can have.
|
| |
|
| |
|
|
|
|
|
|
|
| |
* For C++ code this introduces a "document::config" namespace, which will
sometimes conflict with the global "config" namespace.
* Move all forward-declarations of the types DocumenttypesConfig and
DocumenttypesConfigBuilder to a common header file.
|
|
|
|
|
|
|
|
|
|
|
| |
Only skip deactivating buckets if the entire _node_ is marked as
maintenance state, i.e. the node has maintenance state across all
bucket spaces provided in the bundle. Otherwise treat the state
transition as if the node goes down, deactivating all buckets.
Also ensure that the bucket deactivation logic above the SPI is
identical to that within Proton. This avoids bucket DBs getting
out of sync between the two.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, entering maintenance state would implicitly deactivate
all buckets on the searchnode and cause empty responses to be returned
for searches.
However, container query dispatch uses async health pings to decide
which nodes to route queries to, so it would be possible for a node to
still be used for queries for a few seconds until the ping discovered
that the node should not be used. In the case of multiple groups without
multiple ready replicas within the group, this would cause transient
coverage loss since the dispatcher would not realize it should route
queries to other groups instead.
With this commit, maintenance edge behavior is changed as follows:
- Buckets are _not_ deactivated when going from an available state
to the maintenance state. However, they _are_ deactivate when going
from maintenance state to an available state in order to avoid transient
query duplicates immediately after the change.
- Searches are executed as normal instead of returning empty replies
when the node is in maintenance state.
The following behavior is intentionally _not_ changed:
- The search interface is still marked as offline when in maintenance
state, as this signals that the node should be taken out of rotation.
In particular, it's critical that the RPC health ping response is
explicitly tagged as having zero active docs when the search interface
is offline, even though many buckets may now actually be active.
Otherwise, queries would not be gracefully drained from the node.
|
|
|
|
| |
[run-systemtest]"
|
|
|
|
|
|
|
|
|
|
|
| |
Only skip deactivating buckets if the entire _node_ is marked as
maintenance state, i.e. the node has maintenance state across all
bucket spaces provided in the bundle. Otherwise treat the state
transition as if the node goes down, deactivating all buckets.
Also ensure that the bucket deactivation logic above the SPI is
identical to that within Proton. This avoids bucket DBs getting
out of sync between the two.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, entering maintenance state would implicitly deactivate
all buckets on the searchnode and cause empty responses to be returned
for searches.
However, container query dispatch uses async health pings to decide
which nodes to route queries to, so it would be possible for a node to
still be used for queries for a few seconds until the ping discovered
that the node should not be used. In the case of multiple groups without
multiple ready replicas within the group, this would cause transient
coverage loss since the dispatcher would not realize it should route
queries to other groups instead.
With this commit, maintenance edge behavior is changed as follows:
- Buckets are _not_ deactivated when going from an available state
to the maintenance state. However, they _are_ deactivate when going
from maintenance state to an available state in order to avoid transient
query duplicates immediately after the change.
- Searches are executed as normal instead of returning empty replies
when the node is in maintenance state.
The following behavior is intentionally _not_ changed:
- The search interface is still marked as offline when in maintenance
state, as this signals that the node should be taken out of rotation.
In particular, it's critical that the RPC health ping response is
explicitly tagged as having zero active docs when the search interface
is offline, even though many buckets may now actually be active.
Otherwise, queries would not be gracefully drained from the node.
|
| |
|
|
|
|
|
|
| |
bucket before waiting for the replies.
Prepare RemoveResult to contain more replies.
|
|
|
|
|
| |
bucket doesn't exist:
setActiveState(), put(), remove() creates bucket if it doesn't already exist.
|
|
|
|
|
|
|
| |
bucket doesn't exist:
* getBucketInfo() returns success with empty bucket info
* createIterator() returns success
* iterate() returns empty complete result.
|
| |
|
| |
|