| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
| |
|
|
|
|
|
| |
Avoids some implicit conversions. Add `starts_with` to `vespalib::string`
and `vespalib::stringref` to allow drop-in replacement for Document API code.
|
| |
|
|
|
|
|
| |
Avoids some implicit conversions. Add `starts_with` to `vespalib::string`
and `vespalib::stringref` to allow drop-in replacement for Document API code.
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
consistent with respect to transport, supervisor, and mirror.
|
|
|
|
| |
proton.
|
|
|
|
|
|
| |
* should have same behavior in Java and C++
* extend unit tests to verify
* note various places where we want to change the default on Vespa 8 branch
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We both send requests and process replies in the context of a dedicated task executor pool.
However, MessageBus sending and reply receiving happens in the context of entirely
separate threads. If the backend responds very quickly to visitor requests (such as
if buckets are empty), this can leave us in the following awkward position:
1. Replies arrive from backend, open up the throttle window, reply handling
task gets pushed onto executor queue (but not yet executed).
2. Send loop below continuously get a free send slot, keeps sending visitors
and filling up the set of pending buckets in the progress token.
3. Since visitor session is busy-looping in the send task, reply processing is
consequently entirely starved until the MessageBus throttle window is bursting
at the seams. This can effectively nullify the effects of the throttling policy,
especially if it's dynamic. But a static throttle policy with a sufficiently
high max window size will also potentially cause a runaway visitor train since
the active window size keeps getting decreased by backend replies.
To get around this, we explicitly check for concurrently scheduled message handling
tasks from the transport layer, breaking the loop if at least one handler has been
scheduled. This also has the (positive) effect of draining all reply tasks before we
start sending more work downstream.
Since visitor session progress is edge-triggered and progresses exclusively by sending
new visitors in reply handling tasks, it's critical that we never end up in a situation
where we have no pending CreateVisitors (or scheduled tasks), or we risk effectively
hanging the session. We must therefore be very careful that we only exit the send loop
if we _know_ we have at least one pending task enqueued that will ensure session progress.
This is a subtle thread interaction issue and cannot readily be unit tested, but
manual testing has confirmed both the underlying issue and that the throttling policy
does not run amok after the fix has been applied.
|
|
|
|
| |
Not directly exposed, will be used in the case of heap dump analysis.
|
|\
| |
| | |
Deprecate config.subscription
|
| | |
|
|\ \
| | |
| | |
| | |
| | | |
vespa-engine/vekterli/deprecate-legacy-visitor-functionality
Deprecate legacy visitor functionality
|
| |/
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This has not been relevant since "orderdoc" was a thing, and it
was never really a thing in the first place.
Unfortunately, due to an oversight in how the backend propagates visitor
statistics, the internal usage of 2nd pass statistics cannot be fully
removed before Vespa 8 (where the backend is known to not set the
deprecated statistics fields).
|
|/
|
|
|
|
|
|
| |
The 1st/2nd pass functionality has been deprecated for a long time,
but unfortunately the documents/bytes visited stats have been wired
to be returned as part of 2nd phase statistics instead of the regular
higher-level fields. This commit changes this, but the serialization
will still have to remain in place until Vespa 8.
|
|
|
|
|
|
|
| |
at neglible cost that removes the race on size().
- Also use explicit AtomicReference to get an Immutable snapshot when doing a random sample.
- Done after observing IllegalArgumentException due to negative argument to Random.nextInt(bound).
|
|
|
|
|
| |
* these were stricter than in parent, but to simplify
we can just use compiler args from parent
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
transient errors
The `ContentPolicy` has a failure handling policy where more than _n_ error
replies (a small number in practice) will trigger an implicit random send instead
of using the cached cluster state. This is to force rediscovery of the actual
cluster state, and is useful if a node is bad but we're not sending feed to enough
other nodes to figure it out from them.
However, certain error codes may be used frequently by the content layer for
purposes that do _not_ indicate that a change in cluster state may have happened,
and should therefore not be counted as errors that may indicate a bad node:
* `ERROR_TEST_AND_SET_CONDITION_FAILED`: may happen for any mutating operation
that has an associated TaS condition. Technically an `APP_FATAL_ERROR` since
resending doesn't make sense.
* `ERROR_BUSY`: may happen for concurrent mutations and if distributors are
in the process of changing bucket ownership and the grace period hasn't
passed yet. Also sent if queues are full and client policy should back off
a bit. None of these are errors as per se.
|
| |
|
| |
|
| |
|
|\
| |
| | |
Arnej/hunting unit test fails 1
|
| | |
|
|/
|
|
|
|
| |
Routing all child types to a cluster a parent is added to
may be convenient for some users, but if it's not what you want
it is then harder to prevent it from happening.
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
| |
* For C++ code this introduces a "document::config" namespace, which will
sometimes conflict with the global "config" namespace.
* Move all forward-declarations of the types DocumenttypesConfig and
DocumenttypesConfigBuilder to a common header file.
|
| |
|
|
|
|
| |
(NPE messages changed with JDK 14+ helpful NPEs.)
|
| |
|
| |
|
| |
|
|
|
| |
Co-authored-by: Tor Brede Vekterli <vekterli@yahooinc.com>
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|