aboutsummaryrefslogtreecommitdiffstats
path: root/documentapi
Commit message (Collapse)AuthorAgeFilesLines
* add special DocumentOnly fieldset in Java as wellArne H Juul2022-02-155-0/+10
| | | | | | * should have same behavior in Java and C++ * extend unit tests to verify * note various places where we want to change the default on Vespa 8 branch
* Include needed header files.Tor Egge2022-02-141-0/+1
|
* Reduce code visibility and include only what you need from config library.Henning Baldersheim2022-02-067-32/+45
|
* Avoid visitor reply processing starvation and runaway throttlingTor Brede Vekterli2022-02-031-3/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We both send requests and process replies in the context of a dedicated task executor pool. However, MessageBus sending and reply receiving happens in the context of entirely separate threads. If the backend responds very quickly to visitor requests (such as if buckets are empty), this can leave us in the following awkward position: 1. Replies arrive from backend, open up the throttle window, reply handling task gets pushed onto executor queue (but not yet executed). 2. Send loop below continuously get a free send slot, keeps sending visitors and filling up the set of pending buckets in the progress token. 3. Since visitor session is busy-looping in the send task, reply processing is consequently entirely starved until the MessageBus throttle window is bursting at the seams. This can effectively nullify the effects of the throttling policy, especially if it's dynamic. But a static throttle policy with a sufficiently high max window size will also potentially cause a runaway visitor train since the active window size keeps getting decreased by backend replies. To get around this, we explicitly check for concurrently scheduled message handling tasks from the transport layer, breaking the loop if at least one handler has been scheduled. This also has the (positive) effect of draining all reply tasks before we start sending more work downstream. Since visitor session progress is edge-triggered and progresses exclusively by sending new visitors in reply handling tasks, it's critical that we never end up in a situation where we have no pending CreateVisitors (or scheduled tasks), or we risk effectively hanging the session. We must therefore be very careful that we only exit the send loop if we _know_ we have at least one pending task enqueued that will ensure session progress. This is a subtle thread interaction issue and cannot readily be unit tested, but manual testing has confirmed both the underlying issue and that the throttling policy does not run amok after the fix has been applied.
* Internally track number of buckets splits and merged for ProgressTokenTor Brede Vekterli2022-02-021-1/+7
| | | | Not directly exposed, will be used in the case of heap dump analysis.
* Merge pull request #21013 from vespa-engine/deprecate-config.subscriptionHarald Musum2022-02-023-0/+3
|\ | | | | Deprecate config.subscription
| * Add SuppressWarnings for removal of config.subscription types.gjoranv2022-02-023-0/+3
| |
* | Merge pull request #21001 from ↵Henning Baldersheim2022-01-316-8/+32
|\ \ | | | | | | | | | | | | vespa-engine/vekterli/deprecate-legacy-visitor-functionality Deprecate legacy visitor functionality
| * | Deprecate remnants of two-phase visiting APIs and usageTor Brede Vekterli2022-01-316-8/+32
| |/ | | | | | | | | | | | | | | | | | | This has not been relevant since "orderdoc" was a thing, and it was never really a thing in the first place. Unfortunately, due to an oversight in how the backend propagates visitor statistics, the internal usage of 2nd pass statistics cannot be fully removed before Vespa 8 (where the backend is known to not set the deprecated statistics fields).
* / Don't set deprecated fields when aggregating visitor statisticsTor Brede Vekterli2022-01-312-2/+4
|/ | | | | | | | The 1st/2nd pass functionality has been deprecated for a long time, but unfortunately the documents/bytes visited stats have been wired to be returned as part of 2nd phase statistics instead of the regular higher-level fields. This commit changes this, but the serialization will still have to remain in place until Vespa 8.
* - Synchronize when removing nodes not present anymore. This simplifies code ↵Henning Baldersheim2022-01-121-13/+13
| | | | | | | at neglible cost that removes the race on size(). - Also use explicit AtomicReference to get an Immutable snapshot when doing a random sample. - Done after observing IllegalArgumentException due to negative argument to Random.nextInt(bound).
* unify java warningsArne H Juul2022-01-061-7/+0
| | | | | * these were stricter than in parent, but to simplify we can just use compiler args from parent
* GC deprecated junit assertThat.Henning Baldersheim2021-12-211-9/+8
|
* Update ABI specTor Brede Vekterli2021-12-201-1/+31
|
* Don't trigger implicit ContentPolicy random send fallback on expected ↵Tor Brede Vekterli2021-12-204-26/+144
| | | | | | | | | | | | | | | | | | | | | transient errors The `ContentPolicy` has a failure handling policy where more than _n_ error replies (a small number in practice) will trigger an implicit random send instead of using the cached cluster state. This is to force rediscovery of the actual cluster state, and is useful if a node is bad but we're not sending feed to enough other nodes to figure it out from them. However, certain error codes may be used frequently by the content layer for purposes that do _not_ indicate that a change in cluster state may have happened, and should therefore not be counted as errors that may indicate a bad node: * `ERROR_TEST_AND_SET_CONDITION_FAILED`: may happen for any mutating operation that has an associated TaS condition. Technically an `APP_FATAL_ERROR` since resending doesn't make sense. * `ERROR_BUSY`: may happen for concurrent mutations and if distributors are in the process of changing bucket ownership and the grace period hasn't passed yet. Also sent if queues are full and client policy should back off a bit. None of these are errors as per se.
* Don't emit a bunch of debug logs by default in ContentPolicy unit testsTor Brede Vekterli2021-12-201-1/+1
|
* most unit tests can run with just 1 network threadArne H Juul2021-12-164-5/+5
|
* destroy more framesArne H Juul2021-12-161-0/+12
|
* Merge pull request #20536 from vespa-engine/arnej/hunting-unit-test-fails-1Jon Marius Venstad2021-12-161-1/+9
|\ | | | | Arnej/hunting unit test fails 1
| * stop standalone slobroks when test is finishedArne H Juul2021-12-151-1/+9
| |
* | Match document types in document selections exactly onlyJon Bratseth2021-12-141-6/+4
|/ | | | | | Routing all child types to a cluster a parent is added to may be convenient for some users, but if it's not what you want it is then harder to prevent it from happening.
* Add noexcept specifiers.Tor Egge2021-12-111-1/+1
|
* update ABI for generated buildersArne H Juul2021-12-091-0/+1
|
* update ABI for generated buildersArne H Juul2021-12-091-0/+2
|
* more descriptive name for header fileArne H Juul2021-12-023-3/+3
|
* track namespace move in documenttypes.defArne H Juul2021-12-023-0/+3
| | | | | | | * For C++ code this introduces a "document::config" namespace, which will sometimes conflict with the global "config" namespace. * Move all forward-declarations of the types DocumenttypesConfig and DocumenttypesConfigBuilder to a common header file.
* Use given messagebus config also when slobroks config is self-subJon Marius Venstad2021-11-111-1/+2
|
* Ignore NPE message in test.gjoranv2021-11-031-3/+5
| | | | (NPE messages changed with JDK 14+ helpful NPEs.)
* Separate return statement, add braces to nested ifJon Marius Venstad2021-10-281-1/+2
|
* Avoid double completion when transient error on overtimeJon Marius Venstad2021-10-281-14/+8
|
* Update 2019 Oath copyrights.gjoranv2021-10-271-1/+1
|
* ino -> ionJon Marius Venstad2021-10-271-1/+1
| | | Co-authored-by: Tor Brede Vekterli <vekterli@yahooinc.com>
* Skip to slice after progress token has been updated, after dist bit count changeJon Marius Venstad2021-10-262-3/+23
|
* Address reviewJon Marius Venstad2021-10-253-45/+54
|
* Remove unintended checkJon Marius Venstad2021-10-221-4/+0
|
* Allow specifying slices and sliceId in /document/v1Jon Marius Venstad2021-10-221-0/+4
|
* Allow slicing the bucket space for visitorsJon Marius Venstad2021-10-225-13/+197
|
* Fix warning in javadoc generation.Arnstein Ressem2021-10-071-2/+0
|
* Update Verizon Media copyright notices.gjoranv2021-10-075-5/+5
|
* Update 2018 copyright notices.gjoranv2021-10-075-5/+5
|
* Update 2017 copyright notices.gjoranv2021-10-07271-271/+271
|
* GC unused code.Henning Baldersheim2021-09-242-14/+0
|
* Exceptions yield no proper HTTP responseJon Marius Venstad2021-09-141-3/+2
|
* After the first try it is no longer the first try.Henning Baldersheim2021-09-091-1/+1
|
* No functional changesJon Bratseth2021-06-281-8/+8
|
* drop empty buffers instead of using small buffersHåvard Pettersen2021-06-151-1/+1
|
* admin/slobrok.0 does not always exist ....... anymore.Henning Baldersheim2021-05-271-1/+1
|
* Avoid coredump on bad/empty config.Henning Baldersheim2021-05-251-2/+3
|
* Update ABI specJon Marius Venstad2021-05-191-0/+3
|
* Add deadline to DocumentOperationParameters and use in MessageBusAsyncSessionJon Marius Venstad2021-05-196-11/+67
|