summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #20000 from ↵Tor Brede Vekterli2021-11-1542-107/+611
|\ | | | | | | | | vespa-engine/vekterli/support-unordered-merge-chaining Add configurable support for unordered merge forwarding [run-systemtest]
| * Address PR commentsTor Brede Vekterli2021-11-153-11/+14
| | | | | | | | | | | | * Add `from_distributor()` utility function to `MergeBucketCommand` * Simplify boolean expression by moving sub-expression to own statement * Improve wording of config parameter
| * Add configurable support for unordered merge forwardingTor Brede Vekterli2021-11-1242-105/+606
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Historically the MergeThrottler component has required a deterministic forwarding of merges between nodes in strictly increasing distribution key order. This is to avoid distributed deadlocks caused by ending up with two or more nodes waiting for each other to release merge resources, where releasing one depends on releasing the other. This works well, but has the downside that there's an inherent pressure of merges towards nodes with lower distribution keys. These often become a bottleneck. This commit lifts this ordering restriction, by allowing forwarded, unordered merges to immediately enter the active merge window. By doing this we remove the deadlock potential, since nodes will longer be waiting on resources freed by other nodes. Since the legacy MergeThrottler has a lot of invariant checking around strictly increasing merge chains, we only allow unordered merges to be scheduled towards node sets where _all_ nodes are on a Vespa version that explicitly understands unordered merges (and thus do not self- obliterate upon seeing one). To communicate this, full bucket fetches will now piggy-back version-specific feature sets as part of the response protocol. Distributors then aggregate this information internally.
* | Merge pull request #20013 from ↵gjoranv2021-11-158-64/+14
|\ \ | | | | | | | | | | | | vespa-engine/configure-release-version-safely-via-property Use a custom property for setting relase version for clients.
| * | Use a custom property for setting relase version for clients.gjoranv2021-11-158-64/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | - Always set release version via maven-compiler-plugin, instead of maven property which is overridden by compiler-plugin config. - Using a custom property with self-explanatory name makes comments redundant. - Remove explicit jdkToolchain config, as these modules no longer compile with jdk pre 9, due to the --release flag.
* | | Merge pull request #20019 from vespa-engine/freva/fix-reboot-genHåkon Hallingstad2021-11-151-10/+10
|\ \ \ | | | | | | | | Update internal reboot/restart generations regardless of state
| * | | Update internal reboot/restart generations regardless of stateValerij Fredriksen2021-11-151-10/+10
| | | |
* | | | Merge pull request #20020 from ↵Valerij Fredriksen2021-11-151-2/+4
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | vespa-engine/hakonhall/improve-error-message-with-unexpected-command Improve error message with unexpected command
| * | | | Improve error message with unexpected commandHåkon Hallingstad2021-11-151-2/+4
| |/ / /
* | | | Merge pull request #20015 from vespa-engine/hmusum/remove-unused-rpm-methodHenning Baldersheim2021-11-152-18/+3
|\ \ \ \ | |/ / / |/| | | Remove unused method
| * | | Remove unused methodHarald Musum2021-11-152-18/+3
| |/ / | | | | | | | | | | | | RPC method filedistribution.setFileReferencesToDownload is only used between config servers, remove config proxy implementation
* | | Merge pull request #20016 from vespa-engine/freva/remove-oldValerij Fredriksen2021-11-152-38/+8
|\ \ \ | | | | | | | | Remove old devtools paths
| * | | Remove old devtools pathsValerij Fredriksen2021-11-152-38/+8
| |/ /
* | | Merge pull request #20014 from vespa-engine/bjorncs/jdk17Bjørn Christian Seime2021-11-153-0/+3
|\ \ \ | | | | | | | | Allow reflective access to sun.security.util
| * | | Allow reflective access to sun.security.utilBjørn Christian Seime2021-11-153-0/+3
| |/ /
* | | Merge pull request #20011 from vespa-engine/freva/simplifyValerij Fredriksen2021-11-152-8/+2
|\ \ \ | | | | | | | | Simplify deployedDirectly serialization
| * | | Simplify deployedDirectly serializationValerij Fredriksen2021-11-152-8/+2
| | | |
* | | | Merge pull request #20012 from vespa-engine/hmusum/log-session-idHenning Baldersheim2021-11-153-2/+7
|\ \ \ \ | | | | | | | | | | Log session id when preprocessing application package fails
| * | | | Log session id when preprocessing application package failsHarald Musum2021-11-153-2/+7
| | | | |
* | | | | Merge pull request #20009 from ↵Henning Baldersheim2021-11-155-2/+152
|\ \ \ \ \ | |_|_|/ / |/| | | | | | | | | | | | | | vespa-engine/toregge/optimize-lid-allocator-unregister-lids Optimize proton::documentmetastore::LidAllocator::unregister_lids().
| * | | | Optimize proton::documentmetastore::LidAllocator::unregister_lids().Tor Egge2021-11-155-2/+152
| |/ / /
* | | | Merge pull request #20010 from vespa-engine/ensure-release-1.8gjoranv2021-11-151-23/+1
|\ \ \ \ | |/ / / |/| | | Ensure release 1.8
| * | | Ensure building with release 8 as default.gjoranv2021-11-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - The 'maven.compiler.release' property is overridden by explicit 'release' configuration in maven-compiler-plugin. (While the property seems to override explicit source/target config in the plugin.)
| * | | Remove unused profilesgjoranv2021-11-151-22/+0
|/ / /
* | | Merge pull request #20008 from vespa-engine/use-release-for-vespa-hadoopgjoranv2021-11-151-2/+1
|\ \ \ | | | | | | | | Use release flag to ensure jdk8.
| * | | Use release flag to ensure jdk8.gjoranv2021-11-151-2/+1
|/ / / | | | | | | | | | - release flag in parent pom overrides source/target in this pom.
* | | Merge pull request #20007 from vespa-engine/bratseth/propagate-all-errorsHenning Baldersheim2021-11-151-2/+4
|\ \ \ | | | | | | | | Propagate exceptions from task execution to merged
| * | | Propagate exceptions from task execution to mergedJon Bratseth2021-11-151-2/+4
| |/ /
* | | Merge pull request #20004 from ↵Arnstein Ressem2021-11-151-1/+1
|\ \ \ | | | | | | | | | | | | | | | | vespa-engine/jonmv/look-for-cloud-tenant-base-for-compile-version Update artifact to look for in maven repo, for compile version
| * | | Update artifact to look for in maven repo, for compile versionJon Marius Venstad2021-11-151-1/+1
| | | |
* | | | Merge pull request #20006 from ↵Henning Baldersheim2021-11-151-0/+3
|\ \ \ \ | |_|/ / |/| | | | | | | | | | | vespa-engine/hmusum/log-number-of-active-threads-2 Log number of active threads
| * | | Log number of active threadsHarald Musum2021-11-151-0/+3
|/ / /
* | | Merge pull request #20003 from vespa-engine/hmusum/download-loggingHenning Baldersheim2021-11-152-12/+11
|\ \ \ | | | | | | | | Improve file download logging a bit
| * | | Use correct variable in log messageHarald Musum2021-11-151-2/+2
| | | |
| * | | Improve file download logging a bitHarald Musum2021-11-152-11/+10
| | | |
* | | | Merge pull request #20002 from ↵Geir Storli2021-11-156-117/+291
|\ \ \ \ | |/ / / |/| | | | | | | | | | | vespa-engine/toregge/unit-test-lid-allocator-unregister-lids Unit test proton::documentmetastore::LidAllocator::unregister_lids().
| * | | Unit test proton::documentmetastore::LidAllocator::unregister_lids().Tor Egge2021-11-146-117/+291
|/ / /
* | | Merge pull request #19988 from ↵v7.500.40Henning Baldersheim2021-11-1212-18/+159
|\ \ \ | | | | | | | | | | | | | | | | vespa-engine/balder/less-eager-compaction-during-retirement When node is retired we can relax compaction strategy as we have peaked
| * | | Add test that compaction config is reflected properly in document metastore ↵Henning Baldersheim2021-11-121-0/+31
| | | | | | | | | | | | | | | | and attributes.
| * | | Test that up -> retired -> up is reflectd correctly in document subdbs.Henning Baldersheim2021-11-122-2/+26
| | | |
| * | | Correct config for retired nodes both after reconfig and retired -> up for ↵Henning Baldersheim2021-11-123-16/+28
| | | | | | | | | | | | | | | | both attributes and document meta store.
| * | | Consider retirement on every cluster change and reconfig.Henning Baldersheim2021-11-123-16/+32
| | | |
| * | | When node is retired we can relax compaction strategy as we have peakedHenning Baldersheim2021-11-1210-14/+72
| |/ / | | | | | | | | | in memory usage and will go reduce from here.
* | | Merge pull request #19996 from ↵Henning Baldersheim2021-11-121-0/+1
|\ \ \ | | | | | | | | | | | | | | | | vespa-engine/toregge/add-optional-include-for-bucketdb proton::BucketDB uses std::optional. Add needed include.
| * | | proton::BucketDB uses std::optional. Add needed include.Tor Egge2021-11-121-0/+1
| | | |
* | | | Merge pull request #19998 from vespa-engine/ldalves/device-authorization-flowLeandro Alves2021-11-1213-51/+888
|\ \ \ \ | | | | | | | | | | Ldalves/device authorization flow
| * | | | Wiring of Auth0 access token to cloud specific HTTP requestsBjørn Christian Seime2021-11-127-162/+188
| | | | |
| * | | | update WithIssuerLeandro Alves2021-11-081-2/+1
| | | | |
| * | | | first draft for the device flow supportLeandro Alves2021-11-0610-32/+844
| | | | |
* | | | | Merge pull request #19999 from ↵Geir Storli2021-11-121-0/+4
|\ \ \ \ \ | |_|_|_|/ |/| | | | | | | | | | | | | | vespa-engine/toregge/add-class-comment-for-proton-bucketdb-remove-batch-entry Add class comment for proton::bucketdb::RemoveBatchEntry.