diff options
author | Tor Brede Vekterli <vekterli@vespa.ai> | 2024-02-01 15:25:45 +0000 |
---|---|---|
committer | Tor Brede Vekterli <vekterli@vespa.ai> | 2024-02-16 13:42:49 +0000 |
commit | e0195ce27f47717ad5ba59ea59ab027de31d703f (patch) | |
tree | 44a13aa38fcbf95b23df91e8051c1d2b8bb2f688 /messagebus/src/vespa | |
parent | 42b1512d4913778dde06ebe0b1a08257ead3155a (diff) |
Add new Protobuf-based MessageBus DocumentAPI protocol
This adds an entirely new implementation of the internal MessageBus
DocumentAPI protocol, which shall be functionally 1-to-1 compatible
with the existing legacy protocol.
New protobuf schemas have been added to the top-level documentapi
module, which are separated into different domains of responsibility:
* CRUD messages
* Visiting messages
* Data inspection messages
As well as a schema for shared, common message types.
Both C++ and Java protocol implementations separate serialization and
deserialization into a codec abstraction per message type, which
hides the boilerplate required for Protobuf buffer management. The
Java version is a tad more verbose due to generics type-erasure.
This protocol does _not_ currently support lazy (de-)serialization
in Java, as the existing mechanisms for doing so are inherently tied
to the legacy protocol version. Performance tests will decide if we
need to introduce such functionality to the new protocol version.
To avoid having the new protocol go live in production, this commit
changes the semantics of how MessageBus version reporting works (at
least for the near future); instead of reporting the current Vespa
_release_ version, it reports the highest supported _protocol_ version.
This lets us conditionally enable the new protocol by reporting a
MessageBus version greater than or equal to the protocol version _iff_
the protocol should be active.
The new protocol is disabled by default.
Other changes:
* Protocol tests have been moved up one package directory level to
be aligned with the actual package of the classes they test. This
allows for using package-protected constructors in the serialization
tests.
* `DocumentDeserializer` now exposes the underlying document type
repo/manager. This is done to detangle `Document`/`DocumentUpdate`
deserialization from the underlying wire buffer management.
* `RemoveLocationMessage` at long last contains a bucket space,
which was forgotten when we initially added this concept to the
other messages, and where the pain of adding it in later was
too big (not so anymore!).
Unit tests for both C++ and Java have been hoisted from the legacy
test suite, cleaned up and extended with additional cases. The C++
tests use the old unit test kit and should receive a good follow-up
washing and GTest-rewrite.
**Important**: due to how MessageBus protocol versioning works, the
final protocol version is _not_ yet decided, as setting it requires
syncing against our build systems. A follow-up commit will assign
the final version as well as include all required binary test files.
Diffstat (limited to 'messagebus/src/vespa')
-rw-r--r-- | messagebus/src/vespa/messagebus/network/rpcnetwork.cpp | 21 | ||||
-rw-r--r-- | messagebus/src/vespa/messagebus/network/rpcnetwork.h | 6 |
2 files changed, 25 insertions, 2 deletions
diff --git a/messagebus/src/vespa/messagebus/network/rpcnetwork.cpp b/messagebus/src/vespa/messagebus/network/rpcnetwork.cpp index cacd18430a7..f626e2c325b 100644 --- a/messagebus/src/vespa/messagebus/network/rpcnetwork.cpp +++ b/messagebus/src/vespa/messagebus/network/rpcnetwork.cpp @@ -18,6 +18,8 @@ #include <vespa/vespalib/stllike/asciistream.h> #include <vespa/vespalib/util/size_literals.h> #include <vespa/vespalib/util/stringfmt.h> +#include <cstdlib> +#include <string_view> #include <thread> #include <vespa/log/log.h> @@ -25,6 +27,7 @@ LOG_SETUP(".rpcnetwork"); using vespalib::make_string; using namespace std::chrono_literals; +using namespace std::string_view_literals; namespace mbus { @@ -148,10 +151,26 @@ RPCNetwork::flushTargetPool() _targetPool->flushTargets(true); } +namespace { + +[[nodiscard]] vespalib::Version derive_supported_protocol_version() { + // TODO remove this hilariously leaky abstraction once protobuf protocol is the default :D + // Disallow-version MUST be lower than that used as a protocol lower bound in documentprotocol.cpp + // and the exact same as that used in Java for the same purposes. Or else! + const char* maybe_env_val = getenv("VESPA_MBUS_DOCUMENTAPI_USE_PROTOBUF"); + if (maybe_env_val && (("true"sv == maybe_env_val) || ("yes"sv == maybe_env_val))) { + return {8, 304}; // _Allows_ new protobuf protocol + } + return {8, 303}; // _Disallows_ new protobuf protocol +} + +} + const vespalib::Version & RPCNetwork::getVersion() const { - return vespalib::Vtag::currentVersion; + static vespalib::Version reported_version = derive_supported_protocol_version(); + return reported_version; } void diff --git a/messagebus/src/vespa/messagebus/network/rpcnetwork.h b/messagebus/src/vespa/messagebus/network/rpcnetwork.h index 05ccaecb2c5..40590d4545f 100644 --- a/messagebus/src/vespa/messagebus/network/rpcnetwork.h +++ b/messagebus/src/vespa/messagebus/network/rpcnetwork.h @@ -98,11 +98,15 @@ private: protected: /** - * Returns the version of this network. This gets called when the + * Returns the (protocol) version of this network. This gets called when the * "mbus.getVersion" method is invoked on this network, and is separated * into its own function so that unit tests can override it to simulate * other versions than current. * + * Note that this version reflects the highest supported protocol version, and + * is not necessarily 1-1 with the actual Vespa release version of the + * underlying binary. + * * @return The version to claim to be. */ virtual const vespalib::Version &getVersion() const; |