aboutsummaryrefslogtreecommitdiffstats
path: root/document/src/main/java/com/yahoo/document/serialization
Commit message (Collapse)AuthorAgeFilesLines
* Add support for serializing document update to jsonHarald Musum2024-03-122-2/+4
|
* Support serializing document remove operation to json formatHarald Musum2024-03-112-0/+8
|
* Minor cleanups of serialization versions and buffer sizesTor Brede Vekterli2024-03-042-2/+2
|
* Add new Protobuf-based MessageBus DocumentAPI protocolTor Brede Vekterli2024-02-163-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds an entirely new implementation of the internal MessageBus DocumentAPI protocol, which shall be functionally 1-to-1 compatible with the existing legacy protocol. New protobuf schemas have been added to the top-level documentapi module, which are separated into different domains of responsibility: * CRUD messages * Visiting messages * Data inspection messages As well as a schema for shared, common message types. Both C++ and Java protocol implementations separate serialization and deserialization into a codec abstraction per message type, which hides the boilerplate required for Protobuf buffer management. The Java version is a tad more verbose due to generics type-erasure. This protocol does _not_ currently support lazy (de-)serialization in Java, as the existing mechanisms for doing so are inherently tied to the legacy protocol version. Performance tests will decide if we need to introduce such functionality to the new protocol version. To avoid having the new protocol go live in production, this commit changes the semantics of how MessageBus version reporting works (at least for the near future); instead of reporting the current Vespa _release_ version, it reports the highest supported _protocol_ version. This lets us conditionally enable the new protocol by reporting a MessageBus version greater than or equal to the protocol version _iff_ the protocol should be active. The new protocol is disabled by default. Other changes: * Protocol tests have been moved up one package directory level to be aligned with the actual package of the classes they test. This allows for using package-protected constructors in the serialization tests. * `DocumentDeserializer` now exposes the underlying document type repo/manager. This is done to detangle `Document`/`DocumentUpdate` deserialization from the underlying wire buffer management. * `RemoveLocationMessage` at long last contains a bucket space, which was forgotten when we initially added this concept to the other messages, and where the pain of adding it in later was too big (not so anymore!). Unit tests for both C++ and Java have been hoisted from the legacy test suite, cleaned up and extended with additional cases. The C++ tests use the old unit test kit and should receive a good follow-up washing and GTest-rewrite. **Important**: due to how MessageBus protocol versioning works, the final protocol version is _not_ yet decided, as setting it requires syncing against our build systems. A follow-up commit will assign the final version as well as include all required binary test files.
* - Only call getKey once.Henning Baldersheim2024-01-161-16/+14
| | | | - Just use a presized array of native ints, instead of LinkedList.
* Update copyrightJon Bratseth2023-10-0926-26/+26
|
* Add "create non-existing cells" flag to TensorModifyUpdate.Geir Storli2023-08-252-3/+30
| | | | | | | When this is true, non-existing cells in the input tensor is created before applying the modify update. The default cell value is 0.0 for REPLACE and ADD operations, and 1.0 for MULTIPLY operation.
* Use base64 encoding with padding also for get/visitHenning Baldersheim2023-05-031-2/+1
|
* Skip padding of base64 encoded data.Henning Baldersheim2023-04-191-2/+4
|
* Build with jdk20Jon Bratseth2023-04-171-1/+1
|
* Deprecate xml methodsHenning Baldersheim2023-01-273-339/+3
|
* * refactor to get just one method for struct readingArne Juul2023-01-191-50/+17
| | | | * also, use two simple arrays instead of arraylist of tuples
* no real need to use a temporary bufferArne Juul2023-01-191-25/+6
|
* check complete span node typeArne Juul2023-01-181-6/+6
| | | | * this matches actual serialization format and C++ implementation
* Support direct tensor renderingJon Bratseth2023-01-141-7/+4
|
* Use JsonFormat for JSON renderingJon Bratseth2023-01-131-162/+29
|
* Don't warn when 1 node is downJon Bratseth2022-12-291-3/+2
|
* GC deprecation warnings.Henning Baldersheim2022-06-082-11/+16
|
* Further GC document level compression. Avoids a buffer copy that is no ↵Henning Baldersheim2022-06-082-44/+9
| | | | longer relevant.
* cleanup some unneccessary cruftArne H Juul2022-06-082-2/+0
|
* expand tabs and reindentArne H Juul2022-03-241-6/+6
|
* avoid using getHeader() in serialization codeArne H Juul2022-02-164-27/+31
|
* GC deprecated junit assertThat.Henning Baldersheim2021-12-211-1/+0
|
* always try reading annotation payloadArne H Juul2021-12-081-6/+11
| | | | | | | | * even if the serialized datatype ID doesn't match our config, it's quite possible the payload is OK. * does it really make sense to "handle it gracefully" and just ignore the payload? This seems wrong to me, but keep old behavior for now.
* GC dompression support in java.Henning Baldersheim2021-11-101-17/+3
|
* Update 2017 copyright notices.gjoranv2021-10-0727-27/+27
|
* keep serialized tensor in TensorFieldValueArne Juul2021-02-012-10/+9
| | | | | * do lazy deserialization in getTensor() and getTensorType() * avoid re-serializing when forwarding a TensorFieldValue
* Extend tensor remove update to support not fully specified addresses and ↵Geir Storli2020-11-172-4/+6
| | | | | | | update JSON parser. Previously, all the sparse dimensions of the sparse or mixed tensor type (to remove from) had to be specified in the addresses to remove.
* Store scaling events in ZooKeeperJon Bratseth2020-10-221-1/+1
|
* - Removing body struct from our own usage.Henning Baldersheim2020-06-164-29/+8
| | | | | - Deprecate public methods using body struct. - Update expected generated config.
* Header/body does not exist anymore. Deprecate and remove own internal usage.Henning Baldersheim2020-01-302-2/+2
|
* Remove use of commons-lang from documentBjørn Christian Seime2020-01-061-3/+3
|
* Remove "Vespa 6" from error messagesgjoranv2019-09-232-6/+6
|
* Deprecate getId(version) and remove usage.Henning Baldersheim2019-08-221-2/+2
|
* Update to require atleast vespa 6 version.Henning Baldersheim2019-08-212-697/+0
|
* Remove v6 and v7 from java side.Henning Baldersheim2019-08-213-995/+26
|
* Revert "Revert "doc: -> id:""Henning Baldersheim2019-08-211-3/+2
|
* Revert "doc: -> id:"Henning Baldersheim2019-08-211-2/+3
|
* doc: -> id:Henning Baldersheim2019-08-191-3/+2
| | | | | Unify documentid java/cpp GC old formats v6 and v7
* Add a binary format too.Henning Baldersheim2019-04-252-12/+12
|
* GC unused codeHenning Baldersheim2019-04-242-10/+0
|
* Cleanup of tensor updates - JavaLester Solbakken2019-02-211-1/+1
|
* Don't convert tensor add update type to sparse dimensionsLester Solbakken2019-02-201-3/+1
|
* Non-functional changes onlyLester Solbakken2019-02-201-2/+2
|
* TensorModifyUpdate support for mixed tensorsLester Solbakken2019-02-201-1/+4
|
* TensorRemoveUpdate support for mixed tensorsLester Solbakken2019-02-201-4/+7
|
* Add serialization for TensorRemoveUpdateLester Solbakken2019-02-156-6/+49
|
* Implement serialization and de-serialization for TensorAddUpdate.Geir Storli2019-02-076-3/+41
|
* Support tensor modify update on dense tensors.Geir Storli2019-01-311-4/+5
| | | | Always use a sparse tensor to hold the cells in a tensor modify update.
* Adjust local variable names (operand -> tensor).Tor Egge2019-01-302-5/+5
| | | | | Remove unneded comments. Don't use toString() to check for correct deserialization.