Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Move Jackson util from vespajlib to container-core. | Henning Baldersheim | 2023-11-24 | 3 | -3/+3 |
| | |||||
* | jackson 2.16 changes some of its default settings so we consolidate our use ↵ | Henning Baldersheim | 2023-11-23 | 3 | -8/+7 |
| | | | | | | of the ObjectMapper. Unless special options are used, use a common instance, or create via factory metod. | ||||
* | unpack_bits_from_int8 -> unpack_bits | Arne Juul | 2023-11-10 | 1 | -2/+2 |
| | |||||
* | add simple expandBitTensor function | Arne Juul | 2023-11-10 | 2 | -9/+35 |
| | |||||
* | Add support and upgrade opset | Jo Kristian Bergum | 2023-10-26 | 4 | -7/+31 |
| | |||||
* | Add support for bfloat16 and float16 | Jo Kristian Bergum | 2023-10-26 | 4 | -0/+82 |
| | |||||
* | Less verbose logging when failing to find CUDA and it is optional | Jo Kristian Bergum | 2023-10-26 | 2 | -2/+53 |
| | |||||
* | Disable CPU arena allocator for ONNX | Bjørn Christian Seime | 2023-10-19 | 1 | -0/+1 |
| | | | | | The arena memory allocator pre-allocates excessive of memory up front. Disabling matches the existing configuration in ONNX integration for backend. | ||||
* | Update copyright | Jon Bratseth | 2023-10-09 | 122 | -122/+131 |
| | |||||
* | Don't index PAD and re-factoring | Jo Kristian Bergum | 2023-09-26 | 2 | -41/+37 |
| | |||||
* | Add config options + license | Jo Kristian Bergum | 2023-09-21 | 2 | -0/+2 |
| | |||||
* | Ensure Onnx/Hugginface resources are cleaned up on deconstruction | Bjørn Christian Seime | 2023-09-21 | 1 | -0/+6 |
| | |||||
* | Add ColBERT embedder | Jo Kristian Bergum | 2023-09-21 | 4 | -0/+599 |
| | |||||
* | - Use equals when comparing Optional<Long> | Henning Baldersheim | 2023-09-13 | 2 | -4/+4 |
| | | | | - Minor cleanup | ||||
* | Use thread safe hash map | Bjørn Christian Seime | 2023-08-31 | 1 | -2/+2 |
| | |||||
* | Merge pull request #27969 from vespa-engine/bjorncs/embedder-metrics | Jon Bratseth | 2023-08-31 | 5 | -8/+94 |
|\ | | | | | Add generic metrics for embedders | ||||
| * | Allow sampling of fractional millis | Bjørn Christian Seime | 2023-08-25 | 3 | -15/+10 |
| | | |||||
| * | Add generic metrics for embedders | Bjørn Christian Seime | 2023-08-04 | 5 | -8/+99 |
| | | |||||
* | | Better error message when importing models with illegal names | Lester Solbakken | 2023-08-29 | 1 | -0/+25 |
|/ | |||||
* | Log when GPU configuration is successful | Martin Polden | 2023-07-19 | 1 | -3/+8 |
| | |||||
* | Log warning when failing to use GPU | Martin Polden | 2023-07-19 | 1 | -1/+6 |
| | |||||
* | update onnx.proto | Arne Juul | 2023-06-23 | 4 | -80/+453 |
| | | | | | * use latest version from https://github.com/onnx/onnx/blob/main/onnx/onnx.proto * track API changes (enum -> int32) | ||||
* | Prefer truncation configuration from tokenizer model | Bjørn Christian Seime | 2023-06-12 | 1 | -6/+19 |
| | | | | | | | Only override truncation if not specified or max length exceeds max tokens accepted by model. Use JNI wrapper directly to determine existing truncation configuration (JSON format is not really documented). Simply configuration for pure tokenizer embedder. Disable DJL usage telemetry. | ||||
* | Add missing wiring of pooling strategy | Bjørn Christian Seime | 2023-06-08 | 1 | -11/+1 |
| | |||||
* | Disable padding and make it configurable | Bjørn Christian Seime | 2023-06-08 | 1 | -0/+1 |
| | |||||
* | Merge pull request #27297 from vespa-engine/bjorncs/bert-embedder-services-xml | Bjørn Christian Seime | 2023-06-06 | 4 | -49/+54 |
|\ | | | | | Bjorncs/bert embedder services xml | ||||
| * | Make pooling strategy configurable for Huggingface embedder | Bjørn Christian Seime | 2023-06-05 | 3 | -17/+54 |
| | | |||||
| * | Move config definition to `configdefinitions` | Bjørn Christian Seime | 2023-06-05 | 1 | -32/+0 |
| | | |||||
* | | Add necessary options to use failOnWarnings | gjoranv | 2023-06-05 | 1 | -0/+4 |
|/ | |||||
* | Introduce services.xml syntax for configuring HuggingFace embedders | Bjørn Christian Seime | 2023-06-02 | 2 | -29/+6 |
| | |||||
* | Properly ignore token type ids from tokenizer if disabled | Bjørn Christian Seime | 2023-05-30 | 1 | -2/+2 |
| | |||||
* | Remove dead code | Bjørn Christian Seime | 2023-05-26 | 2 | -43/+0 |
| | |||||
* | Make truncation and max length configurable | Bjørn Christian Seime | 2023-05-26 | 1 | -12/+3 |
| | |||||
* | Use GPU by default if available | Bjørn Christian Seime | 2023-05-22 | 2 | -2/+4 |
| | |||||
* | Revert "Revert "Bjorncs/huggingface tokenizer"" | Bjørn Christian Seime | 2023-05-12 | 6 | -210/+29 |
| | | | | This reverts commit 2bb74878879b3acb1919fd658b8f2c476d8129d6. | ||||
* | Revert "Bjorncs/huggingface tokenizer" | Arnstein Ressem | 2023-05-12 | 6 | -29/+210 |
| | |||||
* | Handle models requiring token type ids | Bjørn Christian Seime | 2023-05-11 | 2 | -13/+20 |
| | |||||
* | Don't lower case | Bjørn Christian Seime | 2023-05-11 | 1 | -1/+1 |
| | |||||
* | Disable special tokens by default | Bjørn Christian Seime | 2023-05-11 | 1 | -0/+1 |
| | |||||
* | Mark HF integration as beta | Bjørn Christian Seime | 2023-05-11 | 1 | -0/+2 |
| | |||||
* | Make HF tokenizer a separate embedder | Bjørn Christian Seime | 2023-05-11 | 5 | -197/+6 |
| | |||||
* | Don't specify both package and namespace | Bjørn Christian Seime | 2023-05-11 | 2 | -1/+1 |
| | |||||
* | Upgrade HF Tokenizer to 0.22.1 | Bjørn Christian Seime | 2023-05-08 | 1 | -1/+1 |
| | |||||
* | Handle nulls | Bjørn Christian Seime | 2023-05-08 | 1 | -0/+4 |
| | |||||
* | fixup! Require GPU when requested and available for Bert + HF embedders | Bjørn Christian Seime | 2023-05-08 | 1 | -1/+1 |
| | |||||
* | Require GPU when requested and available for Bert + HF embedders | Bjørn Christian Seime | 2023-05-08 | 5 | -5/+6 |
| | |||||
* | Require GPU when available for ONNX evaluation in global-phase and embedders | Bjørn Christian Seime | 2023-05-08 | 3 | -5/+42 |
| | |||||
* | Make thread pool size configurable | Bjørn Christian Seime | 2023-05-05 | 5 | -17/+24 |
| | |||||
* | Make normalization optional | Bjørn Christian Seime | 2023-05-05 | 2 | -2/+8 |
| | |||||
* | Allow for manual configuration of GPU | Bjørn Christian Seime | 2023-05-05 | 2 | -1/+8 |
| |