aboutsummaryrefslogtreecommitdiffstats
path: root/linguistics-components/src/main/java/com/yahoo/language/huggingface
Commit message (Expand)AuthorAgeFilesLines
* Update copyrightJon Bratseth2023-10-094-6/+6
* HuggingFace Tokenizer expects path to be a directoryBjørn Christian Seime2023-08-311-2/+18
* Prefer truncation configuration from tokenizer modelBjørn Christian Seime2023-06-122-13/+104
* Test padding with truncationBjørn Christian Seime2023-06-081-1/+1
* Disable padding and make it configurableBjørn Christian Seime2023-06-081-3/+7
* Introduce services.xml syntax for configuring HuggingFace embeddersBjørn Christian Seime2023-06-021-0/+1
* Make truncation and max length configurableBjørn Christian Seime2023-05-261-4/+14
* Implement deconstructBjørn Christian Seime2023-05-161-0/+1
* Revert "Revert "Bjorncs/huggingface tokenizer""Bjørn Christian Seime2023-05-123-0/+172
* Revert "Bjorncs/huggingface tokenizer"Arnstein Ressem2023-05-123-172/+0
* Disable special tokens by defaultBjørn Christian Seime2023-05-111-12/+7
* Mark HF integration as betaBjørn Christian Seime2023-05-112-0/+5
* Make HF tokenizer a separate embedderBjørn Christian Seime2023-05-113-0/+172