summaryrefslogtreecommitdiffstats
path: root/linguistics-components/src/main/resources/configdefinitions/language.huggingface.hugging-face-tokenizer.def
Commit message (Expand)AuthorAgeFilesLines
* Introduce services.xml syntax for configuring HuggingFace embeddersBjørn Christian Seime2023-06-021-13/+0
* Make truncation and max length configurableBjørn Christian Seime2023-05-261-1/+3
* Change parameter type to 'model'Bjørn Christian Seime2023-05-121-1/+1
* Revert "Revert "Bjorncs/huggingface tokenizer""Bjørn Christian Seime2023-05-121-0/+11
* Revert "Bjorncs/huggingface tokenizer"Arnstein Ressem2023-05-121-11/+0
* Disable special tokens by defaultBjørn Christian Seime2023-05-111-0/+2
* Make HF tokenizer a separate embedderBjørn Christian Seime2023-05-111-0/+9