summaryrefslogtreecommitdiffstats
path: root/linguistics-components/src/test/java
Commit message (Expand)AuthorAgeFilesLines
* Add skipping of control tokensLester Solbakken2023-02-102-1/+12
* Add decoding of sentencepiece token sequence to textLester Solbakken2023-02-102-0/+16
* Test segmentation with subwordsJon Bratseth2021-12-172-4/+13
* BERT -> WordPiece, make subword prefix configurableJon Bratseth2021-12-176-129/+119
* Add a BERT embedderJon Bratseth2021-12-162-6/+54
* Encapsulate in a contextJon Bratseth2021-10-011-2/+3
* Update linguisticvs-componentsJon Bratseth2021-09-301-2/+2
* encode -> embedJon Bratseth2021-09-283-23/+23
* Separate component from linguisticsJon Bratseth2021-09-253-0/+197