summaryrefslogtreecommitdiffstats
path: root/linguistics/src/test/java/com/yahoo/language
Commit message (Expand)AuthorAgeFilesLines
* Refactor to separate classesJon Bratseth2021-09-171-2/+1
* Encode to sparse tensorJon Bratseth2021-09-161-0/+6
* Encode to dense tensorJon Bratseth2021-09-162-0/+23
* Make SentencePieceEncoder configurableJon Bratseth2021-09-163-30/+102
* More unit testsJon Bratseth2021-09-141-1/+20
* Pure Java sentencepiece implementationJon Bratseth2021-09-131-0/+78
* Revert "Merge pull request #17754 from vespa-engine/revert-17747-bratseth/spe...Jon Bratseth2021-05-051-0/+40
* Revert "Reapply "Bratseth/special tokens""Jon Bratseth2021-05-051-40/+0
* Revert "Merge pull request #17746 from vespa-engine/revert-17738-revert-17737...Jon Bratseth2021-05-051-0/+40
* Revert "Revert "Revert "Bratseth/special tokens"""Jon Bratseth2021-05-051-40/+0
* Revert "Revert "Bratseth/special tokens""Jon Bratseth2021-05-041-0/+40
* Revert "Bratseth/special tokens"Jon Bratseth2021-05-041-40/+0
* Expose tokens as mapJon Bratseth2021-05-041-5/+3
* Move specialtokens to linguisticsJon Bratseth2021-05-041-0/+42
* No functional changesJon Bratseth2021-04-141-37/+26
* No functional changesJon Bratseth2021-04-148-19/+16
* No functional changesJon Bratseth2021-02-031-0/+19
* handle plugin tokenizer returning tokens with empty original stringArne Juul2020-08-241-0/+51
* Minor unification of tests.Henning Baldersheim2020-08-122-20/+36
* Surrogate aware gram splittingJon Bratseth2020-06-251-9/+37
* Add/corect copyright headersJon Bratseth2020-01-031-0/+1
* Remove deprecated apis in linguistics.gjoranv2019-01-211-27/+0
* Deprecated methods and add OptimaizeDetectorJon Bratseth2018-11-012-6/+36
* use com.optimaize.langdetect for lang detectionJefim Matskin2018-07-241-0/+5
* add opennlp stemmers - revert previous changesJefim Matskin2018-07-183-5/+238
* add lang detection and opennlp stemmersJefim Matskin2018-07-172-0/+6
* Fix author tag for SimonBjørn Christian Seime2018-07-0512-12/+12
* Update copyright headersJon Bratseth2017-06-1420-20/+20
* Revert "Update copyright headers"Jon Bratseth2017-06-1420-20/+20
* Update copyright headersJon Bratseth2017-06-1420-20/+20
* Remove carriage returnJon Bratseth2017-06-141-1/+1
* Revert "Copyright header"Jon Bratseth2017-06-1320-21/+21
* Copyright headerJon Bratseth2017-06-1320-21/+21
* PublishJon Bratseth2016-06-1520-0/+1485