aboutsummaryrefslogtreecommitdiffstats
path: root/linguistics
Commit message (Expand)AuthorAgeFilesLines
...
* Expand test case for language detectionJon Marius Venstad2021-12-201-3/+28
* Upper bound on input size, and use opennlp before simple detectorJon Marius Venstad2021-12-201-6/+3
* Avoid putting nulls in languange mapJon Marius Venstad2021-12-201-2/+5
* Revert "Merge pull request #20578 from vespa-engine/revert-20568-jonmv/replac...Jon Marius Venstad2021-12-2013-176/+246
* Revert "Replace optimaize with OpenNLP language detector [run-systemtest]"Jon Marius Venstad2021-12-1813-246/+176
* Re-add filesJon Marius Venstad2021-12-185-0/+142
* Move model to module where it is needed, to simplify, at the cost of larger b...Jon Marius Venstad2021-12-183-22/+21
* Replace UrlcharSequenceNormalizer with one with an improved regexJon Marius Venstad2021-12-171-6/+0
* Mockito test scopeJon Marius Venstad2021-12-171-0/+1
* Add some javadoc, and no need to handle null return for modelJon Marius Venstad2021-12-172-2/+4
* Replace optimaize with OpenNLP language detectorJon Marius Venstad2021-12-178-170/+102
* Add a BERT embedderJon Bratseth2021-12-161-2/+3
* Time out requests after 200sJon Marius Venstad2021-12-131-1/+0
* Update 2020 Oath copyrights.gjoranv2021-10-272-2/+2
* Update 2019 Oath copyrights.gjoranv2021-10-271-1/+1
* Update Verizon Media copyright notices.gjoranv2021-10-073-3/+3
* Update 2018 copyright notices.gjoranv2021-10-073-3/+3
* Update 2017 copyright notices.gjoranv2021-10-0770-70/+70
* Encapsulate in a contextJon Bratseth2021-10-012-16/+65
* Pass destinationJon Bratseth2021-09-302-8/+14
* encode -> embedJon Bratseth2021-09-283-64/+64
* Separate component from linguisticsJon Bratseth2021-09-2517-1206/+0
* Linguistics cleanupJon Bratseth2021-09-2118-45/+29
* Add 'encode' expressionJon Bratseth2021-09-192-1/+35
* Provide a (non-working) encoder by defaultJon Bratseth2021-09-171-1/+1
* Update ABI specJon Bratseth2021-09-171-20/+46
* CleanupJon Bratseth2021-09-175-9/+2
* Refactor to separate classesJon Bratseth2021-09-178-203/+279
* Encoder interfaceJon Bratseth2021-09-163-4/+55
* Encode to sparse tensorJon Bratseth2021-09-163-0/+17
* Encode to dense tensorJon Bratseth2021-09-163-4/+36
* Use a result builderJon Bratseth2021-09-161-21/+53
* Make SentencePieceEncoder configurableJon Bratseth2021-09-166-56/+283
* Merge pull request #19130 from vespa-engine/bratseth/sp-exportJo Kristian Bergum2021-09-142-0/+79
|\
| * Add to abi specJon Bratseth2021-09-141-0/+72
| * Make publicJon Bratseth2021-09-141-0/+7
* | Merge pull request #19131 from vespa-engine/bratseth/sp-simplifyJon Bratseth2021-09-141-13/+7
|\ \
| * | Slight algorithm simplificationJon Bratseth2021-09-141-6/+4
| * | Slight algorithm simplificationJon Bratseth2021-09-141-6/+3
| * | Slight algorithm simplificationJon Bratseth2021-09-141-11/+10
| |/
* / More unit testsJon Bratseth2021-09-141-1/+20
|/
* Pure Java sentencepiece implementationJon Bratseth2021-09-137-2/+731
* we want to compare Linguistics objects for equivalenceArne Juul2021-08-044-1/+9
* Require replacements to be applied during tokenizationJon Bratseth2021-06-153-12/+11
* Revert "Merge pull request #17754 from vespa-engine/revert-17747-bratseth/spe...Jon Bratseth2021-05-058-13/+336
* Revert "Reapply "Bratseth/special tokens""Jon Bratseth2021-05-058-336/+13
* Revert "Merge pull request #17746 from vespa-engine/revert-17738-revert-17737...Jon Bratseth2021-05-058-13/+336
* Revert "Revert "Revert "Bratseth/special tokens"""Jon Bratseth2021-05-058-336/+13
* Revert "Revert "Bratseth/special tokens""Jon Bratseth2021-05-048-13/+336
* Revert "Bratseth/special tokens"Jon Bratseth2021-05-048-336/+13