aboutsummaryrefslogtreecommitdiffstats
path: root/linguistics/src/main/java/com/yahoo/language/simple/SimpleTokenizer.java
Commit message (Expand)AuthorAgeFilesLines
* Update copyrightJon Bratseth2023-10-091-1/+1
* Don't remove indexable symbols when stemmingJon Bratseth2023-06-021-1/+7
* Use dollar and hour base unitsJon Bratseth2023-05-191-2/+2
* Determine token types considering all charactersJon Bratseth2022-08-161-5/+21
* Update 2017 copyright notices.gjoranv2021-10-071-1/+1
* Revert "Merge pull request #17754 from vespa-engine/revert-17747-bratseth/spe...Jon Bratseth2021-05-051-10/+17
* Revert "Reapply "Bratseth/special tokens""Jon Bratseth2021-05-051-17/+10
* Revert "Merge pull request #17746 from vespa-engine/revert-17738-revert-17737...Jon Bratseth2021-05-051-10/+17
* Revert "Revert "Revert "Bratseth/special tokens"""Jon Bratseth2021-05-051-17/+10
* Revert "Revert "Bratseth/special tokens""Jon Bratseth2021-05-041-10/+17
* Revert "Bratseth/special tokens"Jon Bratseth2021-05-041-17/+10
* Wire in (but don't use) SpecialTokensJon Bratseth2021-05-041-10/+17
* variables in lambdas must be finalArne Juul2020-04-241-5/+8
* Apply suggestions from code reviewArne H Juul2020-04-241-2/+2
* add more tracing and debug logging of stemmingArne Juul2020-04-241-1/+9
* Update copyright headersJon Bratseth2017-06-141-1/+1
* Revert "Update copyright headers"Jon Bratseth2017-06-141-1/+1
* Update copyright headersJon Bratseth2017-06-141-1/+1
* Revert "Copyright header"Jon Bratseth2017-06-131-1/+1
* Copyright headerJon Bratseth2017-06-131-1/+1
* Nonfunctional changes only#Jon Bratseth2017-05-191-1/+1
* PublishJon Bratseth2016-06-151-0/+76