summaryrefslogtreecommitdiffstats
path: root/container-search/src/main/java/com/yahoo/prelude/query/parser/Tokenizer.java
Commit message (Collapse)AuthorAgeFilesLines
* Always treat each symbol as a separate tokenJon Bratseth2023-05-221-1/+3
|
* Revert "- HashMap over TreeMap when order des not matter."Bjørn Christian Seime2023-04-111-5/+9
| | | | This reverts commit b1733875a7303d71abfe384da2d6589af742d779.
* - HashMap over TreeMap when order des not matter.Henning Baldersheim2023-03-281-9/+5
| | | | | | - Avoid creating mutable maps when not necessary. - Moderize iteration for readability. - Unify on Set.of instead of Collections.emptySet.
* Cleanup onlyJon Bratseth2022-08-151-1/+0
|
* Update 2017 copyright notices.gjoranv2021-10-071-1/+1
|
* Revert "Merge pull request #17754 from ↵Jon Bratseth2021-05-051-13/+21
| | | | | | | vespa-engine/revert-17747-bratseth/special-tokens-take-2" This reverts commit a2c9cd4bc04f1a3eaa31524b3970b96be5c2eda9, reversing changes made to 8c61a373af0066fbdf1cca354c24b197c7347321.
* Revert "Reapply "Bratseth/special tokens""Jon Bratseth2021-05-051-21/+13
|
* Revert "Merge pull request #17746 from ↵Jon Bratseth2021-05-051-13/+21
| | | | | | | vespa-engine/revert-17738-revert-17737-revert-17736-bratseth/special-tokens" This reverts commit 491856b396d003885e159345fe3f533f0fa35933, reversing changes made to 3720186303f4aef1d185525eaf61092097a64ec9.
* Revert "Revert "Revert "Bratseth/special tokens"""Jon Bratseth2021-05-051-21/+13
|
* Revert "Revert "Bratseth/special tokens""Jon Bratseth2021-05-041-13/+21
|
* Revert "Bratseth/special tokens"Jon Bratseth2021-05-041-21/+13
|
* Move specialtokens to linguisticsJon Bratseth2021-05-041-0/+1
|
* Make immutableJon Bratseth2021-05-041-12/+19
|
* No functional changesJon Bratseth2021-05-041-4/+4
|
* Non-functional changes onlyJon Bratseth2021-01-141-6/+3
|
* Non-functional changes onlyJon Bratseth2020-03-091-18/+15
|
* Update copyright headersJon Bratseth2017-06-141-1/+1
|
* Revert "Update copyright headers"Jon Bratseth2017-06-141-1/+1
|
* Update copyright headersJon Bratseth2017-06-141-1/+1
|
* Revert "Copyright header"Jon Bratseth2017-06-131-1/+1
|
* Copyright headerJon Bratseth2017-06-131-1/+1
|
* Correct heuristic for urlsJon Bratseth2017-04-261-13/+9
|
* suppress fallthrough warningsArne H Juul2017-04-261-0/+2
| | | | * add comments where they occurred, somebody should look at that
* Detect language after tokenizationJon Bratseth2017-01-201-3/+3
| | | | | | | This is a prerequisite to try to be smarter about what subset of the input text is used for language detection, however it breaks functionality in one subtle way: If an application does not pass language explicitly (such that it must be detected), and the input is CJK, and there are configured special tokens, those special tokens will not be detected if they are surrounded by word characters (instead of e.g space).
* Use github name in @authorJon Bratseth2016-06-161-1/+1
|
* PublishJon Bratseth2016-06-151-0/+550