aboutsummaryrefslogtreecommitdiffstats
path: root/searchlib/src
Commit message (Collapse)AuthorAgeFilesLines
* estimate actual costHåvard Pettersen2024-04-256-7/+109
|
* Merge pull request #30973 from ↵Håvard Pettersen2024-04-196-90/+154
|\ | | | | | | | | vespa-engine/geirst/better-non-strict-cost-estimates-for-btree-iterators Use better non-strict cost estimates for btree and disk index iterators.
| * Use better non-strict cost estimates for btree and disk index iterators.Geir Storli2024-04-196-90/+154
| | | | | | | | | | | | | | | | | | | | | | | | We previously increased the strict cost of heap-based complex blueprints (e.g OR) to avoid the problem of them being forced strict too early. More benchmarking however has shown that its better address the real problem, which is the non-strict cost of btree iterators that are childen of the ORs. Its important that the non-strict cost is most accurate with high inflows (moving towards 1.0), as the time penalty of choosing wrong is at its highest then. This adds additional benchmarks needed for analysis and updates/adjusts the parameters and formulas in flow_tuning.h to match observations.
* | Don't enable direct io for disk index dictionary .ssdat file.Tor Egge2024-04-181-1/+1
| |
* | Revert "Revert "Use memory mapped disk index dictionary .ssdat file when large""Tor Egge2024-04-187-32/+127
| |
* | Revert "Use memory mapped disk index dictionary .ssdat file when large"Henning Baldersheim2024-04-187-127/+32
| |
* | Merge pull request #30956 from ↵Henning Baldersheim2024-04-183-1/+11
|\ \ | | | | | | | | | | | | vespa-engine/havardpe/optimize-single-child-vespa-wand optimize weak and with single child
| * | optimize weak and with single childHåvard Pettersen2024-04-173-1/+11
| |/
* | Merge pull request #30958 from ↵Henning Baldersheim2024-04-182-0/+2
|\ \ | | | | | | | | | | | | vespa-engine/toregge/add-variable-length-array-comments Add variable-length array comments.
| * | Add variable-length array comments.Tor Egge2024-04-182-0/+2
| | |
* | | Merge pull request #30950 from vespa-engine/balder/typedcells-as-value-paramHenning Baldersheim2024-04-1819-222/+107
|\ \ \ | | | | | | | | Balder/typedcells as value param
| * | | Drop assert for hammingHenning Baldersheim2024-04-171-2/+1
| | | |
| * | | Add final to allow better code genrartion of calc_with_limit.Henning Baldersheim2024-04-178-97/+18
| | | |
| * | | Transfer TypedCells as value param as that is more efficient than reference ↵Henning Baldersheim2024-04-1717-127/+92
| |/ / | | | | | | | | | for 16 byte structs.
* | | Merge pull request #30934 from ↵Geir Storli2024-04-187-32/+127
|\ \ \ | |/ / |/| | | | | | | | vespa-engine/toregge/use-memory-mapped-disk-index-dictionary-ssdat-file-when-large-and-with-sufficient-padding Use memory mapped disk index dictionary .ssdat file when large
| * | Factor out common code to static member functions is_padded_for_memory_mapTor Egge2024-04-177-30/+39
| | | | | | | | | | | | and file_units.
| * | Use memory mapped disk index dictionary .ssdat file when largeTor Egge2024-04-165-32/+118
| | | | | | | | | | | | and with sufficient padding. This reduces memory usage spikes.
* | | Ensure result is at least min_rawscoreHenning Baldersheim2024-04-171-2/+3
| | |
* | | Use correct value for has_single_subspace template param.Henning Baldersheim2024-04-173-8/+15
| | |
* | | - Optimize distance calculation for tensors with single dense subspace.Henning Baldersheim2024-04-1628-103/+128
| |/ |/| | | | | | | - Let EmptySubspace be invalid. - Add noexcept to get_tensor(s).
* | Merge pull request #30920 from ↵Geir Storli2024-04-165-29/+29
|\ \ | | | | | | | | | | | | vespa-engine/havardpe/improve-non-strict-and-sorting pass in_flow to intermediate sort functions
| * | pass in_flow to intermediate sort functionsHåvard Pettersen2024-04-155-29/+29
| | | | | | | | | | | | | | | improve non-strict AND sorting by letting allow_force_strict affect ordering.
* | | Merge pull request #30935 from vespa-engine/geirst/adjust-bitvector-strict-costGeir Storli2024-04-163-2/+34
|\ \ \ | | | | | | | | Adjust strict cost of bitvector after benchmarking.
| * | | Adjust strict cost of bitvector after benchmarking.Geir Storli2024-04-163-2/+34
| | | | | | | | | | | | | | | | A bitvector is used to track active lids and is always part of a query.
* | | | Pad disk index dictionary files at end.Tor Egge2024-04-164-18/+24
| |_|/ |/| |
* | | Add comments about overflows.Tor Egge2024-04-161-0/+20
| | |
* | | Handle disk index dictionary .ssdat file being larger than 32 GiB.Tor Egge2024-04-153-3/+3
| | |
* | | Test disk index dictionary file with many long words.Tor Egge2024-04-156-23/+146
|/ /
* | Adjust strict cost of intermediate / complex leaf blueprints.Geir Storli2024-04-153-13/+34
| | | | | | | | | | | | | | This ensures that AND(filter vs complex) makes a better decision regarding when to force the complex blueprint strict. Reverse hash filter cost is adjusted accordingly.
* | Re-adjust cost model params for IN/weightedSet/dotProduct.Geir Storli2024-04-154-22/+43
| | | | | | | | Only combinations that support reverse hash filter have a cheaper non-strict cost.
* | Merge pull request #30913 from ↵Geir Storli2024-04-158-42/+350
|\ \ | |/ |/| | | | | vespa-engine/geirst/benchmarking-of-intermediate-blueprints Add benchmarking of intermediate blueprints with configurable childre…
| * Add benchmarking of intermediate blueprints with configurable children setups.Geir Storli2024-04-158-42/+350
| | | | | | | | | | | | In particular this adds benchmarking of: * AND(filter, IN (100 children)) * AND(filter, OR (100 children))
* | Remove unused variables.Tor Egge2024-04-151-2/+0
|/
* Merge pull request #30905 from ↵v8.330.52Geir Storli2024-04-131-0/+3
|\ | | | | | | | | vespa-engine/havardpe/improve-strict-and-sort-when-allowed improve strict and sorting when allowed
| * improve strict and sorting when allowedHåvard Pettersen2024-04-121-0/+3
| |
* | Merge pull request #30904 from ↵Geir Storli2024-04-131-2/+4
|\ \ | | | | | | | | | | | | vespa-engine/geirst/in-operator-cost-model-adjustment Adjust cost model params for IN/weightedSet/dotProduct after benchmar…
| * | Adjust cost model params for IN/weightedSet/dotProduct after benchmarking.Geir Storli2024-04-121-2/+4
| | |
* | | - Add noexcept, const and final where it makes sense.Henning Baldersheim2024-04-123-34/+31
| |/ |/|
* | Merge pull request #30903 from vespa-engine/havardpe/andflow-strictness-reorderGeir Storli2024-04-122-148/+146
|\ \ | | | | | | add code to AndFlow that can perform additional incremental reordering
| * | add code to AndFlow that can perform additional incremental reorderingHåvard Pettersen2024-04-122-148/+146
| | | | | | | | | | | | | | | | | | | | | | | | of children based on the future possibility of forcing those children to be strict. update unit test to use code from AndFlow and add tests for non-strict AND ordering as well.
* | | Replace all usages of Arrays.asList with List.of where possible.Henning Baldersheim2024-04-1211-78/+80
| |/ |/|
* | Unify on List.ofHenning Baldersheim2024-04-1110-32/+21
| |
* | Merge pull request #30872 from ↵Geir Storli2024-04-117-56/+119
|\ \ | | | | | | | | | | | | vespa-engine/geirst/more-low-level-in-operator-perf-tests Support populating attribute where docid sets matching terms/children…
| * | Only 1 default value is supported when using disjunct terms/children.Geir Storli2024-04-111-2/+4
| | |
| * | Support populating attribute where docid sets matching terms/children are ↵Geir Storli2024-04-107-56/+117
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | disjunct. This also adds testing of IN operator with many tokens (up to 10k), which uses disjunct terms/children. In addition calculation of "alternative cost" is removed, as it doesn't provide any value at the moment.
* | | Unify on Map.ofHenning Baldersheim2024-04-113-12/+12
| |/ |/|
* | consolidate solutions into a single heuristic algorithmHåvard Pettersen2024-04-102-86/+18
| | | | | | | | use error relative to minimal cost rather than error potential
* | add some comparison slack to improve robustnessHåvard Pettersen2024-04-101-7/+7
|/
* Merge pull request #30858 from ↵Henning Baldersheim2024-04-091-0/+1
|\ | | | | | | | | vespa-engine/toregge/disable-restrict-warning-when-compiling-with-gcc-12-on-debian-or-ubuntu Turn off restrict warning when compiling with gcc 12 on ubuntu 22.04
| * Turn off restrict warning when compiling with gcc 12 on ubuntu 22.04Tor Egge2024-04-091-0/+1
| | | | | | | | | | | | or debian 12. Include cstddef to get definition of size_t.