| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|\
| |
| |
| |
| | |
vespa-engine/balder/use-interfaces-for-looking-up-index-from-node
- Avoid going via a temporary IdealNodesList.
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
- Use ConstArrayRef to hide implementation.
- Store all 3 node categories in a single vector.
- Use a small_vector that can handle redundancy up to 5 without requiring extra memory allocation.
- Build a hash_map if redundancy/groups > 32 for constant lookup time.
|
| | |
|
| | |
|
| | |
|
|/ |
|
| |
|
|\
| |
| | |
Move where possible
|
| | |
|
|\ \
| | |
| | |
| | |
| | | |
vespa-engine/balder/prepare-for-better-stats-reset-code
Balder/prepare for better stats reset code
|
| |/ |
|
|/
|
|
| |
std::unordered_set
|
| |
|
| |
|
|
|
|
| |
validFirstByte has always been conducted.
|
|
|
|
|
| |
- Do not hide narrowing to 32 bit.
- Use enum class.
|
|
|
|
|
| |
- Avoid plt indirection and allow more inlining of frequently called code.
- Reapplication of #27646
|
|\
| |
| |
| |
| | |
vespa-engine/revert-27773-revert-27643-balder/use-direct-weighted-set-also-for-filter-fields
Revert "Revert "- Consolidate on isFilter.""
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Looks like GCC gets confused during compilation of inlined xxhash
functions when the input buffer is backed by a `std::array`, even
when the length argument is a runtime value. An xxhash branch that
can only kick in when the input length is >= 9 bytes triggers a
compilation warning (and thus error) that 8 bytes at the start of
the buffer is being read, with GCC staunchly insisting that the
buffer size is only 4 bytes.
We only see this warning when compiling with UBSan instrumentation,
so presumably it injects enough changes into the GCC intermediate
representation to thoroughly confuse it.
For now, suppress the warning when compiling with sanitizers.
Revisit on GCC 13 to see if the warning is gone.
|
|\ \
| | |
| | | |
Prefer std::filesystem::exists over FastOS_StatInfo
|
| | | |
|
|/ / |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
| |
| |
| | |
- Use BitWord as helper class instead of inheriting in many static methods.
|
|/ |
|
| |
|
| |
|
| |
|
|\
| |
| | |
Implement Levenshtein DFA with successor string generation
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This implements code for building and evaluating Levenshtein
Deterministic Finite Automata, where the resulting DFA efficiently
matches all possible source strings that can be transformed to the
target string within k max edits. This allows for O(n) matching
of strings. We currently support k in {1, 2}.
Additionally, when matching using a DFA, in the case where the
source string does _not_ match, we can generate the _successor_ string;
the next matching string that is lexicographically _greater_ than
the source string. This string has the invariant that there are no
possibly matching strings within k edits ordered after the source
string but before the successor.
This lets us do possibly massive leaps forward in an ordered
dictionary, turning a scan for matches into a sublinear operation.
Matching and successor generation is fully Unicode-aware. All input
strings are expected to be in UTF-8 (without nulls), and the generated
successor is also encoded as UTF-8. Internally, matching is done on
UTF-32 code points and the DFA itself is built around UTF-32.
UTF-8 decoding of source strings is done in a streaming fashion and
does not require any allocations.
This commit includes a templated core Levenshtein DFA matching (and
successor generation) algorithm and two separate DFA implementations
that can be used; one explicit and one implicit.
The explicit DFA is an immutable DAG built up-front that represents
all DFA states and transitions as explicit nodes and edges in a
graph. This is currently the fastest to evaluate, but the build time
and memory usage means its usage should be preferred for shorter
strings (up to a few hundred chars).
The implicit DFA does not build any graph up-front, but rather
evaluates state transitions on-demand for any given source string.
This is currently slower than the explicit DFA, but its O(1)
memory usage (aside from the memory used by the target string itself)
means that it can be used for arbitrary string lengths.
This code currently exists as a freestanding vespalib utility, and
is not yet wired to any production code (fuzzy matching or similar).
Future optimizations:
* Redesign sparse state representation and stepping logic to be
much less branching, in turn making the code much less likely to
stall the CPU pipeline.
* Emit as much as possible of the successor string suffix by copying
directly from the target string UTF-8 representation instead of
following the DFA and encoding UTF-32 to UTF-8 chars.
|
|\ \
| | |
| | |
| | |
| | | |
vespa-engine/balder/gc-some-obscure-fastos-file-stdout-stderr-support
GC some obscure support in FastOS_File for stderr and stdout.
|
| |/ |
|
|/ |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|