Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
* | | allow generated PB files outside source tree | Arne H Juul | 2021-09-22 | 1 | -1/+1 | |
|/ | ||||||
* | Remove explicit tracking of enum store and multi-value address space usage. | Geir Storli | 2021-09-21 | 2 | -11/+0 | |
| | | | | | This has been replaced by generic tracking of address space usage among components in attributes vectors in all sub databases. | |||||
* | Change loading of nearest neighbor index to use direct I/O instead of mmapping. | Geir Storli | 2021-09-15 | 12 | -82/+102 | |
| | | | | This should reduce memory spike during loading. | |||||
* | ExpressionTransformer: only setChildren if any child nodes is transformed | Lester Solbakken | 2021-09-15 | 1 | -4/+14 | |
| | ||||||
* | Merge pull request #19123 from ↵ | Henning Baldersheim | 2021-09-14 | 2 | -2/+7 | |
|\ | | | | | | | | | vespa-engine/balder/cache-potentially-large-and-expensive-list Cache list that is potentially hard to compute. | |||||
| * | Update abi | Henning Baldersheim | 2021-09-14 | 1 | -0/+1 | |
| | | ||||||
| * | Use a single cached object to make logic simpler and only update on object. | Henning Baldersheim | 2021-09-14 | 1 | -2/+6 | |
| | | ||||||
* | | Merge pull request #19125 from ↵ | Geir Storli | 2021-09-14 | 22 | -157/+216 | |
|\ \ | |/ |/| | | | | | vespa-engine/geirst/refactor-files-with-generic-headers Refactor to represent files with a generic header in a new class [run-systemtest] | |||||
| * | Refactor to represent files with a generic header in a new class. | Geir Storli | 2021-09-14 | 22 | -157/+216 | |
| | | | | | | | | Also move DIRECT I/O alignment setting to a common place. | |||||
* | | Create as list too | Henning Baldersheim | 2021-09-14 | 1 | -8/+4 | |
|/ | ||||||
* | Reduce memory spike during loading of hnsw index by committing at regular ↵ | Geir Storli | 2021-09-10 | 9 | -53/+147 | |
| | | | | | | | intervals. This ensures that memory buffers on hold lists are trimmed while loading, keeping the excess memory usage at a minimum. | |||||
* | Add a 5x faster handcoded detection of legal feature names that does not ↵ | Henning Baldersheim | 2021-09-05 | 1 | -3/+0 | |
| | | | | require quoting. | |||||
* | Merge pull request #18922 from ↵ | Henning Baldersheim | 2021-08-31 | 1 | -1/+1 | |
|\ | | | | | | | | | vespa-engine/toregge/enable-alternate-visited-nodes-trackers-for-hnsw-index Enable alternate visited nodes trackers for HNSW index. | |||||
| * | Enable alternate visited nodes trackers for HNSW index. | Tor Egge | 2021-08-31 | 1 | -1/+1 | |
| | | ||||||
* | | Lower limit for selecting BitVectorVisistedTracker. | Tor Egge | 2021-08-31 | 1 | -1/+1 | |
|/ | ||||||
* | Add class comments. Fix typo. | Tor Egge | 2021-08-31 | 3 | -1/+12 | |
| | ||||||
* | Prepare for alternate visited nodes trackers for HNSW index. | Tor Egge | 2021-08-30 | 9 | -9/+186 | |
| | ||||||
* | Merge pull request #18898 from ↵ | Henning Baldersheim | 2021-08-30 | 2 | -18/+45 | |
|\ | | | | | | | | | vespa-engine/geirst/avoid-global-filter-calculation-when-not-needed The global filter is only needed when having a nearest neighbor index… | |||||
| * | The global filter is only needed when having a nearest neighbor index (hnsw) ↵ | Geir Storli | 2021-08-30 | 2 | -18/+45 | |
| | | | | | | | | | | | | and doing approximate calculation. This avoids costly calculation of the global filter in cases it is not needed. | |||||
* | | Handle when priorityQ goes from not full to full. | Henning Baldersheim | 2021-08-30 | 1 | -1/+5 | |
| | | ||||||
* | | As doSeek is called alot more frequent than doUnpack just use locking of the ↵ | Henning Baldersheim | 2021-08-30 | 1 | -8/+6 | |
|/ | | | | | | | heap in unpack. In addition to adjusting the priority Q also update the distance_threshold with a relaxed store to an atomic variable. On read the distance threshold can be read cheaply with a relaxed load. | |||||
* | Report address space usage for shared string repo for non-dense tensor ↵ | Geir Storli | 2021-08-23 | 5 | -2/+18 | |
| | | | | attributes. | |||||
* | Report address space usage for components in tensor attributes. | Geir Storli | 2021-08-20 | 12 | -2/+67 | |
| | ||||||
* | Merge pull request #18783 from vespa-engine/toregge/compact-hnsw-index | Geir Storli | 2021-08-20 | 10 | -6/+300 | |
|\ | | | | | Compact HNSW index when ratio of dead bytes / address space is too high | |||||
| * | Factor out common code. | Tor Egge | 2021-08-18 | 1 | -17/+21 | |
| | | ||||||
| * | Compact HNSW index when ratio of dead bytes / address space is too high | Tor Egge | 2021-08-18 | 10 | -6/+296 | |
| | | | | | | | | relative to used bytes / address space. | |||||
* | | Track max address space usage among components in attributes vectors in all ↵ | Geir Storli | 2021-08-20 | 1 | -0/+1 | |
| | | | | | | | | sub databases. | |||||
* | | Include limits when needed. | Tor Egge | 2021-08-18 | 1 | -0/+1 | |
|/ | ||||||
* | Merge pull request #18755 from ↵ | Håvard Pettersen | 2021-08-16 | 7 | -192/+4 | |
|\ | | | | | | | | | vespa-engine/havardpe/move-feature-name-symbol-extractor move FeatureNameExtractor | |||||
| * | move FeatureNameExtractor | Håvard Pettersen | 2021-08-16 | 7 | -192/+4 | |
| | | | | | | | | to make it available for use in vespa-eval-expr | |||||
* | | Merge pull request #18752 from ↵ | Henning Baldersheim | 2021-08-16 | 2 | -4/+7 | |
|\ \ | | | | | | | | | | | | | vespa-engine/toregge/use-4096-buffers-for-hnsw-index-link-array-store Use 4096 buffers for HNSW link array store. | |||||
| * | | Use 4096 buffers for HNSW link array store. | Tor Egge | 2021-08-16 | 2 | -4/+7 | |
| | | | | | | | | | | | | | | | Configure link array store to handle arrays of 193 elements or less without indirect storage. | |||||
* | | | Improve naming and readability | Henning Baldersheim | 2021-08-16 | 1 | -7/+8 | |
| | | | ||||||
* | | | Instead of having one large array of individually allocated vectors use | Henning Baldersheim | 2021-08-16 | 2 | -15/+46 | |
|/ / | | | | | | | | | 2 large, optionally mmapped, vectors where the first just points into the second. In order to avoid resizing, count first. | |||||
* | | Minor code layout | Henning Baldersheim | 2021-08-15 | 1 | -2/+1 | |
| | | ||||||
* | | Better naming. | Henning Baldersheim | 2021-08-15 | 1 | -2/+2 | |
| | | ||||||
* | | Provide more details on memory usage. | Henning Baldersheim | 2021-08-15 | 1 | -1/+9 | |
| | | ||||||
* | | Add a time budget of 100ms. If counting not complete by then, abort, and let ↵ | Henning Baldersheim | 2021-08-15 | 2 | -7/+17 | |
| | | | | | | | | the count be incomplete. | |||||
* | | Use a simple std::vector<bool> for visited markin as most bits will be set. | Henning Baldersheim | 2021-08-15 | 1 | -4/+4 | |
| | | ||||||
* | | Avoid starting a separate thread for completing index insert. | Henning Baldersheim | 2021-08-13 | 1 | -34/+73 | |
| | | | | | | | | | | Use a queue and do completition in the forground. That ensures only a single thread modifying the attribute. | |||||
* | | Notify when _pending reaches zero. | Henning Baldersheim | 2021-08-13 | 1 | -3/+6 | |
| | | ||||||
* | | Refactor for readability and maintenance. | Henning Baldersheim | 2021-08-13 | 2 | -30/+89 | |
| | | ||||||
* | | Use the executor for the part that can be parallell when rebuilding index on ↵ | Henning Baldersheim | 2021-08-13 | 2 | -7/+64 | |
|/ | | | | load. | |||||
* | Add an executor to the AttributeVector::load/onLoad interface so attributes ↵ | Henning Baldersheim | 2021-08-12 | 32 | -39/+46 | |
| | | | | can use multithread load if feasible. | |||||
* | swappable -> paged | Henning Baldersheim | 2021-08-12 | 4 | -7/+7 | |
| | ||||||
* | A swappable attribute will use a file backed memory allocator. | Henning Baldersheim | 2021-08-12 | 6 | -12/+55 | |
| | ||||||
* | swapable -> swappable | Henning Baldersheim | 2021-08-12 | 2 | -3/+3 | |
| | ||||||
* | Control swappable | Henning Baldersheim | 2021-08-12 | 2 | -3/+3 | |
| | ||||||
* | Add swapable attribute option. | Henning Baldersheim | 2021-08-12 | 3 | -85/+42 | |
| | ||||||
* | Merge pull request #18716 from ↵ | Henning Baldersheim | 2021-08-11 | 4 | -10/+20 | |
|\ | | | | | | | | | vespa-engine/havardpe/avoid-crash-on-runtime-onnx-errors avoid crash on run-time onnx errors |