summaryrefslogtreecommitdiffstats
path: root/searchcorespi
diff options
context:
space:
mode:
authorGeir Storli <geirst@verizonmedia.com>2020-06-25 15:31:41 +0000
committerGeir Storli <geirst@verizonmedia.com>2020-06-25 15:53:38 +0000
commit4b5340c5fa6d78af845cc19d1667818fb7d6bf70 (patch)
tree6ae44eb8818fa4be4819faf1e44186ded77e5c6f /searchcorespi
parent4350799ad50b3377d07857838fab9aac3926b8e4 (diff)
Run prepare step of two-phase puts in the shared executor instead of the attribute field executor.
Benchmarking has shown that we get an uneven balance between the underlying threads when using the attribute field executor. This is a sequenced task executor, where each underlying thread only is allowed to handle a subset of the prepare tasks (based on the local document id). Using the shared executor should give an even balance.
Diffstat (limited to 'searchcorespi')
-rw-r--r--searchcorespi/src/vespa/searchcorespi/index/ithreadingservice.h24
1 files changed, 17 insertions, 7 deletions
diff --git a/searchcorespi/src/vespa/searchcorespi/index/ithreadingservice.h b/searchcorespi/src/vespa/searchcorespi/index/ithreadingservice.h
index 2ace9a8ac6b..18c376a4c2b 100644
--- a/searchcorespi/src/vespa/searchcorespi/index/ithreadingservice.h
+++ b/searchcorespi/src/vespa/searchcorespi/index/ithreadingservice.h
@@ -8,26 +8,36 @@ namespace vespalib { class ISequencedTaskExecutor; }
namespace searchcorespi::index {
/**
- * Interface for the thread model used for write tasks.
+ * Interface for the thread model used for write tasks for a single document database.
*
* We have multiple write threads:
*
- * 1. The master write thread used for the majority of write tasks.
+ * 1. The "master" write thread used for the majority of write tasks.
*
- * 2. The index write thread used for doing changes to the memory
+ * 2. The "index" write thread used for doing changes to the memory
* index, either directly (for data not bound to a field) or via
* index field inverter executor or index field writer executor.
*
- * 3. The index field inverter executor is used to populate field
+ * 3. The "summary" thread is used for doing changes to the document store.
+ *
+ * 4. The "index field inverter" executor is used to populate field
* inverters with data from document fields. Scheduled tasks for
* the same field are executed in sequence.
*
- * 4. The index field writer executor is used to sort data in field
+ * 5. The "index field writer" executor is used to sort data in field
* inverters before pushing the data to the memory field indexes.
* Scheduled tasks for the same field are executed in sequence.
*
- * The master write thread is always the one giving tasks to the index
- * write thread.
+ * 6. The "attribute field writer" executor is used to write data to attribute vectors.
+ * Each attribute is always handled by the same thread,
+ * and scheduled tasks for the same attribute are executed in sequence.
+ *
+ * The master write thread is always the one giving tasks to the other write threads above.
+ *
+ * In addition this interface exposes the "shared" executor that is used by all document databases.
+ * This is among others used for compressing / de-compressing documents in the document store,
+ * merging files as part of disk index fusion, and running the prepare step when doing two-phase
+ * puts against a tensor attribute with a HNSW index.
*
* The index write thread extracts fields from documents and gives
* task to the index field inverter executor and the index field