diff options
| author | Henning Baldersheim <balder@yahoo-inc.com> | 2020-05-11 13:22:52 +0000 |
|---|---|---|
| committer | Henning Baldersheim <balder@yahoo-inc.com> | 2020-05-11 17:49:38 +0000 |
| commit | 98d7c8f1c1bd36200db961c0b848a820014ab61f (patch) | |
| tree | fd58c846e8796025b5eda6a3317fd87df434d70f /searchcore | |
| parent | d4b7118d3632acdd9ca6518b3bfcb4dec5caefee (diff) | |
Account and limit number of lids per file to reduce spkies during compaction.
This is to avoid ending up with a very long listof lids that are removed but not compacted away due to lidspace compaction being disabled when
removes and delets are too frequent.
Diffstat (limited to 'searchcore')
| -rw-r--r-- | searchcore/src/vespa/searchcore/config/proton.def | 3 | ||||
| -rw-r--r-- | searchcore/src/vespa/searchcore/proton/server/documentdbconfigmanager.cpp | 1 |
2 files changed, 4 insertions, 0 deletions
diff --git a/searchcore/src/vespa/searchcore/config/proton.def b/searchcore/src/vespa/searchcore/config/proton.def index a210698d5b6..111ee7d1272 100644 --- a/searchcore/src/vespa/searchcore/config/proton.def +++ b/searchcore/src/vespa/searchcore/config/proton.def @@ -250,6 +250,9 @@ summary.log.chunk.skipcrconread bool default=false ## Max size per summary file. summary.log.maxfilesize long default=1000000000 +## Max number of lid entries per file +summary.log.maxnumlids int default=40000000 + ## Max disk bloat factor. This will trigger compacting. summary.log.maxdiskbloatfactor double default=0.1 diff --git a/searchcore/src/vespa/searchcore/proton/server/documentdbconfigmanager.cpp b/searchcore/src/vespa/searchcore/proton/server/documentdbconfigmanager.cpp index d1d88434bd6..68e65acb87d 100644 --- a/searchcore/src/vespa/searchcore/proton/server/documentdbconfigmanager.cpp +++ b/searchcore/src/vespa/searchcore/proton/server/documentdbconfigmanager.cpp @@ -192,6 +192,7 @@ deriveConfig(const ProtonConfig::Summary & summary, const ProtonConfig::Flush::M WriteableFileChunk::Config fileConfig(deriveCompression(chunk.compression), chunk.maxbytes); LogDataStore::Config logConfig; logConfig.setMaxFileSize(log.maxfilesize) + .setMaxNumLids(log.maxnumlids) .setMaxDiskBloatFactor(std::min(flush.diskbloatfactor, flush.each.diskbloatfactor)) .setMaxBucketSpread(log.maxbucketspread).setMinFileSizeFactor(log.minfilesizefactor) .compactCompression(deriveCompression(log.compact.compression)) |