diff options
| author | Jo Kristian Bergum <bergum@yahooinc.com> | 2024-01-06 10:54:58 +0100 |
|---|---|---|
| committer | Jo Kristian Bergum <bergum@yahooinc.com> | 2024-01-06 10:54:58 +0100 |
| commit | 18ae21bce56e018cef2c17d03e63617530af59ae (patch) | |
| tree | 3c1dcee63395fee2e476be9ce33e2437262b00d7 /configdefinitions | |
| parent | e4da75db4556a3cd72b034c4406027f9bba73918 (diff) | |
handle multilingual models better
Diffstat (limited to 'configdefinitions')
| -rw-r--r-- | configdefinitions/src/vespa/col-bert-embedder.def | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/configdefinitions/src/vespa/col-bert-embedder.def b/configdefinitions/src/vespa/col-bert-embedder.def index d525affa7f2..a6dac3f18fd 100644 --- a/configdefinitions/src/vespa/col-bert-embedder.def +++ b/configdefinitions/src/vespa/col-bert-embedder.def @@ -25,6 +25,9 @@ transformerAttentionMask string default=attention_mask transformerStartSequenceToken int default=101 transformerEndSequenceToken int default=102 transformerMaskToken int default=103 +transformerPadToken int default=0 +queryTokenId int default=1 +documentTokenId int default=2 # Output name transformerOutput string default=contextual |