Commit message (Expand) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Update copyright | Jon Bratseth | 2023-10-09 | 1 | -1/+1 |
* | Add skipping of control tokens | Lester Solbakken | 2023-02-10 | 1 | -1/+0 |
* | Add decoding of sentencepiece token sequence to text | Lester Solbakken | 2023-02-10 | 1 | -0/+8 |
* | Test segmentation with subwords | Jon Bratseth | 2021-12-17 | 1 | -1/+4 |
* | BERT -> WordPiece, make subword prefix configurable | Jon Bratseth | 2021-12-17 | 1 | -0/+59 |