machinemind-bpe

A Byte Pair Encoding (BPE) tokenizer package for CPU and GPU.

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install machinemind-bpe

Releases

Version Released Bullseye
Python 3.9
Bookworm
Python 3.11
Files
1.0.1 2024-08-05    
1.0.0 2024-08-05    
0.4.0 2024-08-05    
0.3.0 2024-08-05    

Issues with this package?

Page last updated 2024-08-05 10:21:18 UTC