subword nmtComputer visionReposubword-nmtUnsupervised Word Segmentation for Neural Machine Translation and Text Generationbpemachine-translationneural-machine-translation2.3kPythonMIT2 y agorsennrichShare or keepTake it
python bpeLanguage and translationRepopython-bpeByte Pair Encoding for Python!bpenlppython234PythonMIT4 y agosoaxelbrookeShare or keepTake it
go tiktokenGitHubRepogo-tiktoken✂️ OpenAI's tiktoken tokenizer written in Gobpegolanggpt220GoMIT22 mo agohupe1980Share or keepTake it