gse
Go efficient fast multilingual NLP and text segmentation; support English, Chinese, Japanese and others.
- 2.8k
- Go
- Apache-2.0
- 4 d ago
Go efficient fast multilingual NLP and text segmentation; support English, Chinese, Japanese and others.
Chinese text segmentation with R. R语言中文分词 (文档已更新 🎉 :https://qinwenfeng.com/jiebaR/ )
The jieba-analysis tool for java.(基于结巴分词词库实现的更加灵活优雅易用,高性能的 java 分词实现。支持词性标注。)
We use phonetics as a feature to create a joint semantic-phonetic embedding and improve the neural machine translation between Chinese and Japanese. 🥳
Splits zh_TW/zh_CN input, looks up each part in Hokkien dictionary, then merges dict entries and original input back to a Mandarin -> Hokkien translation prompt - LLMs without Hokkien knowledge can learn from the prompt and give a decent translation