bpe-tokenizer

Open repos and models their makers tagged bpe-tokenizer, most starred first. The same for everyone.

Language and translationRepo

PythonGPT

A 54M parameter GPT transformer implemented from scratch in raw PyTorch (no HuggingFace). Trained on 1GB of filtered Python code from bigcode/the-stack-dedup, fine-tuned on 863 curated algorithm implementations. Features BPE tokenization, top-p nucleus sampling, constrained decoding via Python's tokenize module, and a Gemini-inspired Streamlit UI.

  • 1
  • Python
  • MIT
  • 5 mo ago
Arnav-TyagiiTake it
AI on my own machineRepo

Build-and-Finetune-LLM-From-Scratch-Deploy-via-vLLM-AWS-GCP

A complete end-to-end learning repo covering everything from building Large Language Models (LLMs) from scratch to mastering practical deep learning with PyTorch. Includes tokenizer coding, transformers, attention, training loops, model finetuning, and hands-on PyTorch projects.

  • 0
  • Jupyter Notebook
  • MIT
  • 7 mo ago
MDalamin5Take it
Search GitHub and Hugging Face for bpe-tokenizer

Get , open it once and sign in. Then come back here and press the button again.

bpe-tokenizer: open source repos and models on Take My Repo