rmsnorm

Open repos and models their makers tagged rmsnorm, most starred first. The same for everyone.

Language and translationRepo

H64LM

249M-param MoE transformer built from scratch in PyTorch. GQA, RoPE, SwiGLU, sparse MoE with 3 aux losses, AMP training loop no Trainer abstractions. Architecture mirrors LLaMA/Mistral/Mixtral decisions, fully inspectable.

  • 19
  • Jupyter Notebook
  • Apache-2.0
  • 6 mo ago
Haiderkhan64Take it
Search GitHub and Hugging Face for rmsnorm

Get , open it once and sign in. Then come back here and press the button again.

rmsnorm: open source repos and models on Take My Repo