gqa

Open repos and models their makers tagged gqa, most starred first. The same for everyone.

Language and translationRepo

H64LM

249M-param MoE transformer built from scratch in PyTorch. GQA, RoPE, SwiGLU, sparse MoE with 3 aux losses, AMP training loop no Trainer abstractions. Architecture mirrors LLaMA/Mistral/Mixtral decisions, fully inspectable.

  • 19
  • Jupyter Notebook
  • Apache-2.0
  • 6 mo ago
Haiderkhan64Take it
Search GitHub and Hugging Face for gqa

Get , open it once and sign in. Then come back here and press the button again.