linear-attention

Open repos and models their makers tagged linear-attention, most starred first. The same for everyone.

Language and translationRepo

RWKV-LM

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

  • 15k
  • Python
  • Apache-2.0
  • 2 wk ago
BlinkDLTake it
Search GitHub and Hugging Face for linear-attention

Get , open it once and sign in. Then come back here and press the button again.