DynMoE
[ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
- 165
- Python
- Apache-2.0
- 17 mo ago
[ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
Dynamic weight generation for recursive transformers via input-conditioned LoRA modulation
(NeurIPS-2019 MicroNet Challenge - 3rd Winner) Open source code for "SIPA: A simple framework for efficient networks"
LLM reasoning: use draft agreement to route queries and allocate adaptive thinking budgets without training.
Local EDU AI LAB for looped (latent-recurrent) transformers - run, patch and measure loop counts on local GPU