UL2_5
GPU-optimized UL2 mixture-of-denoisers data collator for T5/FLAN encoder-decoder pretraining. Supports span corruption, prefix LM, infilling, curriculum learning, Flash Attention unpadding, and HuggingFace Trainer integration.
- 0
- Python
- Apache-2.0
- 11 mo ago
