attention-saver
Attention Saver lets you extract entire attention matrices or row-wise statistics (e.g. entropy) from any HuggingFace causal LLM layer for ultra-long context when using flash-attention without running out of GPU memory.
- 0
- Python
- Apache-2.0
- 14 mo ago