mathispink.github.io
- 0
- JavaScript
- MIT
- 2 mo ago
mathispink
👀🧠(x) | x ∈ {👀🧠,🤖} (tricking rocks into remembering)
Built from mathispink's public GitHub profile. Not on Take My Repo yet, so there are no posts or updates here.
Attention Saver lets you extract entire attention matrices or row-wise statistics (e.g. entropy) from any HuggingFace causal LLM layer for ultra-long context when using flash-attention without running out of GPU memory.
A Tensorflow implementation of Batch Entropy Regularization (Peer et al 2022).