LLMSpeculativeSampling
Fast inference from large lauguage models via speculative decoding
- 925
- Python
- Apache-2.0
- 2 y ago
feifeibear
Democratizing AGI
Built from feifeibear's public GitHub profile, Shanghai, China. Not on Take My Repo yet, so there are no posts or updates here.
Fast inference from large lauguage models via speculative decoding
USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
Compare different hardware platforms via the Roofline Model for LLM inference tasks.
A Distributed ResNet on multi-machines each with one GPU card.
Agent-native Seedance 2.0 short-film studio: cli for AI, canvas for human
a highly-efficient library for deep neural networks based on Sunway TaihuLight supercomputer.
PSTensor provides a way to hack the memory management of tensors in TensorFlow and PyTorch by defining your own C++ Tensor Class.