LightLLM
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
- 4.4k
- Python
- Apache-2.0
- 1 min ago
ModelTC
https://light-ai.top/
Built from ModelTC's public GitHub profile. Not on Take My Repo yet, so there are no posts or updates here.
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Qwen-Image-Lightning: Speed up Qwen-Image model with distillation
[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.
[CVPR 2024 Highlight & TPAMI 2025] This is the official PyTorch implementation of "TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models".
Summary of system papers/frameworks/codes/tools on training or serving large model
In our implementation of Qwen-Image-Edit, we employ block causal attention to improve inference speed.
Official implementation of the EMNLP23 paper: Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
LightTTS is a lightweight TTS inference framework optimized for CosyVoice2 and CosyVoice3, enabling fast and scalable speech synthesis in Python and supports stream and bistream modes.
Built upon Megatron-Deepspeed and HuggingFace Trainer, EasyLLM has reorganized the code logic with a focus on usability. While enhancing usability, it also ensures training efficiency.
[ICML 2025] This is the official PyTorch implementation of "🎵 HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration".
LightVAE: Fast video VAE decoders for Wan2.1, Wan2.2, and MiniMax-H3, plus a compressed H3 reference encoder. Pretrained weights and ComfyUI workflows.
[ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models"