llm-compressor
gemma-4-26B-A4B-it-FP8-dynamic
Image text to text model, used with transformers.
- 63
- 1M
- apache-2.0
- 8 mo ago
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-FP8
Text generation model, used with transformers.
- 14
- 485k
- 3 mo ago
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-FP8
Image text to text model, used with transformers.
- 7
- 1.7k
- openmdw-1.1
- 5 wk ago
qwen3.5-quantization
Qwen3.5-9B in FP8, INT4 and NVFP4 inside vLLM and SGLang: 1.58× throughput, 37% lower cost per token, same accuracy.
- 0
- Python
- MIT
- 3 d ago
llm-quantization-comparison
Compare GPTQ, AWQ, RTN, SmoothQuant and FP8 quantization on Qwen3-0.6B with LLM Compressor: model size, perplexity, speed and output quality against the BF16 base model.
- 0
- Jupyter Notebook
- 6 d ago
quant-lab
Quantization pipelines for local inference: recipes, exact module coverage, and side-by-side evaluation (KLD / top-1 / PPL, decode + TTFT) against both full-precision and vendor checkpoints.
- 0
- Python
- Apache-2.0
- 4 wk ago
sm120-quant-bench
NVFP4 and W4A16 quantization on consumer Blackwell (sm_120): recipes, an eval harness that detects the damage aggregate benchmarks hide, reproducible numbers.
- 0
- Python
- MIT
- 3 mo ago
agnes-nvfp4
Scripts that fold Agnes-3.0-Flash Preview into a stock Qwen3.5 graph and quantize it to NVFP4 for one DGX Spark
- 0
- Python
- 7 wk ago
course-summary-fast-efficient-llm-inference-vllm
French course summary of “Fast & Efficient LLM Inference with vLLM,” covering inference, quantization, LLM Compressor, PagedAttention, Continuous Batching, benchmarking, and evaluation.
- 0
- Python
- 4 mo ago
quantizing-qwen3-06b
Using vLLM compressor to quantize Qwen3-0.6B using GPTQ to W4A16
- 0
- Python
- 4 mo ago