int4
neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
- 2.7k
- Python
- Apache-2.0
- 5 h ago
franken_ocr
Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.
- 347
- Rust
- 9 d ago
oido
Oído: open-vocabulary speech recognition on a $5 ESP32-S3, faster than real time. No cloud, no NPU. English + Spanish.
- 140
- C
- GPL-3.0
- 6 d ago
qwen38-flash-next-2x3090
Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).
- 145
- Python
- Apache-2.0
- 5 wk ago
vecgo
🧬🔍 Vecgo is a pure Go, embeddable, hybrid vector database designed for high-performance production workloads. It combines commit-oriented durability with HNSW + DiskANN indexing for best-in-class performance.
- 24
- Go
- Apache-2.0
- 10 mo ago
Mistral-Small-3.1-24B-Instruct-2503-GPTQ-4b-128g
Image text to text model, used with vllm.
- 17
- 25k
- apache-2.0
- 20 mo ago
MiniMax-M2.7-REAP-172B-A10B-AutoRound-W4A16
Text generation model, used with transformers.
- 11
- 179
- other
- 7 mo ago
Qwen3.8-27B-abliterated-AWQ-INT4
Text generation model, used with transformers.
- 3
- 1.3k
- apache-2.0
- 3 mo ago
DeepSeek-V4.1-Flash-W4A16-Engram-AutoRound
Text generation model, used with transformers.
- 2
- 155
- mit
- 2 mo ago
nanbeige43-33b-int4-m5-ultra
Nanbeige4.3-33B-INT4 on one M5 Ultra with TensorFold native Metal: Nanbeige's own QAT int4 release as published (47-48 tok/s), on upstream TensorFold main
- 0
- Python
- MIT
- 2 wk ago

