quantization

Open repos and models their makers tagged quantization, most starred first. The same for everyone.

AI on my own machineRepo

project-maya

Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- and Anthropic-compatible API.

  • 3k
  • C++
  • MIT
  • 7 h ago
mw00Take it
AI on my own machineRepo

HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

  • 2.8k
  • Python
  • Apache-2.0
  • 9 d ago
syv-aiTake it
GitHubRepo

micronet

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 grou

  • 2.3k
  • Python
  • MIT
  • 19 mo ago
666DZY666Take it
Run my own servicesRepo

onnx2tf

A tool for converting ONNX files to LiteRT/TFLite/TensorFlow, PyTorch native code (nn.Module), TorchScript (.pt), state_dict (.pt), Exported Program (.pt2), and Dynamo ONNX. It also supports direct conversion from LiteRT to PyTorch.

  • 1k
  • Python
  • MIT
  • 2 mo ago
PINTO0309Take it
GitHubRepo

tinyengine

[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256KB Memory

  • 970
  • C
  • MIT
  • 24 mo ago
mit-han-labTake it
AI on my own machineRepo

janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

  • 734
  • C
  • GPL-3.0
  • 15 h ago
prabanta-devTake it
Search GitHub and Hugging Face for quantization

Get , open it once and sign in. Then come back here and press the button again.