gptq

Open repos and models their makers tagged gptq, most starred first. The same for everyone.

GitHubRepo

strata-nvfp4

Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8 prompts. Windows / CUDA 13.

  • 255
  • C++
  • MIT
  • 18 h ago
sergqwerTake it
AI on my own machineRepo

llm-inference-bench

Benchmark self-hosted LLM inference across engines (vLLM, Ollama) and quantisation levels (FP16 / INT8 / AWQ / GPTQ). Concurrency sweeps, TTFT / TPOT / P99 tail latency, VRAM curves, Prometheus + Grafana, self-contained HTML reports.

  • 2
  • Python
  • MIT
  • 10 d ago
AI on my own machineRepo

Whetstone

Low-bit LLM inference engine in Rust + CUDA. Converts models to a 4-bit .wstone format (3.75x smaller) and runs them on consumer GPUs. Built for memory-bandwidth-bound decode on Turing, where bytes per weight — not TOPS — sets token throughput.

  • 3
  • Rust
  • Apache-2.0
  • 4 mo ago
GitHubRepo

cutile-forge

CuTileForge v3.0.0: Native tile-fusion compiler & universal quantization engine on NVIDIA CuTile. Zero-ATen CUDA-graph decode at 172 tok/s, 2,377 tok/s batch serving, 13 model families, 70B hybrid on a single 32 GB GPU (sm_89 ↔ sm_120).

  • 1
  • Python
  • MIT
  • 4 h ago
samudraguptoTake it
Search GitHub and Hugging Face for gptq

Get , open it once and sign in. Then come back here and press the button again.