gptq
Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
- 2.4k
- Python
- Apache-2.0
- 3 y ago
Repository for the QUIK project, enabling the use of 4bit kernels for generative inference - EMNLP 2024
Repo IST-DASLab/QUIK, read from GitHub 1 min ago. IST-DASLab has not joined Take My Repo.
IST-DASLab/QUIK is a public repo on GitHub by IST-DASLab. Repository for the QUIK project, enabling the use of 4bit kernels for generative inference - EMNLP 2024. It declares the Apache-2.0 license on GitHub. It is written mainly in C++, has 186 stars and last changed on 2024-04-16. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-21.
IST-DASLab/QUIK is a public repo on GitHub by IST-DASLab. Repository for the QUIK project, enabling the use of 4bit kernels for generative inference - EMNLP 2024.
It declares the Apache-2.0 license on GitHub.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2024-04-16. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".
Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".