gptq
Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
- 2.4k
- Python
- Apache-2.0
- 3 y ago
IST-DASLab
Built from IST-DASLab's public GitHub profile. Not on Take My Repo yet, so there are no posts or updates here.
Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".
Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".
Repository for the QUIK project, enabling the use of 4bit kernels for generative inference - EMNLP 2024
Gumbel-Softmax post-training quantization for LLMs (1–3 bit scalar, INT/GGUF-compatible).
Code for the NeurIPS 2022 paper "Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning".
RCO: Optimization of Non-Decomposable Objectives under Exact Budget Constraints for LLM Quantization and Pruning
Code accompanying the NeurIPS 2020 paper: WoodFisher (Singh & Alistarh, 2020)
Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)
Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry