auto-round good first issues (Python)

[feat] For FP8 models, keep layers specified in `ignore_layers` in their original FP8 format about 1 month ago

good first issue

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

[Feature]: Added breakpoint resume functionality to Model-Free. about 2 months ago

enhancement good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Ensuring correct device dispatch after module replacement about 2 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

[Feature]: Extend the FP8 KV/Attn Calibration to Per-head 3 months ago

enhancement good first issue call for help

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

[Enhancement]: Replace get_module, set_module with torch API get_submodule, set_submodule 6 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

[Feature]: Support Re‑Quantizing Per‑Tensor FP8 Models to Other Dtypes 9 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Support FP8 MLA calibration (for Deepseek) 9 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Maintain `no_init_weights` support in AutoRound after v5 9 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Add support for re-quantize MXFP4/MXFP8 models 9 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Deuqantize block-wise FP8 weight on CPU (G2-Specific) 10 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Align the first input scale of MoE experts for FP8 dispatch 10 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

intel/auto-round

Open Issues Need Help