intel/auto-round - 10 Help Wanted Issues

Open Issues Need Help

[Feature]: Extend the FP8 KV/Attn Calibration to Per-head about 11 hours ago

enhancement good first issue call for help

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

[feat] For FP8 models, keep layers specified in `ignore_layers` in their original FP8 format 8 days ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Ensuring correct device dispatch after module replacement 8 days ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

[Enhancement]: Replace get_module, set_module with torch API get_submodule, set_submodule 3 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

[Feature]: Support Re‑Quantizing Per‑Tensor FP8 Models to Other Dtypes 5 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Support FP8 MLA calibration (for Deepseek) 6 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Maintain `no_init_weights` support in AutoRound after v5 6 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Add support for re-quantize MXFP4/MXFP8 models 6 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Deuqantize block-wise FP8 weight on CPU (G2-Specific) 6 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms

Align the first input scale of MoE experts for FP8 dispatch 6 months ago

good first issue

intel/auto-round

1.5K

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

Python

#diffusers#gguf#int4#llms#mxfp4#nvfp4#omni#quantization#rounding#sglang#transformers#vllm#vlms