vllm-project/llm-compressor

[Feature] Enable GPU offloading for reduced weight movement 3 days ago

enhancement good first issue

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Python 3.10+] Modernize codebase with Python 3.10+ idioms and features 10 days ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Remove iMatrixGatherer about 1 month ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[GPT-OSS] Expanded Support for Activation Quantization about 1 month ago

enhancement good first issue good follow-up issue stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[AWQ] Step 3.5 Mappings about 1 month ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request][Help Wanted] Convert AutoAWQ checkpoints to compressed-tensors about 2 months ago

enhancement good first issue compressed-tensors good follow-up issue awq wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Update `oneshot` args to match `*Arguments` config classes about 2 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

GPTQ ActivationOrdering support for per-channel quantization 2 months ago

enhancement good first issue gptq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

cheap KLD metric 2 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Update Lifecycle.sequential_epoch_end to use a list of modules instead of a subgraph 2 months ago

enhancement good first issue Refactor

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Fast KLD metric 2 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[oneshot] respsect dataset args like max_seq_length and num_calibration_samples when a dataset is passed in 2 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[AWQ] add option to take smooth layer quantization into accout 3 months ago

enhancement good first issue awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Add weight activation ordering for fp8 block 3 months ago

enhancement good first issue fp8

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Refactor] Refactor splits to only use the "calibration" split 3 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] Allow targeting multiples of sequential targets 3 months ago

enhancement good first issue tracing

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Refactor] Remove `PreprocessingFunctionRegistry` 3 months ago

enhancement good first issue Refactor

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Refactor] Remove `LogarithmicEqualizationModifier` 3 months ago

enhancement good first issue Refactor

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[GPTQ] Add support for block-wise quantization 3 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Refactor Activation Ordering across LLMC and CT 3 months ago

enhancement good first issue gptq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Restructure and simplify the AWQModifier to be similar to SmoothQuant 4 months ago

enhancement good first issue good follow-up issue awq keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Good First Issue] Rename `offload_model` and `dispatch_for_sequential` to `set_onload_device` 4 months ago

enhancement good first issue compressed-tensors

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug/ Feature]: Improve the MSE observer, investigate regression 4 months ago

bug enhancement good first issue nvfp4 wNa16 stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[AWQ] Topological order for mappings for AWQ (implement and test) 4 months ago

enhancement good first issue awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Remove training loggers 4 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Evaluate AWQ + FP8 Example 4 months ago

good first issue awq fp8

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[SmoothQuant] DEEPSEEK_V2 regex misses q_a_proj 4 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Move GPTQ out of modifiers/quantization into modifiers 4 months ago

enhancement good first issue gptq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Optimize functionality using torch.compile 5 months ago

enhancement good first issue keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Quantization support (GPTQ/AWQ) for Step3-VL-10B 5 months ago

enhancement good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

W4A16 quantization option to handle/ignore layers with columns%groupsize != 0 5 months ago

enhancement good first issue compressed-tensors gptq wNa16 keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Update models used in tests 5 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Add non-uniform test cases to the e2e tests 5 months ago

good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Use torch.compile to speed up GPTQ algorithm 5 months ago

enhancement good first issue keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[AWQ]: Add mappings for Qwen3-Next-80B-A3B architecture 5 months ago

bug good first issue qwen awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: The `sequential_targets` parameter passed to `oneshot()` is not used by SparseGPTModifier 5 months ago

bug good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Support Fp8 Block Quant for shapes not divisible by 128 5 months ago

enhancement good first issue good follow-up issue fp8

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: `MagnitudePruningModifier` get_layers_params() missing 1 required positional argument: 'module' 5 months ago

bug good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Refactor matching logic 6 months ago

good first issue compressed-tensors keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Deprecate the `marlin24` compressor 6 months ago

good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Add Additional Model Mappings for `AWQ` and `SmoothQuant` 6 months ago

enhancement good first issue keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Generate and validate AWQ W4AFp8 Example 6 months ago

vllm good first issue awq fp8 wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Generate and validate W4AFp8 GPTQ Example 6 months ago

vllm good first issue gptq fp8 wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Generate W4AFp8 Examples for GPTQ and AWQ 6 months ago

vllm good first issue good follow-up issue gptq awq fp8 wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Feature Request: Example script for Qwen3-Reranker 6 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Remove src/llmcompressor/utils/fsdp and all uses 6 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

deduplicate is_attention_module between compressed-tensors and llm-compressor 6 months ago

good first issue compressed-tensors

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] Debug messages when using the `LengthAwareSampler` 6 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[RFC][AWQ] Add masking support for MSE loss & mean activation calculation 6 months ago

enhancement good first issue good follow-up issue awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Tokenzier warning messages 6 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: versions 0.8 AND 0.9: Qwen3-VL is saved with an incorrect format for any Quantization only when loaded locally. "You should set the `fix_mistral_regex=True`" when loaded with vLLM 6 months ago

bug good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

KV Cache Quantization support deepseek v3 7 months ago

enhancement good first issue compressed-tensors

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] Support Zero-point Decompression 7 months ago

enhancement good first issue compressed-tensors good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: suggest to create __init__.py under src/llmcompressor/pipelines/sequential/ast_utils 7 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Update example to apply compressed-tensors to a model 7 months ago

enhancement good first issue compressed-tensors good follow-up issue wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: Qwen2-Audio-7B-support 7 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request] Select quantization via modifier arguments 7 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request][Help Wanted] Convert AutoAWQ checkpoints to compressed-tensors 7 months ago

enhancement good first issue compressed-tensors good follow-up issue awq wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Use the updated MoE calibration interface in Qwen3-VL-MoE 8 months ago

good first issue qwen

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Use the updated MoE calibration interface in Qwen3-Next 8 months ago

good first issue qwen

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request] Respect `HF_ENDPOINT` instead of using HUGGINGFACE_CO_URL_HOME 8 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Remove `sparse_logs`, only log to file if environment variable is set 9 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Create AWQ guide for llm-docs 9 months ago

AI Summary: The `vllm` documentation for AWQ quantization is outdated, referencing the deprecated `AutoAWQ` library. This issue requests updating the existing `vllm` AWQ guide to utilize the `llm-compressor` APIs and workflow, which now handles AWQ functionality. Additionally, all related links within the `vllm` quantization documentation must be updated to reflect these changes.

Complexity: 3/5

documentation enhancement good first issue awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Add AWQ E2E tests 10 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Move OBCQ to SparseGPT (preserve modifiers/pruning/, maintain import compatibility) 10 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Refactor/ Clean up MoE calibration logic 10 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request] [Help Wanted] Parallel Calibration and Model Optimization 10 months ago

enhancement good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Add an AWQ lm-eval test 11 months ago

AI Summary: Implement an lm-eval test case for the LLM Compressor library, specifically using the AWQ quantization algorithm on the meta-llama/Meta-Llama-3-8B-Instruct model. This involves creating a configuration file similar to existing examples, running lm-eval for baseline performance, applying AWQ quantization using the library, and then re-running lm-eval to assess the quantized model's performance. The results should be comparable to existing tests.

Complexity: 4/5

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Update ReadME for `quantization_2of4_sparse_w4a16` 12 months ago

AI Summary: Update the README file for the `quantization_2of4_sparse_w4a16` example to reflect the updated API and steps shown in the `llama7b_sparse_w4a16.py` script. This involves modifying the README to accurately represent the current code and process for applying 2:4 sparse W4A16 quantization.

Complexity: 2/5

documentation good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Doc: Sequential Onloading device_map documentation could be clearer about 1 year ago

AI Summary: Update the documentation for sequential onloading in the LLM Compressor library to explicitly show the `device_map` argument set to `None` in the `from_pretrained` method, clarifying how the model is initially loaded onto the CPU. This involves modifying the README example code and potentially adding a brief explanation.

Complexity: 2/5

documentation good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Open Issues Need Help