llm-compressor good first issues (Python)

Remove Python 3.10 Support about 1 hour ago

enhancement good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] Enable GPU offloading for reduced weight movement 1 day ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Determine how and why Subgraphs are not covering all modules 8 days ago

bug good first issue good follow-up issue tracing

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] Replace nested tensor logic with pytree 9 days ago

enhancement good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] [Help Wanted] `max_workers="auto"` for Model Free PTQ 14 days ago

enhancement good first issue good follow-up issue model_free_ptq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Centralize SSOT for module fusion 22 days ago

enhancement good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Support programmatic 3d moe repacking 28 days ago

enhancement good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Tests] Add example tests to cover `compress_xxx.py` scripts in compressed-tensors about 1 month ago

enhancement good first issue compressed-tensors good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: Loading models with DDP leads to race condition on HF `cached_files` about 1 month ago

bug good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Remove iMatrixGatherer about 1 month ago

enhancement good first issue stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: Noisy gpt_oss test about 2 months ago

bug good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug] Fix ignore list for linear_attn with Qwen3.8 about 2 months ago

bug good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Kimi-K3] Integrate KimiSparseMoeBlock with LinearExperts2D for REAP Support 2 months ago

enhancement good first issue moe

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Update `oneshot` args to match `*Arguments` config classes 2 months ago

good first issue stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

cheap KLD metric 2 months ago

enhancement good first issue stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Performance] Speed up subgraph tracing for large models 3 months ago

enhancement good first issue tracing

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Fast KLD metric 3 months ago

enhancement good first issue stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[ModelFreePTQ] Memory issues with multi-gpu 3 months ago

enhancement good first issue model_free_ptq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

AWQ: no architecture mappings for looped/weight-shared models (NanbeigeForCausalLM) + OOM on the basic pipeline 3 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[AWQ] add option to take smooth layer quantization into accout 3 months ago

enhancement good first issue awq stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[oneshot] Add pre-processing check for model 3 months ago

bug good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Python 3.10+] Modernize codebase with Python 3.10+ idioms and features 4 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[GPT-OSS] Expanded Support for Activation Quantization 5 months ago

enhancement good first issue good follow-up issue stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[AWQ] Step 3.5 Mappings 5 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request][Help Wanted] Convert AutoAWQ checkpoints to compressed-tensors 5 months ago

enhancement good first issue compressed-tensors good follow-up issue awq wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

GPTQ ActivationOrdering support for per-channel quantization 5 months ago

enhancement good first issue gptq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Update Lifecycle.sequential_epoch_end to use a list of modules instead of a subgraph 6 months ago

enhancement good first issue Refactor

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[oneshot] respsect dataset args like max_seq_length and num_calibration_samples when a dataset is passed in 6 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Add weight activation ordering for fp8 block 6 months ago

enhancement good first issue fp8

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Refactor] Refactor splits to only use the "calibration" split 6 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] Allow targeting multiples of sequential targets 6 months ago

enhancement good first issue tracing

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Refactor] Remove `PreprocessingFunctionRegistry` 7 months ago

enhancement good first issue Refactor

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Refactor] Remove `LogarithmicEqualizationModifier` 7 months ago

enhancement good first issue Refactor

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[GPTQ] Add support for block-wise quantization 7 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Refactor Activation Ordering across LLMC and CT 7 months ago

enhancement good first issue gptq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Restructure and simplify the AWQModifier to be similar to SmoothQuant 7 months ago

enhancement good first issue good follow-up issue awq keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Good First Issue] Rename `offload_model` and `dispatch_for_sequential` to `set_onload_device` 7 months ago

enhancement good first issue compressed-tensors

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug/ Feature]: Improve the MSE observer, investigate regression 7 months ago

bug enhancement good first issue nvfp4 wNa16 stale

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[AWQ] Topological order for mappings for AWQ (implement and test) 8 months ago

enhancement good first issue awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Remove training loggers 8 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Evaluate AWQ + FP8 Example 8 months ago

good first issue awq fp8

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[SmoothQuant] DEEPSEEK_V2 regex misses q_a_proj 8 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Move GPTQ out of modifiers/quantization into modifiers 8 months ago

enhancement good first issue gptq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Optimize functionality using torch.compile 8 months ago

enhancement good first issue keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Quantization support (GPTQ/AWQ) for Step3-VL-10B 8 months ago

enhancement good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

W4A16 quantization option to handle/ignore layers with columns%groupsize != 0 8 months ago

enhancement good first issue compressed-tensors gptq wNa16 keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Update models used in tests 9 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Add non-uniform test cases to the e2e tests 9 months ago

good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Use torch.compile to speed up GPTQ algorithm 9 months ago

enhancement good first issue keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[AWQ]: Add mappings for Qwen3-Next-80B-A3B architecture 9 months ago

bug good first issue qwen awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: The `sequential_targets` parameter passed to `oneshot()` is not used by SparseGPTModifier 9 months ago

bug good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Support Fp8 Block Quant for shapes not divisible by 128 9 months ago

enhancement good first issue good follow-up issue fp8

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: `MagnitudePruningModifier` get_layers_params() missing 1 required positional argument: 'module' 9 months ago

bug good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Refactor matching logic 9 months ago

good first issue compressed-tensors keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Deprecate the `marlin24` compressor 9 months ago

good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Add Additional Model Mappings for `AWQ` and `SmoothQuant` 9 months ago

enhancement good first issue keep-open

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Generate and validate AWQ W4AFp8 Example 9 months ago

vllm good first issue awq fp8 wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Generate and validate W4AFp8 GPTQ Example 9 months ago

vllm good first issue gptq fp8 wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Generate W4AFp8 Examples for GPTQ and AWQ 9 months ago

vllm good first issue good follow-up issue gptq awq fp8 wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Feature Request: Example script for Qwen3-Reranker 9 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Remove src/llmcompressor/utils/fsdp and all uses 9 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

deduplicate is_attention_module between compressed-tensors and llm-compressor 9 months ago

good first issue compressed-tensors

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] Debug messages when using the `LengthAwareSampler` 9 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[RFC][AWQ] Add masking support for MSE loss & mean activation calculation 9 months ago

enhancement good first issue good follow-up issue awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Tokenzier warning messages 9 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: versions 0.8 AND 0.9: Qwen3-VL is saved with an incorrect format for any Quantization only when loaded locally. "You should set the `fix_mistral_regex=True`" when loaded with vLLM 10 months ago

bug good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

KV Cache Quantization support deepseek v3 10 months ago

enhancement good first issue compressed-tensors

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature] Support Zero-point Decompression 10 months ago

enhancement good first issue compressed-tensors good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: suggest to create __init__.py under src/llmcompressor/pipelines/sequential/ast_utils 10 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Update example to apply compressed-tensors to a model 10 months ago

enhancement good first issue compressed-tensors good follow-up issue wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Bug]: Qwen2-Audio-7B-support 10 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request] Select quantization via modifier arguments 11 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request][Help Wanted] Convert AutoAWQ checkpoints to compressed-tensors 11 months ago

enhancement good first issue compressed-tensors good follow-up issue awq wNa16

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Use the updated MoE calibration interface in Qwen3-VL-MoE 12 months ago

good first issue qwen

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Use the updated MoE calibration interface in Qwen3-Next 12 months ago

good first issue qwen

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request] Respect `HF_ENDPOINT` instead of using HUGGINGFACE_CO_URL_HOME 12 months ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Remove `sparse_logs`, only log to file if environment variable is set 12 months ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Create AWQ guide for llm-docs about 1 year ago

AI Summary: The `vllm` documentation for AWQ quantization is outdated, referencing the deprecated `AutoAWQ` library. This issue requests updating the existing `vllm` AWQ guide to utilize the `llm-compressor` APIs and workflow, which now handles AWQ functionality. Additionally, all related links within the `vllm` quantization documentation must be updated to reflect these changes.

Complexity: 3/5

documentation enhancement good first issue awq

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Add AWQ E2E tests about 1 year ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Move OBCQ to SparseGPT (preserve modifiers/pruning/, maintain import compatibility) about 1 year ago

good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Help Wanted] Refactor/ Clean up MoE calibration logic about 1 year ago

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

[Feature Request] [Help Wanted] Parallel Calibration and Model Optimization about 1 year ago

enhancement good first issue good follow-up issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Add an AWQ lm-eval test about 1 year ago

AI Summary: Implement an lm-eval test case for the LLM Compressor library, specifically using the AWQ quantization algorithm on the meta-llama/Meta-Llama-3-8B-Instruct model. This involves creating a configuration file similar to existing examples, running lm-eval for baseline performance, applying AWQ quantization using the library, and then re-running lm-eval to assess the quantized model's performance. The results should be comparable to existing tests.

Complexity: 4/5

enhancement good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Update ReadME for `quantization_2of4_sparse_w4a16` about 1 year ago

AI Summary: Update the README file for the `quantization_2of4_sparse_w4a16` example to reflect the updated API and steps shown in the `llama7b_sparse_w4a16.py` script. This involves modifying the README to accurately represent the current code and process for applying 2:4 sparse W4A16 quantization.

Complexity: 2/5

documentation good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

Doc: Sequential Onloading device_map documentation could be clearer over 1 year ago

AI Summary: Update the documentation for sequential onloading in the LLM Compressor library to explicitly show the `device_map` argument set to `None` in the `from_pretrained` method, clarifying how the model is initially loaded onto the CPU. This involves modifying the README example code and potentially adding a brief explanation.

Complexity: 2/5

documentation good first issue

vllm-project/llm-compressor

3.5K

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python

#compression#quantization

vllm-project/llm-compressor

Open Issues Need Help