Open Issues Need Help
View All on GitHub [RFC]: Support ViT Full CUDA Graph (Tracker) about 2 hours ago
help wanted RFC multi-modality kimi
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Tracking Whisper feature requests 3 days ago
help wanted good first issue feature request keep-open multi-modality
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: Wrong timestamps if audio > 30s 10 days ago
bug help wanted good first issue unstale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Support `Phi4Flash` model in V1 16 days ago
help wanted good first issue feature request unstale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[RFC]: Limit the use of envvars in vLLM 25 days ago
good first issue RFC
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
Upgrade to Transformers v5 26 days ago
help wanted
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug][RL]: Port Conflict 26 days ago
bug help wanted good first issue stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Misc]: CMake Clean-up / Refactor Tasks 29 days ago
help wanted good first issue keep-open unstale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Performance]: Compiled `QuantFP8.forward_native` group quantization (1, 128) slower than CUDA on H100/RTX5090 about 1 month ago
help wanted performance torch.compile keep-open
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Error Logging Redesign about 1 month ago
help wanted feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: Add tests for all parallel sampling parameter combinations about 1 month ago
bug good first issue keep-open stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Optimize collectives in TP MoE case using torch.compile pass about 2 months ago
help wanted good first issue performance feature request torch.compile unstale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: `CustomOp` cleanup about 2 months ago
help wanted good first issue feature request keep-open startup-ux
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] InternVL2 about 2 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature][Cleanup]: Unify `vllm.utils.flashinfer` and `vllm.model_executor.layers.quantization.utils.flashinfer_utils` about 2 months ago
help wanted good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Enable LoRA support for tower and connector in more MM models about 2 months ago
help wanted feature request multi-modality
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: Certain Ranks Take a Look Time to Load Weights about 2 months ago
bug help wanted
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[torch.compile] E2E correctness testing for fusions about 2 months ago
help wanted torch.compile
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Better Flashinfer compilation logging about 2 months ago
help wanted feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Integrate fused `kMoEFinalizeARResidualRMSNorm` from FlashInfer about 2 months ago
help wanted feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue RFC keep-open
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Perf][Kernel] Adopt PTX 9.4 `ldmatrix.s8.s4` (hardware INT4→INT8 expanding load) in W4A8-INT8 paths 2 months ago
good first issue performance quantization
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request torch.compile unstale startup-ux
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
good first issue feature request keep-open
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Improve DCP error messages 3 months ago
good first issue feature request unstale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request unstale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Use kernel abstraction for fp4 scaled mm 3 months ago
help wanted feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Add Triton implementation of NVFP4 GEMM 3 months ago
good first issue performance feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[CI] Transformers backend llava-onevision test needs default_torch_num_threads=1 to avoid hanging 3 months ago
help wanted
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: Improve `--kv-cache-dtype` behavior when checkpoint specifies `kv_cache_quant_algo` 3 months ago
bug good first issue unstale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted feature request stale nvidia quantization
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Qwen3 Omni Transcriptions 4 months ago
good first issue feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted performance torch.compile stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] Tarsier2ForConditionalGeneration 4 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[torch.compile][Performance]: Unwrap custom ops and improve fusion (Inductor and custom) 5 months ago
help wanted performance torch.compile stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted torch.compile stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] IsaacForConditionalGeneration 5 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] Base model and LoRA used in test has incorrect `tokenizer_config.json` 5 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: FlashInfer Incompatible with Sleep Mode 5 months ago
bug help wanted stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Support DCP with FP8 KV Cache 5 months ago
help wanted feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Fused Kernel for GPT-OSS Router 5 months ago
help wanted good first issue feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Usage]: How to get query embeddings from ColBERT? 6 months ago
good first issue usage
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request torch.compile stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted ci/build
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[CI Failure]: Fusion E2E TP2 Quick (H100) 6 months ago
help wanted torch.compile ci-failure needs reproduction
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug good first issue stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: Worse EAGLE3 acceptance rates on MRV2 6 months ago
bug help wanted
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[CI Failure]: PyTorch Compilation Passes Unit Tests 6 months ago
help wanted torch.compile ci-failure
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted performance feature request torch.compile keep-open
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted multi-modality
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] Distributed shutdown test timetout 7 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Remove MXFP4 Logic From `fused_experts` 7 months ago
help wanted good first issue feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted feature request keep-open multi-modality
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] SarvamMLAForCausalLM 7 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] MiniCPMV cannot apply processor 7 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] ColBERTJinaRobertaModel 7 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] HCXVisionForCausalLM 7 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Transformers v5] Mistral multimodal models 7 months ago
help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Consolidate GPTQ Quantization 7 months ago
help wanted feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: Qwen3-VL 2:4 sparsity llm-compressor RuntimeError: shape mismatch (0.12, 0.13rc2) 7 months ago
bug help wanted good first issue stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug] DCP/DBO: 'NoneType' error building attention_metadata during DeepSeek-V3.1 deployment dummy run 7 months ago
bug help wanted stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Refactor `aiter_shared_expert_fusion` 7 months ago
help wanted stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature][Help Wanted]: Add tuning script and config files for Mamba selective_state_update kernel 8 months ago
help wanted good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
bug help wanted good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: matryoshka need gpu-memory??? 8 months ago
bug good first issue
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Remove Chunking From FusedMoE 8 months ago
help wanted good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: `swap_space` parameter is unused and useless 8 months ago
bug good first issue keep-open
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted feature request torch.compile stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Bug]: AsyncTP pass has poor perf on B200 8 months ago
bug help wanted torch.compile keep-open
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Qwen-ASR Forced Aligner 8 months ago
good first issue feature request
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted feature request torch.compile
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Better base64 to torch tenser 8 months ago
good first issue feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Implement naive prepare/finalize class to replace naive dispatching in fused_moe/layer.py 8 months ago
help wanted good first issue feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
[Feature]: Disable logging `/metrics` 9 months ago
help wanted good first issue feature request stale
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer
help wanted good first issue feature request torch.compile
vllm-project/vllm vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Open project →
85.0K
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe#openai#pytorch#qwen#qwen3#tpu#transformer