Open Issues Need Help
View All on GitHub fa2-fp8kv: EngineCore dies when fp8 KV runs 2+ sequences and a prefill chunk exceeds 2048 (not TP-specific) about 1 month ago
bug help wanted
noonghunna/club-3090 noonghunna/club-3090 Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp, ik_llama) and model-agnostic. Currently shipping Qwen3.6-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1× and 2× cards.
Open project →
2.3K
Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp, ik_llama) and model-agnostic. Currently shipping Qwen3.6-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1× and 2× cards.
Python