pegainfer-project/pegainfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

631 stars 97 forks 631 watchers Rust Apache License 2.0

cuda cuda-kernels deepseek gpu inference inference-engine kimi kimi-k2 kv-cache llm llm-inference llm-serving model-serving moe openai-api paged-attention qwen qwen3 rust vllm

View on GitHub Website

1 Open Issue Need Help Last updated: Aug 7, 2026

Open Issues Need Help

View All on GitHub

qwen35: migrate GDN prefill to FlashInfer and retire Triton AOT where supported about 3 hours ago

enhancement help wanted qwen35 hw:1-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm