pegainfer good first issues (Rust)

Reduce duplicated CPU operator oracles in the HD256/HD512 QK-RoPE smokes 6 days ago

enhancement help wanted hw:1-gpu stale

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

perf(qwen): reuse PrefillPagedPlan for uncompiled-GQA decode 8 days ago

enhancement help wanted qwen3 qwen35 hw:1-gpu stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

Gate the P2P/RDMA offload data plane behind a cargo feature 9 days ago

enhancement help wanted good first issue hw:none stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

perf(dsv2lite): move decode attention and KV-cache access to CUDA 17 days ago

enhancement help wanted dsv2lite hw:multi-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

qwen35: migrate GDN prefill to FlashInfer and retire Triton AOT where supported 17 days ago

enhancement help wanted qwen35 hw:1-gpu stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

frontend: two shared error messages name a single model 24 days ago

bug good first issue hw:none

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

metrics: report real prefix-cache query/hit counters from the qwen3 scheduler 24 days ago

enhancement help wanted qwen3 observability hw:1-gpu stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

Make explicitly selected Qwen model gates fail closed 24 days ago

bug help wanted qwen3 qwen35 hw:1-gpu stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

perf(kimi-k2): compact local top-1 readback into one pinned transfer about 1 month ago

enhancement help wanted kimi-k2 stale hw:multi-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

kimi-k2: add an integration-test surface (tests/) about 1 month ago

enhancement help wanted kimi-k2 hw:none stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

kimi-k2: gate DeepEP decode graph-replay numerics (full-bucket graph-vs-eager) about 1 month ago

enhancement help wanted kimi-k2 stale hw:multi-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

dsv2lite: reduce mixed and long-prompt HTTP TTFT with prefill scheduling about 1 month ago

enhancement help wanted dsv2lite stale hw:multi-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

glm52: DSpark and prefix caching are mutually exclusive — draft context cold-start at the cache-hit boundary about 1 month ago

enhancement help wanted stale glm52 hw:multi-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

perf(dsv2lite): keep NCCL route selection and route plans device-resident about 1 month ago

enhancement help wanted dsv2lite stale hw:multi-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

[Tracking] Dev Dockerfile / devcontainer about 1 month ago

help wanted roadmap hw:none

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

qwen3: scheduler can park on blocking_recv while an async decode-overlap prefill is still in flight about 1 month ago

bug good first issue qwen3 hw:none

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

[Help wanted] Verify Qwen3.5-9B / Qwen3.5-27B on the qwen35-4b crate about 1 month ago

enhancement help wanted qwen35 hw:1-gpu stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

[Help wanted] Verify Qwen3-14B / Qwen3-32B on the existing qwen3 crate about 1 month ago

enhancement help wanted qwen3 hw:1-gpu stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

qwen3: docs/dead-code cleanup — stale records, deleted-tool references, unused consts about 1 month ago

documentation enhancement good first issue qwen3 hw:none stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

frontend: preserve logprobs request semantics and return prompt logprobs instead of HTTP 500 about 2 months ago

bug help wanted hw:1-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

Structured / guided output (grammar-constrained decoding) for reliable tool calls about 2 months ago

enhancement help wanted roadmap hw:1-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

Migrate Gemma 4 onto the step contract about 2 months ago

enhancement help wanted hw:1-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

qwen3: PerToken decode-graph memory grows linearly with batch bucket and is not budgeted 2 months ago

help wanted qwen3 hw:1-gpu stale

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

Keep Gemma 4 ignored gates compilable and define their maintainer runner 2 months ago

enhancement help wanted hw:1-gpu

pegainfer-project/pegainfer

631

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Rust

#cuda#cuda-kernels#deepseek#gpu#inference#inference-engine#kimi#kimi-k2#kv-cache#llm#llm-inference#llm-serving#model-serving#moe#openai-api#paged-attention#qwen#qwen3#rust#vllm

pegainfer-project/pegainfer

Open Issues Need Help