flashinfer good first issues (Python)

bug(sampling): top_k_arr/min_p_arr are validated against input rows but indexed by output row, reading out of bounds when indices is longer than probs 7 days ago

good first issue op: misc

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

DGX Spark (SM121) Current Support Audit about 1 month ago

good first issue wip op: gemm op: misc arch: sm12x op: linear attention

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

[Compilation] GDN decode cold kernel compilation time is too long on SM90/100/120/121 about 1 month ago

good first issue op: linear attention

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

[Bug] sm12x MoE CuTe-DSL kernels are never disk-cached — cute.compile hardcodes no_cache=True, so every fresh engine re-pays 15–40 s MLIR compiles per kernel shape about 2 months ago

good first issue needs-triage model: qwen3.5 / 3.6 / 3.8 op: moe

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

flashinfer.nvfp4_quantize: make CuTe-DSL backend's `a_global_sf` arg a host side arg (to align CuTe-DSL backend with CUDA backend) 3 months ago

good first issue op: misc

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

Next fix for gemm-allreduce two-shot 3 months ago

good first issue cute-dsl refactor op: gemm op: comm

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

Potentially superfluous check that disables non gated activations in the cutlass fused moe API 6 months ago

good first issue priority: should have (P1) op: moe

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

MoE autotune print a lot failed kernel on SM120 7 months ago

bug good first issue priority: should have (P1) op: moe

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

Integration needed for Qwen3-Coder-480B-A35B-Instruct-NVFP4 8 months ago

help wanted priority: must have (P0) needs-triage op: moe-routing

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

BF16 hidden_states for trtllm_fp4_block_scale_moe 8 months ago

good first issue op: moe

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

Autotuning failsafe fallback to top1 tactic 9 months ago

feature request good first issue op: moe

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

Inaccurate API Docstrings for Attention Prefill 11 months ago

documentation good first issue

flashinfer-ai/flashinfer

5.9K

FlashInfer: Kernel Library for LLM Serving

Python

#attention#cuda#distributed-inference#gpu#jit#large-large-models#llm-inference#moe#nvidia#pytorch

flashinfer-ai/flashinfer

Open Issues Need Help