gpillon

gpillon/ignis

OpenAI-compatible inference server for Qwen3.8 on one RTX 5090 / RTX PRO 6000: the 27B at ~170 tok/s (peaks ~370), Flash-Next (125B MoE) at ~100 tok/s, built for local coding agents. Rust + CUDA, Windows and Linux.

1 good first / help-wanted issue · Rust · last activity Oct 25, 2026

34 stars 6 forks 34 watchers Rust Apache License 2.0
blackwell coding-agents cuda inference-engine llm llm-inference local-llm mixture-of-experts nvfp4 openai-api openai-compatible qwen qwen3 rtx-5090 rust self-hosted speculative-decoding
View on GitHub
1 Open Issue Need Help Last updated: Oct 25, 2026

Open Issues Need Help

View All on GitHub
16 GB card reports about 2 hours ago
enhancement help wanted ready-for-human
gpillon/ignis
34

OpenAI-compatible inference server for Qwen3.8 on one RTX 5090 / RTX PRO 6000: the 27B at ~170 tok/s (peaks ~370), Flash-Next (125B MoE) at ~100 tok/s, built for local coding agents. Rust + CUDA, Windows and Linux.

Rust
#blackwell#coding-agents#cuda#inference-engine#llm#llm-inference#local-llm#mixture-of-experts#nvfp4#openai-api#openai-compatible#qwen#qwen3#rtx-5090#rust#self-hosted#speculative-decoding