Open Issues Need Help
View All on GitHub Benchmark against llama.cpp BLAS builds (BLIS / oneAPI / OpenBLAS) on x86 about 2 months ago
enhancement help wanted good first issue
arizqi/cpubrrr arizqi/cpubrrr Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.
Open project →
12
Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.
Rust
#apple-silicon#cpu#gpt-oss#inference#llm#mixture-of-experts#neon#quantization#rust#simd
Improve prompt-processing (prefill) throughput about 2 months ago
enhancement help wanted
arizqi/cpubrrr arizqi/cpubrrr Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.
Open project →
12
Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.
Rust
#apple-silicon#cpu#gpt-oss#inference#llm#mixture-of-experts#neon#quantization#rust#simd
Port the decode kernels to x86 (AVX-512 / VNNI) about 2 months ago
enhancement help wanted
arizqi/cpubrrr arizqi/cpubrrr Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.
Open project →
12
Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.
Rust
#apple-silicon#cpu#gpt-oss#inference#llm#mixture-of-experts#neon#quantization#rust#simd