Open Issues Need Help
View All on GitHub Build a Pingala-8 file for Llama-3.1-8B-Instruct and report perplexity at equal size about 10 hours ago
help wanted models
Genrius-AI/pingala8-llm Genrius-AI/pingala8-llm 2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Open project →
3
2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Python
#2-bit#ai#cuda#deep-learning#generative-ai#gguf#inference#large-language-models#llama#llama-cpp#llama3#llm#llm-inference#local-llm#machine-learning#model-compression#post-training-quantization#quantization#qwen3#trellis-coded-quantization
Port the decoder beyond CUDA: ROCm/HIP, Metal, Vulkan about 10 hours ago
help wanted kernel
Genrius-AI/pingala8-llm Genrius-AI/pingala8-llm 2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Open project →
3
2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Python
#2-bit#ai#cuda#deep-learning#generative-ai#gguf#inference#large-language-models#llama#llama-cpp#llama3#llm#llm-inference#local-llm#machine-learning#model-compression#post-training-quantization#quantization#qwen3#trellis-coded-quantization
Rebase the patches onto current llama.cpp and report conflicts about 10 hours ago
help wanted kernel
Genrius-AI/pingala8-llm Genrius-AI/pingala8-llm 2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Open project →
3
2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Python
#2-bit#ai#cuda#deep-learning#generative-ai#gguf#inference#large-language-models#llama#llama-cpp#llama3#llm#llm-inference#local-llm#machine-learning#model-compression#post-training-quantization#quantization#qwen3#trellis-coded-quantization
Run the NEON CPU path on real ARM hardware about 10 hours ago
help wanted benchmark kernel
Genrius-AI/pingala8-llm Genrius-AI/pingala8-llm 2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Open project →
3
2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Python
#2-bit#ai#cuda#deep-learning#generative-ai#gguf#inference#large-language-models#llama#llama-cpp#llama3#llm#llm-inference#local-llm#machine-learning#model-compression#post-training-quantization#quantization#qwen3#trellis-coded-quantization
CPU patch: load the Qwen3-4B B4 file (vr.layout 3) about 10 hours ago
help wanted kernel
Genrius-AI/pingala8-llm Genrius-AI/pingala8-llm 2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Open project →
3
2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Python
#2-bit#ai#cuda#deep-learning#generative-ai#gguf#inference#large-language-models#llama#llama-cpp#llama3#llm#llm-inference#local-llm#machine-learning#model-compression#post-training-quantization#quantization#qwen3#trellis-coded-quantization
Benchmark Pingala-8 on your NVIDIA GPU (RTX 30/40/50, T4 and others) about 10 hours ago
good first issue help wanted benchmark
Genrius-AI/pingala8-llm Genrius-AI/pingala8-llm 2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Open project →
3
2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Python
#2-bit#ai#cuda#deep-learning#generative-ai#gguf#inference#large-language-models#llama#llama-cpp#llama3#llm#llm-inference#local-llm#machine-learning#model-compression#post-training-quantization#quantization#qwen3#trellis-coded-quantization
Docs: troubleshooting page for the one-click build about 10 hours ago
good first issue docs
Genrius-AI/pingala8-llm Genrius-AI/pingala8-llm 2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Open project →
3
2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Python
#2-bit#ai#cuda#deep-learning#generative-ai#gguf#inference#large-language-models#llama#llama-cpp#llama3#llm#llm-inference#local-llm#machine-learning#model-compression#post-training-quantization#quantization#qwen3#trellis-coded-quantization
One-click launchers reuse a partly downloaded model shard about 10 hours ago
bug good first issue
Genrius-AI/pingala8-llm Genrius-AI/pingala8-llm 2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Open project →
3
2-bit LLMs for llama.cpp: 85% smaller, beats IQ2_XXS at the same size, no retraining. One-click Windows build.
Python
#2-bit#ai#cuda#deep-learning#generative-ai#gguf#inference#large-language-models#llama#llama-cpp#llama3#llm#llm-inference#local-llm#machine-learning#model-compression#post-training-quantization#quantization#qwen3#trellis-coded-quantization