ankit1057/sipllm

Llama 3.x: apply the "llama3" RoPE scaling about 2 hours ago

enhancement good first issue architecture

0

sipllm — a dependency-free streaming GGUF LLM inference engine in C++17. Runs quantized Llama models on-device with flat RAM (it sips weights off disk), Ollama-style CLI, validated layer-by-layer against llama.cpp.

C++

#arm64#cpp#cpp17#edge-ai#gguf#inference#llama#llama-cpp#llm#machine-learning#neon#on-device#quantization#tinyllama#transformer

Enabling refactor: dispatch on general.architecture (not Llama-only) about 2 hours ago

enhancement good first issue architecture

ankit1057/sipllm

0

sipllm — a dependency-free streaming GGUF LLM inference engine in C++17. Runs quantized Llama models on-device with flat RAM (it sips weights off disk), Ollama-style CLI, validated layer-by-layer against llama.cpp.

C++

#arm64#cpp#cpp17#edge-ai#gguf#inference#llama#llama-cpp#llm#machine-learning#neon#on-device#quantization#tinyllama#transformer

Docs: diagram + walkthrough of the streaming layer loader about 3 hours ago

good first issue

ankit1057/sipllm

0

sipllm — a dependency-free streaming GGUF LLM inference engine in C++17. Runs quantized Llama models on-device with flat RAM (it sips weights off disk), Ollama-style CLI, validated layer-by-layer against llama.cpp.

C++

#arm64#cpp#cpp17#edge-ai#gguf#inference#llama#llama-cpp#llm#machine-learning#neon#on-device#quantization#tinyllama#transformer

CI: publish prebuilt x86_64 and macOS release binaries about 3 hours ago

help wanted

ankit1057/sipllm

0

sipllm — a dependency-free streaming GGUF LLM inference engine in C++17. Runs quantized Llama models on-device with flat RAM (it sips weights off disk), Ollama-style CLI, validated layer-by-layer against llama.cpp.

C++

#arm64#cpp#cpp17#edge-ai#gguf#inference#llama#llama-cpp#llm#machine-learning#neon#on-device#quantization#tinyllama#transformer

Add an AVX2 SIMD path alongside the ARM64 NEON kernels about 3 hours ago

help wanted

ankit1057/sipllm

0

sipllm — a dependency-free streaming GGUF LLM inference engine in C++17. Runs quantized Llama models on-device with flat RAM (it sips weights off disk), Ollama-style CLI, validated layer-by-layer against llama.cpp.

C++

#arm64#cpp#cpp17#edge-ai#gguf#inference#llama#llama-cpp#llm#machine-learning#neon#on-device#quantization#tinyllama#transformer

Add a dequant path for another GGUF format (e.g. Q4_K_S / IQ4_NL) about 3 hours ago

good first issue

ankit1057/sipllm

0

sipllm — a dependency-free streaming GGUF LLM inference engine in C++17. Runs quantized Llama models on-device with flat RAM (it sips weights off disk), Ollama-style CLI, validated layer-by-layer against llama.cpp.

C++

#arm64#cpp#cpp17#edge-ai#gguf#inference#llama#llama-cpp#llm#machine-learning#neon#on-device#quantization#tinyllama#transformer

Add more models to the sipllm registry about 3 hours ago

good first issue

ankit1057/sipllm

0

sipllm — a dependency-free streaming GGUF LLM inference engine in C++17. Runs quantized Llama models on-device with flat RAM (it sips weights off disk), Ollama-style CLI, validated layer-by-layer against llama.cpp.

C++

#arm64#cpp#cpp17#edge-ai#gguf#inference#llama#llama-cpp#llm#machine-learning#neon#on-device#quantization#tinyllama#transformer

Add a top-p / top-k / repetition-penalty sampler about 3 hours ago

good first issue

ankit1057/sipllm

0

sipllm — a dependency-free streaming GGUF LLM inference engine in C++17. Runs quantized Llama models on-device with flat RAM (it sips weights off disk), Ollama-style CLI, validated layer-by-layer against llama.cpp.

C++

#arm64#cpp#cpp17#edge-ai#gguf#inference#llama#llama-cpp#llm#machine-learning#neon#on-device#quantization#tinyllama#transformer

Open Issues Need Help