Open Issues Need Help
View All on GitHub help wanted
bhargavchintam/llm-inference bhargavchintam/llm-inference LLM inference stack in NumPy - attention, KV cache, paged attention, continuous batching, speculative decoding, quantization, sampling, and a toy engine.
Open project →
0
LLM inference stack in NumPy - attention, KV cache, paged attention, continuous batching, speculative decoding, quantization, sampling, and a toy engine.
Python
#continuous-batching#flash-attention#kv-cache#llm#llm-inference#ml-systems#numpy#paged-attention#quantization#speculative-decoding#transformers
Add a chunked prefill check 7 days ago
help wanted
bhargavchintam/llm-inference bhargavchintam/llm-inference LLM inference stack in NumPy - attention, KV cache, paged attention, continuous batching, speculative decoding, quantization, sampling, and a toy engine.
Open project →
0
LLM inference stack in NumPy - attention, KV cache, paged attention, continuous batching, speculative decoding, quantization, sampling, and a toy engine.
Python
#continuous-batching#flash-attention#kv-cache#llm#llm-inference#ml-systems#numpy#paged-attention#quantization#speculative-decoding#transformers
Add a sliding-window attention check 7 days ago
good first issue
bhargavchintam/llm-inference bhargavchintam/llm-inference LLM inference stack in NumPy - attention, KV cache, paged attention, continuous batching, speculative decoding, quantization, sampling, and a toy engine.
Open project →
0
LLM inference stack in NumPy - attention, KV cache, paged attention, continuous batching, speculative decoding, quantization, sampling, and a toy engine.
Python
#continuous-batching#flash-attention#kv-cache#llm#llm-inference#ml-systems#numpy#paged-attention#quantization#speculative-decoding#transformers