ovg-project/kvcached

kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference.

gpu-mutiplexing gpu-sharing inference-engine kvcache kvcached llm sglang vllm

1 Open Issue Need Help Last updated: Sep 14, 2025

Open Issues Need Help

good first issue

kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference.

Python

#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcached#llm#sglang#vllm