kvcached good first issues (Python)

help wanted feature requested

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

kvcached support matrix about 2 months ago

help wanted

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

start mutiple models 2 months ago

good first issue

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

Prefix caching makes gemma-4 generations diverge from vanilla vLLM (output stays coherent, not garbled) 3 months ago

bug help wanted

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

Roadmap/Feature requested/TODOs---Start Here for New Contributors 3 months ago

help wanted

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

CI runs no tests — add a GitHub Actions job for the CPU-only pytest suite 3 months ago

enhancement help wanted

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

Error on gpt-oss with vLLM 11 months ago

good first issue

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

Failed to patch kv_cache_coordinator about 1 year ago

good first issue

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

What is the difference between kvcached and Paged Attention ? about 1 year ago

good first issue

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

Question About kvcached Ability to Dynamically Recognize and Utilize Kubernetes Elastic Scaled GPU Memory Resources about 1 year ago

good first issue

ovg-project/kvcached

1.1K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Python

#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama#online-offline-coserve#serverless#sglang#vllm

ovg-project/kvcached

Open Issues Need Help