llm-d/llm-d-kv-cache

Speculative Indexing for the KV-Cache Aware Routing 2 days ago

help wanted lifecycle/stale

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Add support for externally invalidate/reset the KV cache 2 days ago

good first issue help wanted lifecycle/stale

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

FS Backend: Enable unit tests in CI 2 months ago

good first issue kv-connector lifecycle/stale

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Align repository with llm-d repo template (7 items) 2 months ago

help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

PVC Evictor optimizations 3 months ago

enhancement good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Abstract KVEvents API 3 months ago

good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Profile the different `kvblock.Index` implementations 4 months ago

good first issue kvblock-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Test different attention mechanisms 4 months ago

good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

[Performance] Code Profiling 6 months ago

enhancement good first issue help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

E2E Tests 6 months ago

good first issue help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Document and test the `extra` field in block hashes 6 months ago

good first issue kvevents-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

KVEvents Testing 6 months ago

good first issue kvevents-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Benchmark UDS vs embedded tokenizers 6 months ago

good first issue help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Enable pre-commit checks in CI 6 months ago

good first issue

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Add UDS tokenizer tests to CI 6 months ago

good first issue python

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Update examples to support UDS-only builds (no embedded tokenizers) 6 months ago

good first issue help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

E2E tests for UDS tokenizer 6 months ago

AI Summary: The UDS tokenizer is being integrated into the main codebase. To ensure its proper functioning with the live service, end-to-end tests are required, as current unit tests only mock its behavior and do not interact with the actual external service.

Complexity: 4/5

help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

[CI] examples test action takes 20 mins 7 months ago

good first issue

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Open Issues Need Help