llm-d-kv-cache good first issues (Go)

Speculative Indexing for the KV-Cache Aware Routing about 2 months ago

help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Add support for externally invalidate/reset the KV cache about 2 months ago

good first issue help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

FS Backend: Enable unit tests in CI 4 months ago

good first issue kv-connector lifecycle/stale

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Align repository with llm-d repo template (7 items) 5 months ago

help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

PVC Evictor optimizations 5 months ago

enhancement good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Abstract KVEvents API 5 months ago

good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Profile the different `kvblock.Index` implementations 6 months ago

good first issue kvblock-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Test different attention mechanisms 6 months ago

good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

[Performance] Code Profiling 8 months ago

enhancement good first issue help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

E2E Tests 8 months ago

good first issue help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Document and test the `extra` field in block hashes 8 months ago

good first issue kvevents-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

KVEvents Testing 8 months ago

good first issue kvevents-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Benchmark UDS vs embedded tokenizers 8 months ago

good first issue help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Enable pre-commit checks in CI 8 months ago

good first issue

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Add UDS tokenizer tests to CI 8 months ago

good first issue python

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Update examples to support UDS-only builds (no embedded tokenizers) 8 months ago

good first issue help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

E2E tests for UDS tokenizer 8 months ago

AI Summary: The UDS tokenizer is being integrated into the main codebase. To ensure its proper functioning with the live service, end-to-end tests are required, as current unit tests only mock its behavior and do not interact with the actual external service.

Complexity: 4/5

help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

[CI] examples test action takes 20 mins 9 months ago

good first issue

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

llm-d/llm-d-kv-cache

Open Issues Need Help