kvcache-ai/Mooncake

[TE] Refactor: unify libfabric-based transports (EFA / CXI) shared infrastructure 6 days ago

good first issue

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[TE] Capture GPU device at registration time in MnnvlTransport / NVLinkTransport instead of inferring from request pointers 7 days ago

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Feature Request]: Mooncake Python Package Version Verification 19 days ago

good first issue feature request stale

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Feature Request]: Tiered Caching for Mooncake Store 20 days ago

good first issue stale

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Feature Request]: Use Builder Pattern for Mooncake Store Client Construction 20 days ago

good first issue feature request stale

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[RFC]: dynamic replica management 21 days ago

enhancement good first issue stale

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[RFC]: Hot Standby Mode for Master Service Metadata High Availability 29 days ago

good first issue stale

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[RFC]: Mooncake-Conductor: Design and Implementation of a Global Scheduler Module for KV-Cache-Centric Disaggregated Architecture 3 months ago

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[RFC]: Support hugepage memory allocation via mmap to bypass RDMA memory registration limits 5 months ago

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Help Wanted] High-performance send/recv implementation for Mooncake PG 5 months ago

help wanted

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Feature Request]: Add Code Coverage Support in CI 7 months ago

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Call for Contribution]: Performance Benchmarking for Mooncake Master 7 months ago

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[RFC]: Independently build Store and TransferEngine 8 months ago

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Installation]: Support Independent Compilation of Master to Reduce Dependencies 8 months ago

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Feature Request]: Add Comprehensive Test Suite for Python Binding of mooncake-store 9 months ago

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Bug]: fix disk-backed replicas in size calculation & migrate to tl::expected 12 months ago

AI Summary: The task involves fixing a bug in the Mooncake Store's size calculation function. The current code only considers memory-based replicas, neglecting disk-backed replicas, leading to test failures. The solution requires creating an interface to abstract the size calculation and replacing exception handling with `tl::expected` for better error management.

Complexity: 4/5

bug good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

[Question] 如何配置 mooncake_master 监听特定网卡及启用 IPv6 通信？ about 1 year ago

AI Summary: The task is to determine how to configure the `mooncake_master` to listen on a specific network interface and enable IPv6 communication. This involves modifying the configuration to select a particular IP address (IPv4 or IPv6) for the master server and ensuring client connections use the specified IPv6 address.

Complexity: 4/5

good first issue

kvcache-ai/Mooncake

5.7K

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

#disaggregation#inference#kvcache#llm#rdma#reinforcement-learning#sglang#tokenspeed#trt-llm#vllm

Open Issues Need Help