Open Issues Need Help
View All on GitHub Benchmark a native-Linux worker to isolate WSL2 overhead about 2 hours ago
help wanted good first issue hardware-report
Run large language models across several consumer machines over ordinary Ethernet. A model that fits in no single GPU you own can fit across all of them.
Shell
#consumer-hardware#distributed-inference#edge-ai#gpu#llama-cpp#llm#local-llm#pipeline-parallelism
Add the ring hook to more model architectures about 2 hours ago
help wanted good first issue
Run large language models across several consumer machines over ordinary Ethernet. A model that fits in no single GPU you own can fit across all of them.
Shell
#consumer-hardware#distributed-inference#edge-ai#gpu#llama-cpp#llm#local-llm#pipeline-parallelism
Test the ring with three or more machines (--world 3) about 2 hours ago
help wanted good first issue
Run large language models across several consumer machines over ordinary Ethernet. A model that fits in no single GPU you own can fit across all of them.
Shell
#consumer-hardware#distributed-inference#edge-ai#gpu#llama-cpp#llm#local-llm#pipeline-parallelism
Lockstep between ranks prevents serving an OpenAI-compatible HTTP API about 2 hours ago
help wanted open-problem
Run large language models across several consumer machines over ordinary Ethernet. A model that fits in no single GPU you own can fit across all of them.
Shell
#consumer-hardware#distributed-inference#edge-ai#gpu#llama-cpp#llm#local-llm#pipeline-parallelism