An Open-Source Load Balancer for LLM Inference. Supports all of subscription based (pay-as-you-go/provisioned), free-tier, and local Inference with a simple OpenAI style endpoint

1 stars 0 forks 1 watchers Python Apache License 2.0
3 Open Issues Need Help Last updated: Jul 26, 2026

Open Issues Need Help

View All on GitHub
bug enhancement help wanted good first issue

An Open-Source Load Balancer for LLM Inference. Supports all of subscription based (pay-as-you-go/provisioned), free-tier, and local Inference with a simple OpenAI style endpoint

Python
enhancement help wanted good first issue

An Open-Source Load Balancer for LLM Inference. Supports all of subscription based (pay-as-you-go/provisioned), free-tier, and local Inference with a simple OpenAI style endpoint

Python
enhancement help wanted good first issue

An Open-Source Load Balancer for LLM Inference. Supports all of subscription based (pay-as-you-go/provisioned), free-tier, and local Inference with a simple OpenAI style endpoint

Python