Open Issues Need Help
View All on GitHubAn Open-Source Load Balancer for LLM Inference. Supports all of subscription based (pay-as-you-go/provisioned), free-tier, and local Inference with a simple OpenAI style endpoint
An Open-Source Load Balancer for LLM Inference. Supports all of subscription based (pay-as-you-go/provisioned), free-tier, and local Inference with a simple OpenAI style endpoint
An Open-Source Load Balancer for LLM Inference. Supports all of subscription based (pay-as-you-go/provisioned), free-tier, and local Inference with a simple OpenAI style endpoint