ferro-labs

ferro-labs/ai-gateway

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

162 stars 28 forks 162 watchers Go Apache License 2.0

ai-gateway ai-infrastructure guardrails litellm llm llm-cost llm-gateway llm-proxy llm-strategy llmops mcp openai-compatible pii-detection prompt-management semantic-cache token-saver

View on GitHub Website

29 Open Issues Need Help Last updated: Jul 2, 2026

Open Issues Need Help

View All on GitHub

Moonshot provider: SupportedModels() lists retired model IDs 11 days ago

bug help wanted priority: medium area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Add live model discovery to providers that only ship a static model list 11 days ago

enhancement help wanted priority: low area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Cloudflare provider: upstream errors not returned in OpenAI error shape 11 days ago

bug good first issue priority: low area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Ollama Cloud provider: OpenAI sampling params silently dropped 11 days ago

bug help wanted priority: medium area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Replicate provider: chat requests flatten history and drop most parameters 11 days ago

bug help wanted priority: medium area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Hugging Face provider: default base URL returns HTTP 410 Gone 11 days ago

bug good first issue priority: high area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Tests: add error-path (non-2xx upstream) coverage for providers 12 days ago

good first issue help wanted priority: low testing area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Perplexity provider: model discovery requests a non-existent path (404) 12 days ago

bug good first issue priority: medium area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Add live model discovery to self-hosted ollama provider 19 days ago

enhancement good first issue priority: medium go

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

docs: document local development setup, pre-commit hook, and contribution workflow in CONTRIBUTING.md 19 days ago

documentation good first issue priority: low

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

De-stale SupportedModels() — derive from catalog / DiscoverModels 26 days ago

enhancement good first issue priority: medium release-1.1.5

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Capability gaps vs upstream APIs (Azure embeddings/images, image gen, embeddings, discovery) 26 days ago

enhancement good first issue release-1.1.5

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

SupportsModel prefix gaps reject valid models (OpenAI codex/sora, Bedrock cross-region profiles) about 1 month ago

bug good first issue priority: medium release-1.1.5

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Capability-miss returns HTTP 500 (should be 400/404); add capability↔interface alignment tests about 1 month ago

bug good first issue priority: medium release-1.1.5

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

perf: prompt_regex content conditions recompile the regex on every streaming request about 1 month ago

bug help wanted priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Catalog cost lookup does an O(n) scan on every key miss about 1 month ago

bug good first issue priority: medium catalog release-1.1.2

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

cost-optimized routing treats null-priced models as $0 (always picked as cheapest) about 2 months ago

bug good first issue priority: medium release-1.1.1

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Streaming aborts on SSE lines > 64 KB (missing scanner.Buffer in 9 providers) about 2 months ago

bug good first issue priority: high release-1.1.1

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

feat: add streaming support to the Replicate provider 3 months ago

enhancement good first issue help wanted priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

feat: webhook event output plugin — forward request/response events to an HTTP endpoint 4 months ago

enhancement help wanted priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

feat: semantic cache plugin using embedding-based similarity matching 4 months ago

enhancement help wanted priority: low

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

feat: OpenTelemetry distributed tracing for gateway Route and RouteStream 4 months ago

enhancement help wanted priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

docs: add runnable example for content-based routing and A/B testing strategies 4 months ago

documentation good first issue priority: low

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

bug: response-cache cacheKey sorts messages, breaking ordered conversation context 4 months ago

bug good first issue priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

bug: response-cache plugin silently stops caching when max_entries is reached (no eviction) 4 months ago

bug good first issue priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

/v1/completions proxy path can become invalid when provider BaseURL already ends with /v1 4 months ago

bug good first issue go

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Fallback strategy returns malformed %which gh && gh auth status --hostname github.com(<nil>) error when no provider supports model 4 months ago

bug good first issue go

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Add unit tests for ferrogw-cli command behavior 5 months ago

enhancement good first issue go

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

ferrogw-cli plugins does not list built-in rate-limit plugin 5 months ago

bug good first issue

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver