ferro-labs

ferro-labs/ai-gateway

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

29 good first / help-wanted issues · Go · last activity Jul 2, 2026

162 stars 28 forks 162 watchers Go Apache License 2.0

ai-gateway ai-infrastructure guardrails litellm llm llm-cost llm-gateway llm-proxy llm-strategy llmops mcp openai-compatible pii-detection prompt-management semantic-cache token-saver

View on GitHub Website

29 Open Issues Need Help Last updated: Jul 2, 2026

Open Issues Need Help

View All on GitHub

Moonshot provider: SupportedModels() lists retired model IDs 4 months ago

bug help wanted priority: medium area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Add live model discovery to providers that only ship a static model list 4 months ago

enhancement help wanted priority: low area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Cloudflare provider: upstream errors not returned in OpenAI error shape 4 months ago

bug good first issue priority: low area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Ollama Cloud provider: OpenAI sampling params silently dropped 4 months ago

bug help wanted priority: medium area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Replicate provider: chat requests flatten history and drop most parameters 4 months ago

bug help wanted priority: medium area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Hugging Face provider: default base URL returns HTTP 410 Gone 4 months ago

bug good first issue priority: high area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Tests: add error-path (non-2xx upstream) coverage for providers 4 months ago

good first issue help wanted priority: low testing area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Perplexity provider: model discovery requests a non-existent path (404) 4 months ago

bug good first issue priority: medium area: providers

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Add live model discovery to self-hosted ollama provider 4 months ago

enhancement good first issue priority: medium go

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

docs: document local development setup, pre-commit hook, and contribution workflow in CONTRIBUTING.md 4 months ago

documentation good first issue priority: low

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

De-stale SupportedModels() — derive from catalog / DiscoverModels 4 months ago

enhancement good first issue priority: medium release-1.1.5

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Capability gaps vs upstream APIs (Azure embeddings/images, image gen, embeddings, discovery) 4 months ago

enhancement good first issue release-1.1.5

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

SupportsModel prefix gaps reject valid models (OpenAI codex/sora, Bedrock cross-region profiles) 5 months ago

bug good first issue priority: medium release-1.1.5

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Capability-miss returns HTTP 500 (should be 400/404); add capability↔interface alignment tests 5 months ago

bug good first issue priority: medium release-1.1.5

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

perf: prompt_regex content conditions recompile the regex on every streaming request 5 months ago

bug help wanted priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Catalog cost lookup does an O(n) scan on every key miss 5 months ago

bug good first issue priority: medium catalog release-1.1.2

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

cost-optimized routing treats null-priced models as $0 (always picked as cheapest) 5 months ago

bug good first issue priority: medium release-1.1.1

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Streaming aborts on SSE lines > 64 KB (missing scanner.Buffer in 9 providers) 5 months ago

bug good first issue priority: high release-1.1.1

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

feat: add streaming support to the Replicate provider 6 months ago

enhancement good first issue help wanted priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

feat: webhook event output plugin — forward request/response events to an HTTP endpoint 8 months ago

enhancement help wanted priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

feat: semantic cache plugin using embedding-based similarity matching 8 months ago

enhancement help wanted priority: low

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

feat: OpenTelemetry distributed tracing for gateway Route and RouteStream 8 months ago

enhancement help wanted priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

docs: add runnable example for content-based routing and A/B testing strategies 8 months ago

documentation good first issue priority: low

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

bug: response-cache cacheKey sorts messages, breaking ordered conversation context 8 months ago

bug good first issue priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

bug: response-cache plugin silently stops caching when max_entries is reached (no eviction) 8 months ago

bug good first issue priority: medium

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

/v1/completions proxy path can become invalid when provider BaseURL already ends with /v1 8 months ago

bug good first issue go

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Fallback strategy returns malformed %which gh && gh auth status --hostname github.com(<nil>) error when no provider supports model 8 months ago

bug good first issue go

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

Add unit tests for ferrogw-cli command behavior 8 months ago

enhancement good first issue go

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver

ferrogw-cli plugins does not list built-in rate-limit plugin 8 months ago

bug good first issue

ferro-labs/ai-gateway

162

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Go

#ai-gateway#ai-infrastructure#guardrails#litellm#llm#llm-cost#llm-gateway#llm-proxy#llm-strategy#llmops#mcp#openai-compatible#pii-detection#prompt-management#semantic-cache#token-saver