llm-d/llm-d-router

Add span for picker about 8 hours ago

help wanted observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Add span for filter about 8 hours ago

help wanted observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Migrate e2e tests from GAIE about 8 hours ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Improve datalayer observability 1 day ago

help wanted needs-triage observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for program-aware-fairness plugin 2 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for mm-embeddings-cache-producer plugin 2 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for approx-prefix-cache-producer plugin 2 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for precise-prefix-cache-producer plugin 2 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Add pluginName to the default DumpState 2 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Compatibility] Validate and document TurboQuant (KV cache quantization) serving support 3 days ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for predicted-latency-producer plugin 5 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for file-discovery plugin 5 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for no-hit-lru-scorer plugin 5 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Tracking: implement DumpState for all in-tree plugins 5 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Observability] An EPP metric that reports inflight tokens and requests 12 days ago

help wanted needs-triage observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Observaibility] epp/metrics: unbounded label cardinality from user-controlled inputs (metrics cardinality DoS) 14 days ago

help wanted needs-triage kind/bug observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Observability] P/D sidecar observability: add Prometheus /metrics, wire trace propagation, document enablement 17 days ago

help wanted needs-triage observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Logs are not correlated with traces: trace_id/span_id never injected into the logger 17 days ago

help wanted needs-triage kind/bug observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

A session-affinity-filter plugin 19 days ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Observability] Add fairness_id and objective labels to EPP TTFT/TPOT metrics for per-tenant SLO observability 21 days ago

help wanted observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[E2E Tests] Failing on a few PRs 22 days ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

token-producer: explicit backend selector field 25 days ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

more granular request-based load-aware scorers 28 days ago

good first issue help wanted needs-triage

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[E2E Tests] Use standalone mode for e2e tests about 1 month ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Review and types of fix CRD fields about 1 month ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[migration] GAIE changes and migration about 1 month ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

EPP should signal invalid plugin config fields via error/warning about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Add helm charts builds to ci about 1 month ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Migration] rename all headers managed by epp to have the prefix x-llm-d about 2 months ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Migration] Remove the InferencePool CRD definitions. about 2 months ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Migration]: Migrate the InferenceObjective and InferenceModelRewrite APIs to llm-d.ai group name about 2 months ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Migration]: rename the docker images about 2 months ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Open Issues Need Help