redhat-et/kuadrant-llm-integration

AI Summary: This task requires researching and comparing Kubecost and Koku, documenting their features, pros, and cons in a matrix format. Then, based on the findings, a demo integration with Kubecost needs to be created and added to the existing Kuadrant project, potentially extending the existing chargeback metrics demo to include Kubecost integration.

AI Summary: Integrate Keycloak as an Identity and Access Management (IAM) provider for the Kuadrant-based vLLM inference gateway, replacing the existing static secret authentication with Keycloak's OIDC capabilities. This involves creating a Keycloak realm, configuring OIDC settings, and updating the Kuadrant configuration to use Keycloak for authentication. A demo showcasing this integration with Google as an OIDC provider is also required.

AI Summary: The task is to create a new demo showcasing Kuadrant's `TokenRateLimitPolicy` for rate limiting. This involves creating Kubernetes manifests, integrating them with the existing Kuadrant setup, and writing documentation explaining the implementation and usage. The demo should demonstrate tiered rate limiting based on user groups (free and gold tiers in this example).

AI Summary: Research and define the necessary Prometheus metrics for billing in a basic rate-limiting system, specifically for the Limitador and TokenRateLimitPolicy components. This involves analyzing existing metrics (provided in the issue description), identifying gaps, and proposing new metrics for accurate cost calculation and chargeback reporting. The goal is to create a proposal for upstream discussion and potential implementation in the Limitador Wasm.

AI Summary: Investigate if Project Koku's features align with the billing patterns demonstrated in the Kuadrant project, specifically focusing on usage tracking and cost allocation. If a fit is found, create a new demo showcasing Koku's integration with Kuadrant for billing purposes.

AI Summary: Validate the Kuadrant vLLM integration project on a Red Hat OpenShift AI (RHOAI) cluster. This involves deploying the project's components (Kuadrant, Gateway API, Istio, Prometheus) on RHOAI, running the provided demos (basic authorization, rate limiting, chargeback metrics), and verifying their functionality as documented. The validation should confirm that the project works seamlessly within the RHOAI environment.