A prompt management, versioning, testing, and evaluation inference server and UI toolkit. Provider agnostic and OpenAI API compatible.

1 Open Issue Need Help Last updated: Jun 24, 2025

Open Issues Need Help

View All on GitHub

AI Summary: Implement a fallback mechanism in Llmkit to handle provider or model failures. This involves detecting failures (e.g., network errors, API rate limits, model unavailability), selecting an alternative provider/model from a pre-configured list, and seamlessly switching to the fallback option without interrupting the user experience. The solution should be configurable, allowing users to specify their preferred fallback options and potentially prioritize certain providers based on factors like cost or latency.

Complexity: 4/5
help wanted good first issue

A prompt management, versioning, testing, and evaluation inference server and UI toolkit. Provider agnostic and OpenAI API compatible.

Rust