Open Issues Need Help
View All on GitHubAI Summary: Implement a fallback mechanism in Llmkit to handle provider or model failures. This involves detecting failures (e.g., network errors, API rate limits, model unavailability), selecting an alternative provider/model from a pre-configured list, and seamlessly switching to the fallback option without interrupting the user experience. The solution should be configurable, allowing users to specify their preferred fallback options and potentially prioritize certain providers based on factors like cost or latency.
A prompt management, versioning, testing, and evaluation inference server and UI toolkit. Provider agnostic and OpenAI API compatible.