Open Issues Need Help
View All on GitHub Add Joule result about 1 month ago
good first issue wontfix
lightbench: a lightweight tool for benchmarking LLMs 🛠️
Python
#benchmark#llm#llm-inference
Handle system prompt message in ClaudeLoader 4 months ago
AI Summary: The task is to modify the `ClaudeLoader` in the `lightbench` project to correctly handle system messages when interacting with the Anthropic API. This involves parsing the input prompt to separate the system message from the user prompt and then passing them as separate arguments to the Anthropic API's `messages.create` function.
Complexity:
3/5
help wanted good first issue
lightbench: a lightweight tool for benchmarking LLMs 🛠️
Python
#benchmark#llm#llm-inference