lightbench: a lightweight tool for benchmarking LLMs 🛠️

benchmark llm llm-inference
2 Open Issues Need Help Last updated: Jun 19, 2025

Open Issues Need Help

View All on GitHub

AI Summary: The task is to modify the `ClaudeLoader` in the `lightbench` project to correctly handle system messages when interacting with the Anthropic API. This involves parsing the input prompt to separate the system message from the user prompt and then passing them as separate arguments to the Anthropic API's `messages.create` function.

Complexity: 3/5
help wanted good first issue

lightbench: a lightweight tool for benchmarking LLMs 🛠️

Python
#benchmark#llm#llm-inference
Add Joule result 2 months ago

AI Summary: Add a 'Joule' result calculation to the lightbench project. This involves calculating energy consumption by multiplying the existing 'watt' and 'time' measurements. This requires modifying the existing code to include this calculation and potentially updating the output to display the Joule value.

Complexity: 2/5
good first issue

lightbench: a lightweight tool for benchmarking LLMs 🛠️

Python
#benchmark#llm#llm-inference