local-llm-quality-bench good first issues (Python)

enhancement help wanted

michal-konik-human/local-llm-quality-bench

0

Bilingual (Polish + English) quality evals for local open LLMs as agents and assistants: tool calling, multi-step tasks, frozen-web shopping, instruction following, writing. Every result paired with a replicable run spec.

Python

#agents#benchmark#bilingual#evaluation#function-calling#llama-cpp#llm#llm-as-a-judge#llm-evaluation#local-llm#open-source-llm#polish#polish-language#tool-calling

Simulated user for multi-turn agent tasks 15 days ago

enhancement help wanted

michal-konik-human/local-llm-quality-bench

0

Bilingual (Polish + English) quality evals for local open LLMs as agents and assistants: tool calling, multi-step tasks, frozen-web shopping, instruction following, writing. Every result paired with a replicable run spec.

Python

#agents#benchmark#bilingual#evaluation#function-calling#llama-cpp#llm#llm-as-a-judge#llm-evaluation#local-llm#open-source-llm#polish#polish-language#tool-calling

Human labels to calibrate the LLM judge (PL and EN) 15 days ago

help wanted

michal-konik-human/local-llm-quality-bench

0

Bilingual (Polish + English) quality evals for local open LLMs as agents and assistants: tool calling, multi-step tasks, frozen-web shopping, instruction following, writing. Every result paired with a replicable run spec.

Python

#agents#benchmark#bilingual#evaluation#function-calling#llama-cpp#llm#llm-as-a-judge#llm-evaluation#local-llm#open-source-llm#polish#polish-language#tool-calling

Native-speaker review of the Polish cases 15 days ago

help wanted good first issue

michal-konik-human/local-llm-quality-bench

0

Bilingual (Polish + English) quality evals for local open LLMs as agents and assistants: tool calling, multi-step tasks, frozen-web shopping, instruction following, writing. Every result paired with a replicable run spec.

Python

#agents#benchmark#bilingual#evaluation#function-calling#llama-cpp#llm#llm-as-a-judge#llm-evaluation#local-llm#open-source-llm#polish#polish-language#tool-calling

michal-konik-human/local-llm-quality-bench

Open Issues Need Help