Open Issues Need Help
View All on GitHub Validation wanted: annotate the frozen 270-response human sample about 1 month ago
help wanted human-validation
junru-zhu/failure-transparent-agents junru-zhu/failure-transparent-agents Benchmarking whether LLM agents admit tool failure instead of hallucinating success — 100 tasks, 1,800 responses, deterministic simulator, reproducible harness.
Open project →
1
Benchmarking whether LLM agents admit tool failure instead of hallucinating success — 100 tasks, 1,800 responses, deterministic simulator, reproducible harness.
Python
#ai-agents#ai-evaluation#benchmark#guardrails#hallucination#llm#reproducibility#tool-use
Replication wanted: evaluate an additional open-weight model about 1 month ago
help wanted model-result replication
junru-zhu/failure-transparent-agents junru-zhu/failure-transparent-agents Benchmarking whether LLM agents admit tool failure instead of hallucinating success — 100 tasks, 1,800 responses, deterministic simulator, reproducible harness.
Open project →
1
Benchmarking whether LLM agents admit tool failure instead of hallucinating success — 100 tasks, 1,800 responses, deterministic simulator, reproducible harness.
Python
#ai-agents#ai-evaluation#benchmark#guardrails#hallucination#llm#reproducibility#tool-use