Open Issues Need Help
View All on GitHub documentation help wanted good first issue
homayoun-safarpour/judge-reliability-kit homayoun-safarpour/judge-reliability-kit Diagnose whether a low-kappa LLM judge panel fails from item ambiguity or rubric underspecification.
Open project →
0
Diagnose whether a low-kappa LLM judge panel fails from item ambiguity or rubric underspecification.
Python
#ai-evaluation#bias-detection#cohen-kappa#evaluation-metrics#fleiss-kappa#inter-rater-reliability#llm-as-a-judge#llm-evaluation#python#reliability
good first issue
homayoun-safarpour/judge-reliability-kit homayoun-safarpour/judge-reliability-kit Diagnose whether a low-kappa LLM judge panel fails from item ambiguity or rubric underspecification.
Open project →
0
Diagnose whether a low-kappa LLM judge panel fails from item ambiguity or rubric underspecification.
Python
#ai-evaluation#bias-detection#cohen-kappa#evaluation-metrics#fleiss-kappa#inter-rater-reliability#llm-as-a-judge#llm-evaluation#python#reliability
good first issue
homayoun-safarpour/judge-reliability-kit homayoun-safarpour/judge-reliability-kit Diagnose whether a low-kappa LLM judge panel fails from item ambiguity or rubric underspecification.
Open project →
0
Diagnose whether a low-kappa LLM judge panel fails from item ambiguity or rubric underspecification.
Python
#ai-evaluation#bias-detection#cohen-kappa#evaluation-metrics#fleiss-kappa#inter-rater-reliability#llm-as-a-judge#llm-evaluation#python#reliability