Open Issues Need Help
View All on GitHub Add a second live judge backend (OpenAI or NVIDIA NIM) for cross-judge kappa about 2 months ago
help wanted good first issue
jbisaccia-9/kappa-gate jbisaccia-9/kappa-gate LLM-as-judge evaluation harness where the judge itself must pass a Cohen's-kappa calibration gate
Open project →
0
LLM-as-judge evaluation harness where the judge itself must pass a Cohen's-kappa calibration gate
Python
#ai-governance#anthropic#cohens-kappa#evals#inter-rater-reliability#llm-as-a-judge#llm-evaluation#python
Add 10 harder boundary cases to the golden set about 2 months ago
help wanted good first issue
jbisaccia-9/kappa-gate jbisaccia-9/kappa-gate LLM-as-judge evaluation harness where the judge itself must pass a Cohen's-kappa calibration gate
Open project →
0
LLM-as-judge evaluation harness where the judge itself must pass a Cohen's-kappa calibration gate
Python
#ai-governance#anthropic#cohens-kappa#evals#inter-rater-reliability#llm-as-a-judge#llm-evaluation#python
Rewrite Completeness (C) rubric anchors — current weighted kappa is 0.20 about 2 months ago
help wanted good first issue
jbisaccia-9/kappa-gate jbisaccia-9/kappa-gate LLM-as-judge evaluation harness where the judge itself must pass a Cohen's-kappa calibration gate
Open project →
0
LLM-as-judge evaluation harness where the judge itself must pass a Cohen's-kappa calibration gate
Python
#ai-governance#anthropic#cohens-kappa#evals#inter-rater-reliability#llm-as-a-judge#llm-evaluation#python