evaleval

evaleval/every_eval_ever

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

35 good first / help-wanted issues · Python · last activity Jul 5, 2026

agent-evaluation ai-evaluation evaluations infra llm-evaluation
35 Open Issues Need Help Last updated: Jul 5, 2026

Open Issues Need Help

View All on GitHub
enhancement help wanted converters
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
documentation good first issue
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
help wanted priority:high datastore adapters converters big-ticket
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
enhancement help wanted datastore
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
enhancement good first issue validator
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
good first issue help wanted schema datastore
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
enhancement help wanted priority:high
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
enhancement help wanted validator
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
help wanted schema
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation
enhancement good first issue priority:high schema
evaleval/every_eval_ever
90

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Python
#agent-evaluation#ai-evaluation#evaluations#infra#llm-evaluation