Eval-core/evalcore

Which model providers & gateways should we document first? about 9 hours ago

help wanted question

5

Know when your AI gets worse — before your users do. Records how your LLM apps and agents behave and checks every change: free, offline, deterministic CI. Trials, model comparison, OTel agent trajectories.

Rust

#ci#ci-cd#cicd#evaluation#evaluation-framework#llm#llm-eval#llm-evaluation#llm-evaluation-framework#llmops#pentesting#prompt-engineering#prompt-testing#prompts#rag#red-teaming#testing#vulnerability-scanners

Docs cookbook: RAG faithfulness & context-recall recipe about 9 hours ago

documentation good first issue

Eval-core/evalcore

5

Know when your AI gets worse — before your users do. Records how your LLM apps and agents behave and checks every change: free, offline, deterministic CI. Trials, model comparison, OTel agent trajectories.

Rust

#ci#ci-cd#cicd#evaluation#evaluation-framework#llm#llm-eval#llm-evaluation#llm-evaluation-framework#llmops#pentesting#prompt-engineering#prompt-testing#prompts#rag#red-teaming#testing#vulnerability-scanners

Add a `length` / word-count scorer with min/max bounds about 9 hours ago

enhancement good first issue

Eval-core/evalcore

5

Know when your AI gets worse — before your users do. Records how your LLM apps and agents behave and checks every change: free, offline, deterministic CI. Trials, model comparison, OTel agent trajectories.

Rust

#ci#ci-cd#cicd#evaluation#evaluation-framework#llm#llm-eval#llm-evaluation#llm-evaluation-framework#llmops#pentesting#prompt-engineering#prompt-testing#prompts#rag#red-teaming#testing#vulnerability-scanners

GitHub Action: annotate the PR with an eval diff summary about 9 hours ago

enhancement help wanted

Eval-core/evalcore

5

Know when your AI gets worse — before your users do. Records how your LLM apps and agents behave and checks every change: free, offline, deterministic CI. Trials, model comparison, OTel agent trajectories.

Rust

#ci#ci-cd#cicd#evaluation#evaluation-framework#llm#llm-eval#llm-evaluation#llm-evaluation-framework#llmops#pentesting#prompt-engineering#prompt-testing#prompts#rag#red-teaming#testing#vulnerability-scanners

`evalcore init`: scaffold a new eval suite in one command about 9 hours ago

enhancement good first issue

Eval-core/evalcore

5

Know when your AI gets worse — before your users do. Records how your LLM apps and agents behave and checks every change: free, offline, deterministic CI. Trials, model comparison, OTel agent trajectories.

Rust

#ci#ci-cd#cicd#evaluation#evaluation-framework#llm#llm-eval#llm-evaluation#llm-evaluation-framework#llmops#pentesting#prompt-engineering#prompt-testing#prompts#rag#red-teaming#testing#vulnerability-scanners

enhancement good first issue help wanted

Eval-core/evalcore

5

Know when your AI gets worse — before your users do. Records how your LLM apps and agents behave and checks every change: free, offline, deterministic CI. Trials, model comparison, OTel agent trajectories.

Rust

#ci#ci-cd#cicd#evaluation#evaluation-framework#llm#llm-eval#llm-evaluation#llm-evaluation-framework#llmops#pentesting#prompt-engineering#prompt-testing#prompts#rag#red-teaming#testing#vulnerability-scanners

Red-teaming: adversarial test generation & attack replay about 9 hours ago

enhancement help wanted roadmap red-team

Eval-core/evalcore

5

Know when your AI gets worse — before your users do. Records how your LLM apps and agents behave and checks every change: free, offline, deterministic CI. Trials, model comparison, OTel agent trajectories.

Rust

#ci#ci-cd#cicd#evaluation#evaluation-framework#llm#llm-eval#llm-evaluation#llm-evaluation-framework#llmops#pentesting#prompt-engineering#prompt-testing#prompts#rag#red-teaming#testing#vulnerability-scanners

Open Issues Need Help