Giskard-AI

Giskard-AI/giskard-oss

🐢 Open-Source Evaluation & Testing library for LLM Agents

32 good first / help-wanted issues · Python · last activity Jul 19, 2026

5.7K stars 498 forks 5.7K watchers Python Apache License 2.0
agent-evaluation ai-red-team ai-security ai-testing fairness-ai llm llm-eval llm-evaluation llm-security llmops ml-testing ml-validation mlops rag-evaluation red-team-tools responsible-ai trustworthy-ai
View on GitHub Website
32 Open Issues Need Help Last updated: Jul 19, 2026

Open Issues Need Help

View All on GitHub
feat(checks): add a safe run_sync() convenience method 2 months ago
enhancement good first issue Scope: Checks
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `contains_any` / `contains_all` check 3 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `hallucination` check 3 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `readability` check 4 months ago
enhancement good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `faithfulness` check 4 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
CLI tool for running checks 5 months ago
Help wanted
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `bias` check 5 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `sentiment` check 5 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
`giskard-llm` `__all__` contains phantom entry `ResponseOutputFunctionCall` 5 months ago
bug good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
feat(giskard-checks): extend input_type support to all pydantic-compatible types 5 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Adopt `TypeVar(default=...)` for generic base classes so callers can omit bounded type parameters 6 months ago
enhancement good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `pii_detection` check 6 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Allow running a Scenario multiple times (`multiple_runs`) 6 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Improve ergonomics for composing checks in scenarios 7 months ago
enhancement good first issue d:easy
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Observability integration (OpenTelemetry/OpenLLMmetry) 7 months ago
Help wanted
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add JUnit XML export for SuiteResult 7 months ago
Help wanted
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `json_valid` check 7 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add multiple embedding providers for SemanticSimilarity 7 months ago
Help wanted
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add check composition operators: `AllOf`, `AnyOf`, `Not` 7 months ago
Help wanted
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add retry/majority-voting for LLM-based checks 7 months ago
Help wanted
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `prompt_injection` check 7 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `toxicity` check 7 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `latency` check 7 months ago
good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `AnswerRelevance` check 7 months ago
enhancement good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add JUnit XML export for SuiteResult 7 months ago
enhancement good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Add `ContextRelevance` check 7 months ago
enhancement good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Include scenario errors details when displaying suite report 7 months ago
enhancement good first issue Easy
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Make `Suite.append()` chainable (fluent suite builder) 7 months ago
enhancement good first issue Easy
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Feature: reexport `resolve` at root level 7 months ago
enhancement good first issue Easy
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Testset generation lost when a single generation fail 9 months ago
enhancement good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Allow user to specify N retries about 1 year ago
enhancement good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai
Generate synthetic questions with listing choices about 1 year ago
enhancement good first issue
Giskard-AI/giskard-oss
5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python
#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai