hermes-labs-ai

hermes-labs-ai/hermes-jailbench

Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench

1 good first / help-wanted issue · Python · last activity Sep 29, 2026

9 stars 2 forks 9 watchers Python Apache License 2.0
agent-skills ai-red-team ai-reliability ai-safety benchmark claude-code-plugin deterministic-testing gemini-cli-extension github-actions hermes-labs jailbreak llm llm-evaluation llm-security prompt-injection prompt-security python red-teaming regression-testing
1 Open Issue Need Help Last updated: Sep 29, 2026

Open Issues Need Help

View All on GitHub
hermes-labs-ai/hermes-jailbench
9

Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench

Python
#agent-skills#ai-red-team#ai-reliability#ai-safety#benchmark#claude-code-plugin#deterministic-testing#gemini-cli-extension#github-actions#hermes-labs#jailbreak#llm#llm-evaluation#llm-security#prompt-injection#prompt-security#python#red-teaming#regression-testing