basitalisandhu

basitalisandhu/agentdojo-mcp

Run AgentDojo against MCP servers: map a task suite's tools onto a real MCP server's tools, run the benchmark's user tasks and injection tasks against that tool surface, and report utility, targeted attack success and every MCP call per task. Mapping proposer and validator, offline replay, stdlib core.

6 good first / help-wanted issues · Python · last activity Oct 4, 2026

0 stars 0 forks 0 watchers Python MIT License
agent-security agentdojo ai-agents ai-security benchmark cli evaluation hacktoberfest llm-evaluation llm-security mcp mcp-server model-context-protocol prompt-injection python red-teaming security-tools
View on GitHub Website
6 Open Issues Need Help Last updated: Oct 4, 2026

Open Issues Need Help

View All on GitHub
Add description similarity to map scoring 9 days ago
help wanted hacktoberfest
basitalisandhu/agentdojo-mcp
0

Run AgentDojo against MCP servers: map a task suite's tools onto a real MCP server's tools, run the benchmark's user tasks and injection tasks against that tool surface, and report utility, targeted attack success and every MCP call per task. Mapping proposer and validator, offline replay, stdlib core.

Python
#agent-security#agentdojo#ai-agents#ai-security#benchmark#cli#evaluation#hacktoberfest#llm-evaluation#llm-security#mcp#mcp-server#model-context-protocol#prompt-injection#python#red-teaming#security-tools
Support glob patterns in --tasks and --injection-task 20 days ago
help wanted good first issue hacktoberfest
basitalisandhu/agentdojo-mcp
0

Run AgentDojo against MCP servers: map a task suite's tools onto a real MCP server's tools, run the benchmark's user tasks and injection tasks against that tool surface, and report utility, targeted attack success and every MCP call per task. Mapping proposer and validator, offline replay, stdlib core.

Python
#agent-security#agentdojo#ai-agents#ai-security#benchmark#cli#evaluation#hacktoberfest#llm-evaluation#llm-security#mcp#mcp-server#model-context-protocol#prompt-injection#python#red-teaming#security-tools
Add a mapping for a public MCP server 21 days ago
help wanted good first issue hacktoberfest
basitalisandhu/agentdojo-mcp
0

Run AgentDojo against MCP servers: map a task suite's tools onto a real MCP server's tools, run the benchmark's user tasks and injection tasks against that tool surface, and report utility, targeted attack success and every MCP call per task. Mapping proposer and validator, offline replay, stdlib core.

Python
#agent-security#agentdojo#ai-agents#ai-security#benchmark#cli#evaluation#hacktoberfest#llm-evaluation#llm-security#mcp#mcp-server#model-context-protocol#prompt-injection#python#red-teaming#security-tools
Show per-call timing in the Markdown report 21 days ago
help wanted good first issue hacktoberfest
basitalisandhu/agentdojo-mcp
0

Run AgentDojo against MCP servers: map a task suite's tools onto a real MCP server's tools, run the benchmark's user tasks and injection tasks against that tool surface, and report utility, targeted attack success and every MCP call per task. Mapping proposer and validator, offline replay, stdlib core.

Python
#agent-security#agentdojo#ai-agents#ai-security#benchmark#cli#evaluation#hacktoberfest#llm-evaluation#llm-security#mcp#mcp-server#model-context-protocol#prompt-injection#python#red-teaming#security-tools
Add inspect --format markdown 21 days ago
help wanted good first issue hacktoberfest
basitalisandhu/agentdojo-mcp
0

Run AgentDojo against MCP servers: map a task suite's tools onto a real MCP server's tools, run the benchmark's user tasks and injection tasks against that tool surface, and report utility, targeted attack success and every MCP call per task. Mapping proposer and validator, offline replay, stdlib core.

Python
#agent-security#agentdojo#ai-agents#ai-security#benchmark#cli#evaluation#hacktoberfest#llm-evaluation#llm-security#mcp#mcp-server#model-context-protocol#prompt-injection#python#red-teaming#security-tools
Add replay --diff between two results files 21 days ago
help wanted good first issue hacktoberfest
basitalisandhu/agentdojo-mcp
0

Run AgentDojo against MCP servers: map a task suite's tools onto a real MCP server's tools, run the benchmark's user tasks and injection tasks against that tool surface, and report utility, targeted attack success and every MCP call per task. Mapping proposer and validator, offline replay, stdlib core.

Python
#agent-security#agentdojo#ai-agents#ai-security#benchmark#cli#evaluation#hacktoberfest#llm-evaluation#llm-security#mcp#mcp-server#model-context-protocol#prompt-injection#python#red-teaming#security-tools