shubhamofbce

shubhamofbce/vaanieval

Open-source evaluation platform for Voice AI agents. Import real conversations, score calls with LLMs, replay audio and transcripts, and track quality over time. Built for ElevenLabs, Vapi, and production voice agents.

10 stars 2 forks 10 watchers Python MIT License
elevenlabs evaluation evaluation-framework vapi voice voice-ai
View on GitHub Website
6 Open Issues Need Help Last updated: Jul 3, 2026

Open Issues Need Help

View All on GitHub
Improve Import Page for UX about 2 hours ago
good first issue Frontend
shubhamofbce/vaanieval
10

Open-source evaluation platform for Voice AI agents. Import real conversations, score calls with LLMs, replay audio and transcripts, and track quality over time. Built for ElevenLabs, Vapi, and production voice agents.

Python
#elevenlabs#evaluation#evaluation-framework#vapi#voice#voice-ai
Add proper screenshots in readme 3 days ago
good first issue
shubhamofbce/vaanieval
10

Open-source evaluation platform for Voice AI agents. Import real conversations, score calls with LLMs, replay audio and transcripts, and track quality over time. Built for ElevenLabs, Vapi, and production voice agents.

Python
#elevenlabs#evaluation#evaluation-framework#vapi#voice#voice-ai
Evaluator Provider should support more providers and models 3 days ago
good first issue Backend
shubhamofbce/vaanieval
10

Open-source evaluation platform for Voice AI agents. Import real conversations, score calls with LLMs, replay audio and transcripts, and track quality over time. Built for ElevenLabs, Vapi, and production voice agents.

Python
#elevenlabs#evaluation#evaluation-framework#vapi#voice#voice-ai
Home Page full of AI Slop, Need cleanup 3 days ago
good first issue Frontend
shubhamofbce/vaanieval
10

Open-source evaluation platform for Voice AI agents. Import real conversations, score calls with LLMs, replay audio and transcripts, and track quality over time. Built for ElevenLabs, Vapi, and production voice agents.

Python
#elevenlabs#evaluation#evaluation-framework#vapi#voice#voice-ai
Add frontend tests for critical user workflows 7 days ago
enhancement help wanted
shubhamofbce/vaanieval
10

Open-source evaluation platform for Voice AI agents. Import real conversations, score calls with LLMs, replay audio and transcripts, and track quality over time. Built for ElevenLabs, Vapi, and production voice agents.

Python
#elevenlabs#evaluation#evaluation-framework#vapi#voice#voice-ai
Add contract tests for the Vapi provider adapter 7 days ago
enhancement good first issue
shubhamofbce/vaanieval
10

Open-source evaluation platform for Voice AI agents. Import real conversations, score calls with LLMs, replay audio and transcripts, and track quality over time. Built for ElevenLabs, Vapi, and production voice agents.

Python
#elevenlabs#evaluation#evaluation-framework#vapi#voice#voice-ai