attenlabs/hotato

RFC: a share-safe role-play fixture format (scripted defects, recorded on purpose) about 3 hours ago

help wanted rfc

0

Regression testing for voice agents. The transcript passed; the call failed. hotato scores the turn timing and checks what the agent said against what the backend did, then gates CI. Self-hosted, MIT.

Python

#ai-agents#ci#cli#conversation-qa#evaluation#llm-evaluation#mcp#offline#ollama#pytest#regression-testing#self-hosted#turn-taking#voice#voice-agent-testing#voice-agents#voice-ai#voice-testing

Conformance check: DTMF logged-as-sent vs present-in-delivered-audio about 3 hours ago

good first issue

attenlabs/hotato

0

Regression testing for voice agents. The transcript passed; the call failed. hotato scores the turn timing and checks what the agent said against what the backend did, then gates CI. Self-hosted, MIT.

Python

#ai-agents#ci#cli#conversation-qa#evaluation#llm-evaluation#mcp#offline#ollama#pytest#regression-testing#self-hosted#turn-taking#voice#voice-agent-testing#voice-agents#voice-ai#voice-testing

Fixture: browser-vs-telephony parity on silence gaps (one scenario, two renders) about 3 hours ago

good first issue

attenlabs/hotato

0

Regression testing for voice agents. The transcript passed; the call failed. hotato scores the turn timing and checks what the agent said against what the backend did, then gates CI. Self-hosted, MIT.

Python

#ai-agents#ci#cli#conversation-qa#evaluation#llm-evaluation#mcp#offline#ollama#pytest#regression-testing#self-hosted#turn-taking#voice#voice-agent-testing#voice-agents#voice-ai#voice-testing

Fixture: interruption while a tool call is in flight (duplicate side-effect class) about 3 hours ago

help wanted

attenlabs/hotato

0

Regression testing for voice agents. The transcript passed; the call failed. hotato scores the turn timing and checks what the agent said against what the backend did, then gates CI. Self-hosted, MIT.

Python

#ai-agents#ci#cli#conversation-qa#evaluation#llm-evaluation#mcp#offline#ollama#pytest#regression-testing#self-hosted#turn-taking#voice#voice-agent-testing#voice-agents#voice-ai#voice-testing

Fixture: structured utterances with internal pauses (digit strings and emails) about 3 hours ago

good first issue

attenlabs/hotato

0

Regression testing for voice agents. The transcript passed; the call failed. hotato scores the turn timing and checks what the agent said against what the backend did, then gates CI. Self-hosted, MIT.

Python

#ai-agents#ci#cli#conversation-qa#evaluation#llm-evaluation#mcp#offline#ollama#pytest#regression-testing#self-hosted#turn-taking#voice#voice-agent-testing#voice-agents#voice-ai#voice-testing

Fixture: sample-boundary interruption with a short leading phoneme (dropped-leading-audio class) about 3 hours ago

good first issue

attenlabs/hotato

0

Regression testing for voice agents. The transcript passed; the call failed. hotato scores the turn timing and checks what the agent said against what the backend did, then gates CI. Self-hosted, MIT.

Python

#ai-agents#ci#cli#conversation-qa#evaluation#llm-evaluation#mcp#offline#ollama#pytest#regression-testing#self-hosted#turn-taking#voice#voice-agent-testing#voice-agents#voice-ai#voice-testing

Open Issues Need Help