AgentPostmortem/Evalgate

temperature and maxTokens are plumbed through the provider but cannot be set by a suite about 1 hour ago

enhancement good first issue

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

Cost is one hardcoded rate per provider, applied to every model and both token directions about 1 hour ago

bug enhancement help wanted

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

Provider calls have no timeout and no retry, so a 429 fails the build about 1 hour ago

bug enhancement help wanted

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

One provider or scorer error discards the entire run about 1 hour ago

bug enhancement help wanted

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

Deleting a failing case makes the gate green: removed cases are never regressions about 1 hour ago

bug enhancement help wanted

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

An invalid pattern in a JSON schema throws and kills the whole run about 1 hour ago

bug good first issue

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

exact-match always collapses internal whitespace, and trim: false does not turn it off about 2 hours ago

good first issue

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

Docs: --concurrency, --junit and --degrade are missing from evalgate --help about 2 hours ago

documentation good first issue

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

Bug: a run with zero cases reports passed and exits 0, so the gate silently disables itself about 2 hours ago

bug help wanted

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

--tags and --concurrency are ignored by baseline and compare about 2 hours ago

bug help wanted

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

evalgate list hardcodes the provider names instead of asking the registry about 2 hours ago

enhancement good first issue

AgentPostmortem/Evalgate

0

Prompt and agent regression CI. The build fails when your prompt gets dumber. GitHub Action with PR delta comments.

TypeScript

#ai#ci#evals#github-action#llm#prompt-engineering#typescript

Open Issues Need Help