Agenta-AI

Agenta-AI/agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

4.3K stars 555 forks 4.3K watchers TypeScript Other
agents evaluation llm-as-a-judge llm-evaluation llm-framework llm-monitoring llm-observability llm-platform llm-playground llm-tools llmops observability prompt-engineering prompt-management rag-evaluation
View on GitHub Website
36 Open Issues Need Help Last updated: Jul 1, 2026

Open Issues Need Help

View All on GitHub
Commit modal shows wrong variant name for new variants 2 days ago
good first issue Frontend ux bug Playground
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Swap sendgrid by SMTP 7 days ago
good first issue Backend Feature Request
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Run errors not shown correctly in evaluation table 7 days ago
bug good first issue Frontend Evaluation Bug Report
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[Bug] In the observability trace table, we are not displaying the exceptions in the output column 20 days ago
bug good first issue Frontend
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Trace drawer entity chips show IDs on hover 22 days ago
bug good first issue Frontend
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Text inputs in test case drawer auto-switch type 23 days ago
good first issue Frontend UX ux bug Playground Bug Report
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Open evaluator registry button links incorrectly 23 days ago
bug good first issue Frontend Bug Report
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[feat][UX/UI] Hide mapping and preview until a test set is selected 24 days ago
good first issue Frontend Observability
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Create API key modal overflows API key text 26 days ago
bug good first issue UI
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Fix SDK eval breadcrumbs showing `Auto Evals` 28 days ago
bug good first issue Frontend
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[AGE-2166] Add Reset Password Functionality to Workspace UI about 1 month ago
good first issue Frontend UI feature
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[UX bug] Misleading error message in evaluation table about 1 month ago
good first issue Frontend UX ux bug
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Together AI model run fails with InvalidSecretsV0Error about 1 month ago
bug good first issue Backend SDK Evaluation
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[UI/UX] Improve click behavior in tables about 1 month ago
good first issue Frontend UI UX Improvement
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Improve dark mode polish for header and tool menu about 1 month ago
good first issue Frontend UI UX
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[AGE-1858] Add llms.txt to docs about 1 month ago
documentation good first issue Low priority
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
fix(frontend): block deletion of an app's last revision 3 months ago
bug good first issue Frontend UI UX
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[bug] Auto-scroll issue in variant drawer with large prompts 4 months ago
bug good first issue Frontend UI ux bug
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Curly template formatter raises false TemplateFormatError when substituted values contain {{variable}} patterns 4 months ago
bug good first issue SDK
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[Bug] Inconsistent Password Validation Rule in Signup and Signin 5 months ago
bug good first issue unconfirmed bug
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
SDK: curly template format fails when substituted values contain same variable names 5 months ago
bug good first issue SDK
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Bug: Embedding traces incorrectly shown with chat UI in trace drawer 5 months ago
bug good first issue Frontend UI
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Bug: Comments (strings) in annotation tab not shown in trace drawer 5 months ago
bug good first issue Frontend UI Observability
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Commit button disabled when clearing max token field 5 months ago
bug good first issue Frontend Playground
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[chore] Print warning messages when any instrumentation method is called before ag.init() 6 months ago
good first issue dev experience SDK
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Add indentation and symbols to tools list for clarity 6 months ago
good first issue UI UX
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[docs] Create an observability example with Anthropic/TS 6 months ago
documentation good first issue example Observability
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Add copy button to error / value popovers in evaluation scenarios table 6 months ago
bug good first issue Frontend UI feature Evaluation
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Variant state changes to draft with large prompts 7 months ago
bug good first issue Frontend ux bug
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[UX] Deleting a trace should load the next trace 7 months ago
good first issue Frontend UX
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[chore] Add warning when ag.init() is called and API key is not set 7 months ago
good first issue dev experience SDK Chore
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[AGE-3376] Fix incorrect height of load application container in evaluator playground 7 months ago
bug good first issue Frontend UI
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[docs] Create example and write docs for Google ADK integration 7 months ago
documentation good first issue example Observability integration
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Fix UI bug on survey page 7 months ago
bug good first issue Frontend UX
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
Empty state configure provider 8 months ago
enhancement good first issue Frontend UX
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation
[AGE-3362] [bug] Text overlap issue in Playground chat mode UI 8 months ago
bug good first issue Frontend UI Playground
Agenta-AI/agenta
4.3K

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

TypeScript
#agents#evaluation#llm-as-a-judge#llm-evaluation#llm-framework#llm-monitoring#llm-observability#llm-platform#llm-playground#llm-tools#llmops#observability#prompt-engineering#prompt-management#rag-evaluation