hvardhan878/agentcheck

AI Summary: Integrate support for Anthropic Claude and Google Gemini into the AgentCheck tool. This involves creating provider-specific modules to handle API calls, cost calculations, and different response formats, updating the trace analysis, adding demo agents, and writing tests and documentation.

AI Summary: Improve the user experience of the `agentcheck` CLI tool by enhancing error messages, help text, and input validation. This involves modifying the `cli.py` and `utils.py` files to provide more informative feedback to users, including suggestions and troubleshooting tips, and adding tests to cover error scenarios.

AI Summary: Implement JSONPath support in the `agentcheck assert` command to allow for more flexible assertions on trace data. This involves adding a new dependency (`jsonpath-ng`), parsing JSONPath expressions, supporting various comparison operators, handling array indexing and nested objects, and adding comprehensive tests and updated documentation.

AI Summary: Enhance the AgentCheck analytics dashboard by adding interactive cost analysis charts. This involves creating a cost over time line chart, a cost breakdown pie chart, and displaying cost efficiency metrics. The charts should be interactive, load data from trace files, and integrate seamlessly into the existing dashboard's design.

AI Summary: Improve the test coverage of the `agentcheck` Python library to over 90% by writing unit and integration tests for core tracing, utility, and CLI functions, including error handling and edge cases. This involves creating or modifying test files and using pytest for execution and coverage reporting.

AI Summary: Implement a new CLI command `agentcheck stats` for the agentcheck project. This command should parse one or more trace JSON files, calculate statistics (total steps, LLM calls, total cost, duration, models used, average response length), and present the results in a user-friendly format. The implementation will involve modifying `agentcheck/cli.py`, `agentcheck/utils.py`, and adding tests in `tests/test_cli.py`.

AI Summary: Fix the implementation of the `@agentcheck.deterministic_replay()` decorator in the `agentcheck` Python library. This involves ensuring the decorator correctly passes trace instances to agent functions, implements automatic baseline establishment, integrates with trace context management, updates the demo to use the decorator, and adds tests for the decorator's functionality.