sdp5/green-gov-rag

AI Summary: This GitHub issue proposes implementing a query classification system to categorize user queries (e.g., planning, emissions) within a RAG system. The goal is to improve document routing and filtering. It involves formalizing existing topic extraction into a new `QueryClassifier` class, enhancing classification logic (potentially with LLMs), expanding categories, integrating it into the API pipeline for automatic source filtering, returning classifications in API responses, and tracking accuracy.

AI Summary: This issue aims to implement real-time streaming of LLM responses to the frontend, significantly improving perceived response times and user experience. It involves updating backend LLM calls and API endpoints to use Server-Sent Events (SSE), and modifying the frontend to consume these streams with visual feedback like typing indicators.

AI Summary: This GitHub issue proposes implementing an automated evaluation system for RAG quality using standard metrics like retrieval accuracy, answer relevance, and faithfulness. It involves creating a dedicated evaluation framework, collecting ground truth Q&A pairs, integrating the RAGAS library for specific metrics, and setting up a pytest suite with CI/CD integration. Additionally, the plan includes tracking metrics over time and adding an API endpoint for manual evaluation.

AI Summary: This issue proposes implementing an optional re-ranking step using a cross-encoder model (e.g., `ms-marco-MiniLM-L6-v2`) to improve the relevance of search results. After an initial hybrid search, the top 20 results will be re-ranked to return the most relevant 5, with the feature being configurable via an environment variable and benchmarked for its impact on both relevance and latency.

AI Summary: This GitHub issue outlines the creation of a health monitoring system for document sources, primarily government websites, to proactively detect availability issues such as downtime or broken links. It involves developing a Python service to ping source URLs, track metrics like HTTP status and response time, store this data in a database, and expose source health status via a new admin API endpoint.