An AI agent can retrieve architecture documentation, API specifications, and troubleshooting procedures, but that knowledge has an important limitation: it describes how a system is designed to work, not necessarily what is happening right now. This distinction becomes especially important in operational environments, where investigating an incident requires more than retrieving the correct document. Teams also need current evidence from application metrics, logs, and distributed traces. Combining Retrieval-Augmented Generation (RAG) with observability offers an interesting direction for making AI agents more useful in technical investigations.
Consider an API experiencing intermittent latency. A traditional RAG implementation might retrieve the relevant service documentation or troubleshooting runbook and explain the expected request flow. However, it cannot establish the actual cause of the slowdown from documentation alone. Live operational signals provide another layer of context: metrics can reveal latency changes, logs can expose application errors, and distributed traces can help identify which service or dependency contributed to the delay. The agent could retrieve the relevant documentation first, then use authorized tools to query these operational signals and correlate the results.
This creates an opportunity to extend an Agentic RAG architecture beyond document retrieval. The knowledge layer provides relatively stable information, such as architecture diagrams, runbooks, and API contracts. A separate operational context layer provides time-sensitive information through observability APIs. An orchestration layer can coordinate both: identify the relevant documentation, determine which telemetry is needed, call the appropriate tools, and produce an investigation summary with references to the evidence collected. Importantly, the agent should distinguish observed facts from hypotheses and avoid presenting correlation as proof of root cause.
In deepVerified's current Agentic RAG implementation, LangGraph coordinates the retrieval and generation workflow, while LangSmith provides visibility into agent execution through traces. These capabilities provide a starting point for exploring this architecture, but they are not the same as querying live infrastructure telemetry. Integrating external metrics, logs, and distributed traces would require additional connectors, permissions, and tool interfaces. A possible next step would be to expose selected observability queries as controlled agent tools, allowing the workflow to retrieve operational evidence without giving the model unrestricted access to monitoring systems.
The larger opportunity is to move from an AI agent that answers questions using documented knowledge toward one that can assist with evidence-based operational investigations. RAG explains what a system is expected to do; observability helps establish what it is actually doing. Bringing these sources together could make technical investigations more contextual, traceable, and useful to engineering teams. This is a potential architectural direction for Agentic RAG—not a claim that live observability integration is already implemented in deepVerified.