Designing an Agentic RAG API: What the Response Actually Tells You - Part One

An API response can look simple, but each field in the contract matters. In deepVerified, the /v1/chat endpoint accepts a model ID, a knowledge-base ID, and a user message. The response contains three fields: id, model, and content. The id identifies the API response, while model echoes the requested public model ID. Neither should be confused with an execution trace or the underlying runtime model.

API call samples using two different models

Model selection also matters. When we submitted the same question about China's 2026 GDP growth forecast using two different models, qwen returned a forecast of 5.0%, while deepseek-r1 said the available context did not contain a forecast. These are examples of different model outputs, not verification of the forecast itself. The API provides a consistent response structure, but the content can vary with the selected model.

How can a client discover what a public model ID actually represents? DeepVerified's /v1/models endpoint provides the available model IDs, their providers, and the underlying runtime models currently mapped to them. For example, a client can request qwen without needing to specify the Ollama runtime model directly. This separates the API's public interface from the model implementation behind it.

Observability is another distinct concern. The API response ID identifies the returned response; it is not a LangSmith trace ID. deepVerified uses LangSmith separately to inspect execution traces and investigate application behavior. Keeping the response contract focused on client needs while maintaining a separate path for developer diagnostics avoids exposing internal execution details unnecessarily.

There is one more question worth exploring: how does an API request connect the model to the documents that should support its answer? A platform may support document uploads, but that does not by itself mean every chat request retrieves and uses those documents. In Part 2, we'll look at how document references and retrieval could become an explicit part of an Agentic RAG API, moving from model-generated responses toward answers grounded in supporting evidence.