When a Document Is Deleted, Does RAG Really Forget It?

Deleting a document from a Retrieval-Augmented Generation (RAG) system sounds straightforward: remove the file, delete its metadata, and clean up the vector index. But how do we know the document's content is no longer available to retrieval? In RAG, deletion is not just a database operation. It is a lifecycle operation that needs to be verified across the storage and retrieval layers.

While extending deepVerified's document-management API, I implemented document deletion across three components: the stored PDF, the SQLite document registry, and the Chroma vector index. The deletion endpoint returns 204 No Content on success. But an HTTP status code only tells us that the operation completed according to the API contract; it does not, by itself, demonstrate that the deleted document can no longer influence retrieval.

Lifecycle Diagram - a document deletion test should verify both the underlying storage state and the resulting retrieval behavior.

To test the behavior, I uploaded a PDF containing a unique marker: DELETE-RETRIEVAL-59381. Before deletion, a query through /v1/chat returned that exact marker. I then deleted the document through the API and repeated the query. This time, the response was: “There is no unique deletion marker identified in the provided context.” The document was also absent from the knowledge-base document listing, and a direct check of Chroma confirmed that zero chunks remained for the deleted document.

Validation session

The important part is that these checks verify different outcomes. The API confirms the deletion request succeeded; the document listing confirms the record is no longer listed; the vector-index check confirms that the document's chunks have been removed; and the follow-up retrieval test checks the behavior that matters to an application consuming the knowledge base. Using a unique marker makes the test particularly useful because it is much easier to distinguish from unrelated content than a common phrase.

The lesson: verify deletion through both storage state and retrieval behavior. A RAG system should not be considered correct merely because its deletion endpoint returns success. The complete upload–retrieve–delete–verify sequence should be repeatable, so future changes to document management can be checked against the same expectations. For deepVerified, this is one step toward making document lifecycle behavior as explicit and testable as the API itself.