The RAG Pipeline Behind Sovereign Document Intelligence: How Your Firm's Files Stay On Your Server | Insights

Imagine asking a 100-page PDF commercial lease a question in plain English and getting an answer with source citations — without transmitting a single page of that document to a third party. That capability is called Retrieval-Augmented Generation (RAG). It is the feature that transforms a general-purpose AI chat interface into a sovereign document intelligence system. And in Private AI Ltd’s architecture, every step of the RAG pipeline — document upload, chunking, embedding, vector storage, retrieval, and response generation — occurs entirely within the firm’s own contracted bare-metal server, behind a zero-egress perimeter.

No external embedding API. No cloud vector database. No third-party document processor. The firm’s files enter the server through the secure encrypted tunnel and never leave.

For a deeper dive into the zero-egress architecture that makes this possible, read our Zero-Egress AI explainer. This piece explains how the RAG pipeline works, what each stage does to the firm’s documents, and why every stage is confined within the zero-egress boundary.

What RAG Does — And Why It Matters for Regulated Firms

A large language model without RAG is a general-purpose text generator. It can draft, summarise, and reason — but only from its pre-trained knowledge. It has never read your firm’s contracts, your precedent library, or your client’s matter files. It cannot answer questions about them because it does not know they exist.

RAG solves this by giving the model access to the firm’s own documents at query time. When a user asks “what are the break clauses in the Henderson lease?”, the system searches the firm’s uploaded documents, retrieves the relevant passages, and feeds them to the model alongside the question. The model generates an answer grounded in the actual document text — with source citations pointing to the specific passages it relied upon.

For a law firm, this means your team can interrogate precedent packs across multiple matters, extract key terms from lengthy commercial contracts in seconds, compare clause language across different versions of a document, and receive research synthesis grounded in the organisation’s own internal guidance rather than the model’s general training data.

For an accountancy practice, it means associates can query tax guidance documents, extract figures from prior year financial statements, and cross-reference HMRC correspondence against the firm’s internal advisory notes.

The compliance question is not whether RAG is useful. It is whether the documents — client-privileged, commercially sensitive, statutorily confidential — are processed within the firm’s controlled perimeter or transmitted to a third-party cloud service.

Stage 1: Document Upload

Users upload documents — PDFs, Word files, text — through the user interface. The upload travels through the secure encrypted tunnel to the bare-metal server. The file is stored on the encrypted persistent block storage, with encryption at rest provided by the infrastructure provider under the firm’s direct data-processing agreement.

The document never passes through Private AI Ltd’s systems. It never passes through an external document processing API. It never leaves the firm’s contracted infrastructure.

Stage 2: Chunking

The uploaded document is split into smaller segments — “chunks” — typically a few hundred tokens each. Chunking ensures that the retrieval system can locate specific passages rather than returning entire documents. The chunking process runs within the application runtime on the bare-metal server.

No external service is involved. The chunking algorithm operates on the local file, stored on local encrypted storage, using local compute resources. The chunk boundaries are stored alongside the document metadata in the local database.

Stage 3: Embedding

Each chunk is converted into a vector embedding — a high-dimensional numerical representation that captures the semantic meaning of the text. This is the mathematical step that makes retrieval possible: similar meanings produce similar vectors, allowing the system to find passages relevant to a query even when the exact words differ.

The embedding model runs locally within the retrieval pipeline on the bare-metal server. No external embedding API is called. This is a critical architectural distinction. Many cloud-based RAG systems use OpenAI’s embedding API (api.openai.com/v1/embeddings) or similar third-party services. Every document chunk sent to such an API is transmitted to a third-party server — creating a Data Processor relationship, an international transfer, and a data sovereignty breach.

Private AI Ltd’s architecture eliminates this vector entirely. The embedding model is a local file running local inference on local hardware. The document text enters the embedding function and the resulting vector exits — both on the same machine. No network packet is generated. The zero-egress perimeter ensures it could not be, even if the software attempted it.

Stage 4: Vector Storage

The generated embeddings are stored in the vector database running alongside the user interface on the bare-metal server. The vector database’s data directory is mapped exclusively to the encrypted persistent storage.

The vector store persists across application upgrades and server restarts because the storage volume is architecturally decoupled from the application layer. Upgrading the user interface does not touch the vector database’s data directory. The firm’s document embeddings survive every maintenance cycle intact.

The embeddings are governed by the same role-based access controls as the general chat interface. Document collections can be scoped to specific user roles — a family law team sees only family law precedents, an M&A team sees only their deal room documents. Access is controlled at the application layer and auditable via the application database.

Stage 5: Retrieval

When a user submits a query, the system embeds the query using the same local embedding model, then performs a vector similarity search against the stored embeddings. The vector database returns the most semantically relevant document chunks — the passages that are closest in meaning to the question.

This search is entirely local. The query embedding is generated on the bare-metal server. The similarity search is performed against vectors stored on the local encrypted volume. No external search API is called. No query metadata is transmitted to any third party.

Stage 6: Augmented Generation

The retrieved document chunks are prepended to the original query and submitted to the local inference layer for open-weight model inference. The model generates a response that is grounded in the firm’s actual document text — not in its general pre-trained knowledge. Source citations reference the specific document passages that informed the answer.

The entire generation step occurs on the bare-metal server. The prompt — query plus retrieved chunks — enters the inference layer locally. The response exits locally. The user sees it in the user interface via the secure encrypted tunnel. No external inference API is called. No third-party AI vendor processes any part of the augmented prompt.

The Six-Stage Sovereignty Guarantee

Every stage of the RAG pipeline satisfies the same constraint: no data leaves the firm’s contracted bare-metal server.

Stage	Process	Location	External API?
Upload	Document stored on encrypted volume	Bare-metal server	No
Chunking	Document split into segments	Application runtime on server	No
Embedding	Chunks converted to vectors	Local embedding model	No
Storage	Vectors stored in vector database	Encrypted block storage	No
Retrieval	Similarity search against vectors	Vector database on server	No
Generation	Model produces response with citations	Inference layer on server	No

Six stages. Six “No.” The entire document intelligence pipeline is confined within the zero-egress boundary. The firm’s files enter the server and never leave.

What This Means for the IT Director

Sovereign document Q&A is not a theoretical capability. It is a production-ready system running on dedicated hardware, using enterprise-grade open-weight models, with a vector database that persists across maintenance cycles and a role-based access layer that governs access at the document collection level.

The question for your firm is not “can this work?” The question is: “are we currently sending client-privileged documents to a third-party cloud to get the same capability — and what does that cost us in compliance exposure?”

Talk to our team to see the RAG pipeline in action — upload a sample document, query it in plain English, and evaluate the response quality and source citation accuracy against your organisation’s specific use cases.

Private AI Ltd (SC881079)