Skip to content
AIKI

Document AI for Hundreds of Thousands of Pages

Document AI for hundreds of thousands of pages: how retrieval works, plus EU-region API, private cloud and on-premises options compared.

6 min lesing

Key takeaways

  1. AI indexes large archives and retrieves only the relevant passages for each question.
  2. Retrieval stays necessary for a growing archive with per-document access rights, even when a model has a large context window.
  3. Endpoint choice decides where data is processed: EU region, data zone or global.
  4. Check region, training use, source citations, access control, OCR quality and test results before you buy.
Two colleagues review a printed document at a laptop in an archive room while an AI colleague with a white sphere head lifts an archive box from a shelf

How does AI handle hundreds of thousands of pages?

AI systems handle hundreds of thousands of pages by indexing the documents in advance, retrieving the few passages that match a question, and writing an answer with references to the source pages. You can run this in three ways: a managed API pinned to an EU region, your own cloud account with the provider's models pinned to a region or data zone, or an on-premises installation with open-weight models.

What a large context window leaves open

Model vendors do advertise large context windows. Anthropic's documentation lists a 1M-token context window for its latest models and 200k for others, such as the deprecated Claude Sonnet 4.5. Google's Gemini API documentation says many Gemini models come with context windows of 1 million or more tokens.

Google describes 1 million tokens as "8 average length English novels". An archive of contracts, reports and e-mails over many years is far larger, and it keeps growing.

Three practical limits remain even with a large window:

  • The collection is larger than any window. An archive keeps growing, while a context window is fixed per request.
  • Every request uses window space for what you send. Anthropic's documentation states that all input and output components count toward the context window. Sending a large archive with every question fills that space each time.
  • Access rights get lost. If you paste everything into one prompt, the model sees documents that the person asking may not be allowed to read.

Google's long-context documentation notes that such techniques remain valuable in specific scenarios. Our reading: a large, changing archive with different access rights per document is one of those scenarios.

How document analysis with retrieval works

Retrieval-augmented generation (RAG) splits the work into a preparation phase and a question phase.

Preparation, done once and kept up to date

  1. OCR for scanned pages. Scanned contracts and old reports are images. Optical character recognition turns them into text, and the quality of this step sets a ceiling for everything after it.
  2. Chunking. Each document is cut into passages, typically a few paragraphs, with the document name, page number and section kept as metadata.
  3. Embeddings. A model turns each passage into a list of numbers that represents its meaning. Passages about the same topic end up close to each other, even when they use different words.
  4. Index with access rights. The passages and their embeddings go into a search index. Each passage carries the same access rights as its source document.

Answering a question

  1. Search with access filter. The question is turned into an embedding and matched against the index, restricted to passages the asking user may read. Many setups combine this with ordinary keyword search, which helps with contract numbers, names and exact clauses.
  2. Re-ranking. A second model sorts the top hits by how well they answer the question.
  3. Answer with source references. The language model gets the best passages and writes the answer. The answer cites a document and page for each claim, so a person can open the original and check it.

The language model only sees a few pages per question, so the work behind one answer stays roughly the same as the archive grows. What changes with scale is how often the right passage is found, which is why a test set matters.

An illustrative example: a lawyer asks which supplier contracts allow termination with three months' notice. The search finds the relevant clauses in a handful of contracts, the model lists them, and each line links to the contract and page. Questions that need every document at once, such as "list every contract with a termination clause", are harder for retrieval, because the answer depends on finding all matches. For those, run the question as a batch job over every document and store each result.

A single long document, such as a large tender or an annual report, may fit in a large context window on its own; test it with the model you plan to use. Retrieval becomes necessary when the question spans many documents or the collection keeps growing.

Deployment options compared

All three options follow the same retrieval steps. They differ in who operates the models, which contract covers the data, and where prompts and documents are processed. In the first two, a provider hosts the models and the endpoint or zone you choose decides residency. In the third, you host the models yourself.

OptionWhere data is processedWhat the provider documentsTypical fit
Provider-hosted API with an EU endpoint (example: Google Vertex AI)Inside the EU if you choose an EU endpointGoogle: jurisdictional endpoints keep ML processing inside the chosen region such as the EU; global endpoints give no residency guaranteeFast start, low operations load
Model service inside your own cloud tenant (Azure, AWS), pinned to a zoneWithin the data zone or geography you pickMicrosoft Foundry: Data Zone types process data only within the specified zone (US, EU or APAC); Amazon Bedrock: geographic cross-Region inference keeps data processing within boundaries such as the EUExisting cloud contracts, central identity and logging
On-premises with open-weight modelsOn your own serversMistral lists open-weight models under Apache 2.0, such as Mistral Small 4; Gemma has its own terms of useData stays on your servers, you own the operations

Sources: the provider pages linked in each row, fetched 15 November 2026.

  • Endpoint choice decides residency. On Google's platform, data stored at rest stays in the location you chose, but requests to a global endpoint may be processed in any Google Cloud location. Microsoft documents the same split: global types may process data in any Azure region, data zone types stay in the zone. Amazon Bedrock's documentation points to geographic cross-Region inference when you have data residency requirements.
  • Open weights come with different licenses. Mistral's table shows Apache 2.0 for some models and a separate "Mistral Ecosystem Open Weight License" for Mistral Large 4. Have someone read the license for the exact model you plan to use.
  • On-premises moves the work to you. You size the GPU servers, patch the stack, monitor quality and replace models when better ones arrive. For the cost side of that trade-off, see local, cloud or hybrid for AI agents. For the Gemma specifics, see Gemma for Norwegian businesses.

A solution can be set up so that documents, the index and model calls all stay in one chosen EU region, or on servers you control. If your data classification requires that no third party, including a cloud host, processes the text, on-premises is the option that meets it. AI and GDPR in Norway covers the legal side of data residency and control.

Private cloud usually means open-weight models on GPU servers in a data center you control, rented or owned. It carries the same operations load as on-premises. A hybrid pattern keeps documents and the index on your own servers and sends only the retrieved passages to a hosted model; the archive stays with you, while the passages still leave, so your data classification has to allow that.

What to check before you buy

QuestionWhat a good answer looks like
Where are prompts and documents processed?A named region or data zone, written into the contract, with no fallback to a global endpoint
Is data used to train models?A written statement from the provider that your data is not used for training
Does the answer cite its source?Every claim links to a document and page that a person can open
Are access rights kept per document?The search itself is restricted by the asking user's rights, so the model never sees other text
How are scanned pages handled?OCR quality is tested on your worst scans
What happens when documents change?New and edited files reach the index automatically, deleted files disappear from it
Can you measure quality?A test set of real questions with known answers, run before launch and after each change
What does it cost at your volume?A price per month at your document count and question count, including indexing and operations

For how to compare suppliers on these points, see how to choose an AI vendor in 2026.

How to start

Pick one document type with a clear owner, for example supplier contracts, and a few dozen real questions that people ask today. Build the pipeline on that set, measure how often the right passage is found, and only then widen the scope. Decide the deployment option once you know the data classification of that first set. Moving to a different embedding model later means re-indexing, which often happens when you change deployment option, so plan for that.

If you want help scoping a pilot, our AI development team can go through which deployment option fits your requirements.

Frequently asked questions

Del:LinkedInXFacebook

New articles by email

We use your email address to send you a confirmation link. Only after you confirm do we add you to the newsletter list at Resend, which sends on our behalf. You can unsubscribe in every email. Read the privacy policy.

Start with a pilot

  • Price after scoping

    excl. VAT, fixed price

  • One task

    One defined automation in the systems you already use

  • 4 weeks

    until the solution is live

Agreed goal before start. If we miss it, we fix it for free.

Afterwards: AI Partner at Price after scoping (normally 6 months' commitment), and the pilot amount is deducted from the first month. Or you stop.

In new projects, personal data is only processed on servers in the EU. Read about data and security