Searching decades of email: keywords first, meaning second

Pointing a language model at a mailbox and asking it questions sounds like a weekend project. Then it confidently tells you the wrong contract date, because it read a quoted copy of an old email inside a newer reply, or it cannot find invoice 10442 because the number has no "meaning" for a vector search to latch onto. Good search over a mail archive is mostly not about the model. It is about the data you feed it and the way you retrieve. This is how we approach it.

Email is messier than it looks

A mail archive that spans years, several mail systems and a few migrations has problems that ruin search before any AI is involved:

• Duplicates. The same message sits in Inbox, a project folder and an old export. Search returns it three times; an AI counts it as three sources agreeing.
• Quoted history. Every reply carries the whole thread below it. Index that naively and the oldest email in a thread appears in every reply, out of context and with the wrong date.
• Encodings. Older mail is full of legacy character sets, encoded headers and mislabelled parts. Decode it wrongly and names and non-English text turn into garbage.
• Attachments. The contract is the PDF, not the email that says "attached". If you do not extract attachment text, you cannot find the most important documents.
• Deletions. Mail deleted from the mailbox for retention reasons must disappear from every index too.

Clean before you index

Deduplicate on Message-ID, falling back to a hash of key headers and body when it is missing. Store one document per message and keep the list of folders it appeared in as metadata.

Thread on References and In-Reply-To, not on subject lines. Subjects change and "Re: invoice" is not unique. With proper threading you can show a whole conversation in order and let the retrieval step pull a thread together instead of single fragments.

Separate new text from quoted text. For each message, index the full body for keyword search, but embed only the part the author actually wrote. A simple start:

import re
from email import policy
from email.parser import BytesParser

QUOTE_MARKERS = [
    re.compile(r"^On .+ wrote:$", re.M),
    re.compile(r"^-{2,}\s*Original Message\s*-{2,}", re.M | re.I),
    re.compile(r"^From: .+\nSent: .+", re.M),
]

def new_text(raw: bytes) -> str:
    msg = BytesParser(policy=policy.default).parsebytes(raw)
    part = msg.get_body(preferencelist=("plain", "html"))
    text = part.get_content() if part else ""
    cut = len(text)
    for rx in QUOTE_MARKERS:
        m = rx.search(text)
        if m:
            cut = min(cut, m.start())
    lines = [l for l in text[:cut].splitlines() if not l.startswith(">")]
    return "\n".join(lines).strip()

Real mail needs more markers than this, in every language your company writes in, but the principle holds: the model should read what each person said, once, with the right date and sender.

Extract attachment text with standard tools (pdftotext for PDFs, library parsers for office formats) and index it as part of the message, while keeping a link to the original file.

Two indexes, because they answer different questions

A full text index answers questions with exact handles: a sender, a domain, a date range, an invoice number, a phrase, "has a PDF". It is fast, precise and explainable. Use one with language-aware stemming, so "invoices" finds "invoice", and with synonyms across the languages your company uses, so a search in one language finds mail written in the other.

A vector index answers questions about meaning: "when did the supplier first push back on the delivery schedule?" There may be no single keyword for that. An embedding model turns each cleaned message into a vector, and the search finds messages close in meaning to the question.

Neither is enough on its own. Vectors are weak at identifiers and numbers; keywords are weak at paraphrase. Run both, and combine:

1. Parse filters out of the question first: people, domains, dates, "with attachment".
2. Apply those filters to both searches, so the vector search never wanders outside the period or the people the question is about.
3. Merge the two result lists. Reciprocal rank fusion is simple and works well: score each message by the sum of 1 / (k + rank) across both lists.
4. Expand top hits to their threads, so the model reads conversations, not fragments.

Make the model show its work

Only then does the language model come in. Give it the retrieved messages, each with an ID, date and sender, and instruct it to answer only from them and to cite the ID for every statement. In the interface, turn each citation into a link that opens the original email.

Citations do two jobs. The user can check every claim in a click, which is the only responsible way to use a model on business records. And you can see immediately when retrieval failed: an answer citing the wrong messages is a retrieval problem, not a model problem, and it tells you where to look.

Tell users plainly that a model can misread. With sources one click away, that is a manageable risk.

Keep the data in the building

Every component in this design runs comfortably on your own hardware: the full text index on ordinary servers, the vector database next to it, and the embedding model and language model on GPU servers you control. The mail, the questions and the answers never need to leave your network.

Test it like a system, not a demo

Before trusting it, build a small set of questions with known answers: a specific invoice, the first email in a known dispute, a message written in your second language, a message that was deleted last month (it should not be found). Run them after every change to cleaning, indexing or prompts, and keep the results. If a change makes known questions worse, you will see it.

And watch the pipeline itself. A sync that stopped quietly, an index that has not caught up, or a vector backlog after a reboot all look like "search works" until someone needs last week's mail. Check the whole chain end to end, on a schedule, and alert when any stage falls behind.

In short

• Deduplicate, thread, decode and strip quoted text before indexing anything.
• Index attachments; that is where the contracts are.
• Run keyword and vector search together, with filters applied to both.
• Answer only from retrieved mail, with a citation on every statement.
• Keep every component on hardware you control, and test with known questions.

We build private mail archives like this, with the AI running on your own private GPUs. If you are planning one, we are happy to help.

Talk to an engineer

Related: Private mail archive with local AI search · Managed office documents on Nextcloud