Encrypting documents and leaving the vector index in the clear protects the copy nobody attacks and exposes the one they do.
ConceptWhat it is
Encryption at rest and in transit is the least controversial control in security, and in an AI system it is routinely applied to the wrong half of the estate. The source documents get encrypted because they look like documents. The vector index does not, because it looks like numbers.
An embedding is a lossy but far from empty representation of its text. Published inversion work recovers substantial parts of short passages from their vectors alone. Treating the index as derived, non-sensitive data is the mistake this page exists to name.
How it worksThe mechanics
Three surfaces need covering and they are usually owned by different people. The object store holding source documents, the vector index holding embeddings and the chunk text beside them, and every hop between services — including the call to the model provider, which leaves your network entirely.
At rest, that means provider-managed or customer-managed keys on the buckets and on the index, with the choice driven by who must be able to prove they can destroy the key. In transit it means TLS everywhere, which is mostly free, plus a clear-eyed decision about what leaves for inference and under what contractual terms it is retained.
At a glanceSee it
Three surfaces, not one. The vector index is the copy most often left in the clear, and it is a recoverable representation of the text it came from.
When to use itWhere it fits
- Always, for anything not already public — this is a baseline control, not a design choice.
- Customer-managed keys specifically, when a contract or regime requires you to prove you can render data unrecoverable.
- Whenever a vector index is hosted by a third party, where it is a separate trust boundary from your document store.
- Before a security review, because the vector index is the question a good reviewer asks and a weak one misses.
When NOT to use itLimits & anti-patterns
- As evidence of privacy compliance — encryption protects against the wrong party reading data, not against your own system using it in ways consent never covered.
- As a reason to skip access control; an encrypted index the application decrypts for everyone is an index everyone can read.
- Client-side encryption of chunk text you intend to search semantically, which simply does not work — the search needs the plaintext vectors.
- As a substitute for deletion. An encrypted record you were required to erase is still a record you kept.
Trade-offsAdvantages & costs
Advantages
- Cheap, well understood, and largely handled by the platform rather than by your code.
- Customer-managed keys turn deletion into a key operation, which is the only practical answer to erasure across backups.
- Closes the most common finding in a vendor security questionnaire before it is raised.
- Protects against the realistic threat — a misconfigured bucket or a stolen snapshot — rather than a theoretical one.
Trade-offs & costs
- Gives a strong feeling of safety that does not extend to any of the AI-specific risks on this tile.
- Customer-managed keys add an operational failure mode: lose the key and the corpus is gone, which is the point and also the hazard.
- The provider hop is the one surface encryption cannot solve — that is a contract and a data-residency question.
- Encrypted-at-rest vector stores can carry a measurable indexing cost, which shows up as slower rebuilds rather than slower queries.
ExampleIn the real world
A clinical-notes assistant encrypts its document bucket with customer-managed keys and runs its vector index on a managed service left at provider defaults. The threat model on paper says the notes are protected. In practice the index holds chunk text beside the vectors, in a different account, under a key the team cannot destroy — so the erasure promise in the contract cannot be kept for the copy that is easiest to query.
ToolsHow to implement it
- AWS KMS, GCP Cloud KMS or Azure Key Vaultcustomer-managed keys where proof of destruction is required.
- pgvector inside an encrypted Postgreskeeps the index under the same key and the same backup policy as the rest of the data.
- TLS everywhere by defaultservice meshes and managed load balancers make in-transit the cheap part; the work is auditing that nothing opted out.
- A zero-retention or enterprise agreement with the model providerthe only control over the hop that leaves your network.
Cost & effortWhat it takes
Near zero at the margin for at-rest and in-transit; managed keys carry a small per-key and per-operation charge that is noise against inference spend. Effort is a day or two of platform configuration plus an audit pass, and the recurring cost is key rotation. The expensive item is not encryption — it is discovering late that the index needed the same treatment and re-hosting it.