TL;DR: A vector database stores the embeddings that power retrieval in a RAG system. Choose based on scale (millions vs billions of vectors), where it must run (managed cloud vs self-hosted for compliance), filtering needs, and cost. Start simple with pgvector if you're on Postgres; move to a dedicated vector DB when scale or features demand it.
For most teams, the best vector database is the one that fits your scale, stack and compliance — not the most hyped name. If you already run Postgres, pgvector is often the pragmatic starting point. For large scale or advanced filtering, Qdrant, Weaviate, Milvus or Pinecone earn their place. The right choice depends on data size, latency targets and whether data must stay in your own perimeter.
What is a vector database and why does RAG need one?
A vector database stores text as numerical embeddings and finds the passages most similar in meaning to a query. RAG uses it to retrieve the right context to feed the LLM. Without fast, accurate vector search, retrieval quality drops — and in RAG, retrieval quality is answer quality.
How do you choose the right vector database?
Weigh five factors:
- Scale — thousands, millions or billions of vectors? Some tools shine only at scale.
- Deployment — managed SaaS (less ops) vs self-hosted (required when data can't leave your environment).
- Metadata filtering — can it filter by tenant, permission or date efficiently? Critical for multi-tenant and access-controlled RAG.
- Hybrid search — does it combine vector and keyword search natively?
- Cost and ops burden — managed services trade money for fewer headaches.
A practical comparison
| Option | Sweet spot | Deployment | Notes |
|---|---|---|---|
| pgvector (Postgres) | Teams already on Postgres, small–mid scale | Self-host or managed | Simplest; one fewer system to run |
| Qdrant | Strong filtering, performance, self-hosting | Self-host or cloud | Popular for compliance-sensitive builds |
| Weaviate | Hybrid search, modules | Self-host or cloud | Good developer ergonomics |
| Milvus | Very large scale | Self-host or cloud | Built for billions of vectors |
| Pinecone | Fully managed, minimal ops | Managed only | Fast to start; data leaves your perimeter |
When is pgvector enough — and when should you upgrade?
pgvector is enough when you're already on Postgres, your corpus is in the thousands-to-low-millions of chunks, and you want one less system to operate. Upgrade to a dedicated vector DB when you hit very large scale, need advanced metadata filtering at speed, or require performance features pgvector can't match. Don't add infrastructure you don't need yet — it's easy to migrate later.
What about compliance and data residency?
If you're in a regulated industry, the deciding factor may not be features at all — it's whether the vectors (which encode your content) can leave your environment. In that case, choose a self-hostable option like pgvector, Qdrant, Weaviate or Milvus and run it inside your own cloud or on-prem perimeter, rather than a managed-only service.
