Vector Databases Explained Without the Hype

Vector databases have become one of the most hyped infrastructure categories in AI, with dozens of startups and established companies competing for attention. But beneath the marketing, the core concept is straightforward, and understanding it helps you make better decisions about whether and how to use one.

What Vector Databases Actually Do

A vector database stores and efficiently searches high-dimensional vectors. That is it. The vectors themselves are typically embeddings — numerical representations of text, images, or other data produced by machine learning models. The database job is to find which stored vectors are most similar to a query vector.

Traditional databases excel at exact matches: find the row where id equals 42, or where name equals “Smith.” Vector databases excel at similarity matches: find the ten vectors most similar to this query vector, where similarity is measured by cosine distance, dot product, or Euclidean distance in high-dimensional space.

Why They Exist

The naive approach to similarity search — comparing your query vector against every stored vector — works fine for small datasets. With a thousand vectors of 1536 dimensions, brute-force search takes milliseconds. But with a million vectors, it takes seconds. With a billion vectors, it is impractical.

Vector databases use approximate nearest neighbor (ANN) algorithms to make similarity search fast at scale. They trade a small amount of accuracy (you might miss the absolute nearest neighbor occasionally) for dramatic speed improvements (milliseconds instead of seconds for million-vector datasets).

The two dominant ANN algorithms are:

HNSW (Hierarchical Navigable Small World): Builds a graph structure where similar vectors are connected. Search navigates this graph from a random entry point toward the query vector. Fast queries, higher memory usage, good for datasets that fit in RAM.

IVF (Inverted File Index): Clusters vectors into groups and only searches the most relevant clusters for each query. Lower memory usage, slightly slower queries, better for very large datasets.

When You Need One

You need a vector database when:

• You are building RAG and need to retrieve relevant documents from a large knowledge base
• You are building semantic search (finding similar items by meaning rather than keywords)
• You are building recommendation systems based on content similarity
• You have more than roughly 100,000 vectors and need sub-second query times

You probably do not need a dedicated vector database when:

• Your dataset is small (under 100K vectors) — pgvector or even brute-force search works fine
• You only need keyword search — traditional full-text search is simpler and often better
• You are prototyping — start with the simplest option and add complexity when scale demands it

The Options

pgvector (PostgreSQL Extension)

Adds vector similarity search to your existing PostgreSQL database. No new infrastructure, familiar SQL interface, supports both exact and approximate search.

Best for: teams already using PostgreSQL, datasets under a few million vectors, applications where you need to combine vector search with traditional SQL queries (filtering by metadata, joining with other tables).

Limitations: performance degrades at very large scale compared to purpose-built solutions, limited to what PostgreSQL can handle in terms of concurrent queries.

Pinecone (Managed Service)

Fully managed vector database with a simple API. No infrastructure to manage, automatic scaling, built-in filtering and metadata support.

Best for: teams that want zero infrastructure management, applications that need to scale without engineering effort, rapid prototyping that might grow to production.

Limitations: vendor lock-in, cost can be high at scale, less control over performance tuning.

Weaviate (Open Source)

Feature-rich open-source vector database with built-in vectorization, hybrid search (vector + keyword), and a GraphQL API.

Best for: teams that want open-source with rich features, applications needing hybrid search, teams comfortable managing their own infrastructure.

Limitations: more complex to operate than managed services, resource-intensive for large deployments.

Qdrant (Open Source)

High-performance open-source vector database written in Rust. Focuses on speed and efficiency with a clean REST API.

Best for: performance-sensitive applications, teams that want open-source with excellent performance characteristics, deployments where resource efficiency matters.

Limitations: smaller ecosystem than some competitors, fewer built-in integrations.

Chroma (Open Source)

Lightweight, developer-friendly vector database designed for AI applications. Runs in-process or as a server.

Best for: local development and prototyping, small to medium datasets, developers who want the simplest possible setup.

Limitations: not designed for very large scale, fewer production features than mature alternatives.

Choosing the Right Option

The decision tree is simpler than vendors want you to believe:

1. Already using PostgreSQL and dataset under 5M vectors? Use pgvector.
2. Want zero infrastructure management and budget allows? Use Pinecone.
3. Need open-source with rich features? Use Weaviate or Qdrant.
4. Just prototyping? Use Chroma or pgvector.
5. Need maximum performance at massive scale? Evaluate Qdrant, Weaviate, and Milvus with your specific workload.

Performance Considerations

The factors that most affect vector database performance:

Dimensionality: Higher-dimensional vectors (1536 vs 384) require more storage and slower search. Use the minimum dimensions that maintain quality for your use case.

Index type: HNSW is faster for queries but uses more memory. IVF uses less memory but queries are slightly slower. Choose based on your memory budget and latency requirements.

Filtering: If you need to filter results by metadata (only search documents from a specific category), the database ability to combine vector search with metadata filtering matters enormously. Some databases filter before search (fast but might miss results), others filter after (accurate but slower).

Update frequency: If your data changes frequently, consider how the database handles index updates. Some require periodic re-indexing, others handle real-time updates efficiently.

• Vector databases enable fast similarity search over high-dimensional embeddings using ANN algorithms
• pgvector is sufficient for most teams starting out — no new infrastructure needed
• You probably do not need a dedicated vector database until you exceed 100K-1M vectors
• Choose based on scale, infrastructure preferences, and whether you need managed vs self-hosted
• Dimensionality, index type, and filtering strategy are the key performance levers

The vector database market is noisy, but the underlying technology is well-understood. Start with the simplest option that meets your current needs, and migrate to something more specialized only when you have concrete evidence that your current solution is insufficient.