Vector Databases, Explained (Searching Beyond Keywords)

Traditional databases are excellent at finding information when you know exactly what you're looking for. Give a database a customer ID, email address, or order number, and it can retrieve the corresponding record efficiently. But searching for information when you don't know the exact terminology can be a different challenge.

Imagine searching your company's internal documentation for "Why is my application responding slowly?" The relevant article might be titled "Troubleshooting High API Latency." Both describe a similar problem, but a basic keyword search might struggle to connect them.

Vector search offers another approach by searching for information based on similarity rather than relying entirely on matching words.

The Basics

A vector database stores and searches numerical representations of information called embeddings. An embedding is a collection of numbers generated by a machine learning model that represents certain characteristics of text, images, or other data.

When two pieces of information have similar meanings, their embeddings will often be positioned relatively close to each other in a mathematical space. For example, "database connection timeout" and "unable to connect to PostgreSQL" might produce similar embeddings even though the wording is different.

When a user submits a query, the application generates an embedding for that query and compares it against stored embeddings. The database identifies nearby vectors using similarity measures such as cosine similarity or distance calculations, returning the closest matches.

Why It Exists

Traditional keyword search works well when users know the terminology used in the underlying data. Modern search engines can also handle synonyms, variations in wording, and ranking, but semantic similarity provides another way to retrieve relevant information.

Consider a customer support knowledge base containing thousands of troubleshooting articles. Customers rarely describe technical problems using the same terminology as the engineers who wrote the documentation. Someone might search for "website keeps crashing" when the relevant article discusses application memory exhaustion. Vector search can help connect those descriptions because it considers relationships represented by embeddings rather than just the words themselves.

This makes vector search useful for semantic search, recommendation systems, document retrieval, and Retrieval-Augmented Generation (RAG), where applications need to identify relevant information before sending it to a language model.

Common Pitfalls

Assuming similarity means accuracy: Two pieces of text can be semantically similar without containing the same answer.
Choosing embeddings without testing: Different embedding models can produce different search results.
Ignoring metadata: Semantic similarity alone may not account for document dates, customer permissions, or other important filters.
Poor document chunking: Large or poorly divided documents can reduce the relevance of search results.
Replacing traditional databases unnecessarily: Vector search complements relational databases and keyword search rather than replacing them.

Why It Matters

Vector databases have become an important component of AI application infrastructure, particularly for applications using RAG. However, implementing vector search doesn't always require deploying an entirely new database. PostgreSQL, for example, supports vector similarity search through the pgvector extension, allowing applications to store embeddings alongside relational data and combine traditional SQL queries with vector search.

Dedicated vector databases such as Qdrant and Milvus offer specialized capabilities for managing and searching large collections of embeddings. These systems can use approximate nearest-neighbor indexing techniques to improve search performance, although approximate search can trade some retrieval accuracy for speed.

Another option is hybrid search, which combines vector similarity with traditional keyword-based retrieval. This can be particularly useful when queries contain both conceptual language and exact technical terms, such as error codes or product names.

The TAM Lens

Choosing the right database architecture depends on the application's scale and existing infrastructure. For an internal knowledge assistant searching a relatively small collection of documents, PostgreSQL with pgvector may be sufficient, particularly if the application already uses PostgreSQL for other data. Introducing another database would also mean another service to deploy, monitor, secure, and maintain.

Larger applications may have different requirements. A platform searching millions of embeddings across multiple customers needs to consider indexing performance, query concurrency, metadata filtering, and tenant isolation.

Storage costs are also worth evaluating. Every document chunk produces an embedding, and those embeddings consume space alongside metadata and search indexes. The number of vectors, their dimensions, and the indexing strategy all influence resource consumption. Performance testing with representative workloads is more useful than choosing a technology based solely on its advertised capabilities.

How to Stay Sane

Evaluate existing infrastructure: Check whether your current database can support vector search before introducing another service.
Test embedding models: Compare retrieval results using queries representative of your application.
Use metadata filtering: Combine semantic similarity with structured attributes such as tenant IDs, document types, and timestamps.
Consider hybrid search: Keyword matching can improve results for exact terms that semantic search may overlook.
Monitor index growth: Account for embeddings, metadata, and indexing overhead as document collections expand.
Measure retrieval quality: Evaluate whether search results are actually relevant, not just how quickly they're returned.

Final Thoughts

Vector databases allow applications to search for information based on similarity rather than relying entirely on exact keywords or structured identifiers. They're particularly useful in AI applications that need to retrieve relevant information from large collections of documents, but they aren't necessarily a replacement for existing database technologies.

For many applications, adding vector search capabilities to an existing database may be all that's required. The right approach depends on the volume of data, the complexity of the queries, and how the application is expected to grow.