We are excited to announce the launch of Lexical Prefilters for MongoDB Vector Search, providing developers with a way to use advanced text and geo analysis filters alongside vector search. This new capability simplifies the challenge of building advanced applications by helping developers build richer and more sophisticated search experiences through the combination of semantic understanding and precise text filtering.
What are Lexical Prefilters for Vector Search?
Lexical Prefilters for Vector Search are implemented via new syntax that enables lexical (analyzed text) prefilters for vector search operations. This capability is accessed using the vectorSearch operator within the $search aggregation stage.
Developers can now use advanced text and geo analysis filters—such as fuzzy search, phrase matching, wildcards, and geoWithin—as prefilters for vector similarity searches.
To leverage this functionality, users must define a new vector data type field within their $search index definitions. This allows developers to use text analysis capabilities to dramatically narrow down the results before the vector search calculation is performed, resulting in better performance (due to searching a smaller set of fewer vectors), greater precision (hard requirements are not enforced before the ranking, not after), and cost (because less compute per query results in lower cost).
Unlike standard filters in $vectorSearch, which only support basic operators (like equals, range, or exists), Lexical Prefilters add full analyzed text capabilities such as fuzzy, phrase, wildcard, and queryString.
Demonstration of Lexical Prefilters for MongoDB Vector Search
The power of precision
Combining semantic understanding with precise text filtering is essential for advanced search use cases. Lexical prefilters solve a number of challenges for companies building advanced applications, including:
Supporting advanced filtering: MongoDB now provides a way to use advanced text and geospatial filters with vector search. This capability eliminates the need for the knnBeta operator and knnVector field type, allowing users to move to fully supported features.
Increased efficiency: Performing a vector search on a massive dataset is computationally expensive. Lexical prefilters improve search performance by narrowing down the dataset prior to an extensive search being performed, processing fewer candidates which results in a more optimized use of resources.
Improving performance: Analyzed text prefilters can improve performance compared to using post-filtering after the vector search is complete.
Enabling complex logic: Developers can now build more complex filtering logic that search engineers are accustomed to, such as complex boolean logic using the queryString operator.
Together, this enables developers to more easily build advanced semantic search experiences and RAG applications that leverage keyword filtering, fuzzy text matching, geospatial filtering, and other hard filters alongside semantic understanding for robust hybrid search.
Important migration note for existing users: The knnVector field type and the $search.knnBeta operator are being deprecated. Customers using these must migrate to the new vector field type and the $search.vectorSearch operator. For those planning their migration, please see our query or index docs for more on beginning your migration.
Getting Started
This capability is available to all MongoDB Atlas users across AWS, Google Cloud, and Microsoft Azure, and is included in standard pricing.
To begin using Lexical Prefilters, create a $search index with vector type fields and use $search.vectorSearch in your aggregation pipelines, applying lexical filters using any Atlas Search operator.