100x Faster Facets and Counts with MongoDB Atlas Search: Public Preview

Mat Keep and Marcus Eagan
November 9, 2021 | Updated: June 3, 2022
#Atlas Search

Today we’ve released one of the most powerful features of Atlas Search in public preview, and ready for your evaluation: lightning fast facets and counts over large data sets.

Faceted search allows users to filter and quickly navigate search results by categories and see the total number of results per category for at-a-glance statistics.

With the new facet operator, facet and count operations are pushed down into Atlas Search’s embedded Lucene index and processed locally – taking advantage of 20+ years of Lucene optimizations – before returning the faceted result set back to the application.

What this means is that now facet-heavy workloads such as ecommerce product catalogs, content libraries, and counts run up to 100x faster.

The power of facets and counts in full-text search

Faceting is a popular search and analytics capability that allows an application to group information into related categories by applying filters to query results. Users can narrow their search results by simply selecting a facet value as a filter criteria. They can intuitively explore complex data sets, providing fast and convenient navigation to quickly drill into the data that is of most interest.

A common use of faceting is navigating product catalogs. With travel starting to reopen, let's take a travel site as an example. By using faceted search, the site can present vacation options by destination region, trip type (i.e. hotel, self-catering, beach, ski, city break), price band, season, and more, enabling users to quickly navigate to the category that is most relevant to them.

Facets also enable fast results counting. Extending our travel site example, business analysts can use facets to quickly compare sales statistics by counting the number of trips sold by region and season.

Prior to the new facet operator, the only way Atlas Search could facet and count data was to retrieve the entire result set back to MongoDB’s internal $facet aggregation pipeline stage. While that was OK for smaller data sets, it became slow when the result set exceeded tens of thousands of documents.

This all changes as now operations are pushed down to Atlas Search’s embedded and optimized Lucene library in a single $search pipeline stage. From our internal testing of a collection with one million documents, the new Atlas Search faceting improves performance by 100x.

How to use faceting in Atlas Search

Our new Atlas Search facets tutorial will help you get started. It describes how to:

Create an index with a facet definition on string, date, and numeric fields in the sample_mflix.movies collection.
Then run an Atlas Search query against those fields for results grouped by values for the string field and by ranges for the date and numeric fields, including the count for each of those groups.

To use Atlas Search facets, you must be running your Atlas cluster on MongoDB 4.4.11 and above or MongoDB 5.0.4 and above. These clusters must be running on the M10 tier or higher. Facets and counts currently work on non-sharded collections. Support for sharded collections is scheduled for next year.

The power of Atlas Search in a unified data platform in the cloud

MongoDB Atlas Search makes it easy to build fast, relevant full-text search on top of your data in the cloud. A couple of API calls or clicks in the Atlas UI, and you instantly expose your data to sophisticated search experiences that boost engagement and improve satisfaction with your applications. Your data is immediately more discoverable, usable, and valuable.

By embedding the Apache Lucene library directly alongside your database, data is automatically synchronized with the search index; developers get to work with a single API; there is no separate system to run and pay for; and everything is fully-managed for you on any cloud you choose.

Atlas Search provides a fully-integrated full-text search engine, automatically synchronized with your database — ***Figure 1: Rather than bolting-on a separate search engine to your database, Atlas Search provides a fully integrated platform.***

Atlas Search provides the power you get with Lucene — including faceted navigation, autocomplete, fuzzy search, built-in analyzers, highlighting, custom scoring, and synonyms — combining it with the productivity you get fromMongoDB. As a result, developers can ship search applications and new features 30%+ faster.

Next steps

You can try out Atlas Search with the public preview of lightning-fast facets and counts today:

If you are new to Atlas Search, simply spin up a cluster (M10 tier or above) and get started with our Atlas Search facets tutorial.
If you are already using Atlas Search on M10 tiers and above then update your indexes to use the facet field mapping, and then start querying! Your data remains searchable while it is being re-indexed.

If you want to dig into the use cases you can serve with Atlas Search — along with users who are already taking advantage of it today — download our new Atlas Search whitepaper.

Safe Harbor

The development, release, and timing of any features or functionality described for our products remains at our sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality.

← Previous

Sync or Stink: Why Some Apps Can't Handle Mobile Data

>> Announcement: Some features mentioned below will be deprecated on Sep. 30, 2025. Learn more . Mobile apps have transformed the way businesses work with data. For example, by automating and digitizing what were previously manual and error-prone tasks, mobile apps have made frontline workers more productive and the operational data they generate more actionable. But that is only true when those workers have modern mobile apps that are fully integrated with back-end systems at the core of the network. Most mobile apps are often slow to load and sync data unreliably, frequently returning stale data or just outright crashing. The reason for this is that data sync is hard. A lot of developer teams take a DIY approach to building sync. But building a sync tool that works in an offline-first environment, like mobile, can take months of complex work and require thousands of lines of code. And when development teams build mobile sync themselves, they often oversimplify the solution — so app data may only sync a few times a day or won’t sync bidirectionally. Building sync the right way — keeping data up-to-date in real-time whenever devices are connected — requires complicated networking and conflict resolution code. MongoDB Realm eliminates this complexity We cover many of the technical difficulties of developing sync tools, including the pitfalls of DIY approaches, in our recent white paper, “ Unlocking Value in Mobile Apps: 3 Best Practices for Mobile Data Sync .” Data sync is absolutely essential for delivering good customer experiences, harnessing the value of mobile data for the business, and ultimately proving that investments in mobile technology are worth it. From getting real-time information to frontline workers to tracking fleets and getting timely inventory updates, mobile data has the potential to deliver true competitive advantage, but only if you’re working with a highly performant mobile platform. When apps work right, users reward you by using the app and, hopefully, leaving positive feedback. Another good sign of a well-built app is that users ask for new features. Development teams that don’t have to worry about complicated conflict resolution code and an overly complex app architecture can focus on quickly developing new features and, in turn, creating better app experiences. MongoDB Realm and Realm Sync include pre-built conflict resolution out of the box. The Realm Mobile Database — used alongside Realm Sync — is object-oriented, so it’s intuitive to mobile developers. Realm enables developers to focus on delivering competitive differentiation instead of worrying about building complicated data sync and conflict resolution tools. This speeds development and enables teams to turn around feature requests faster. Realm Sync works alongside Realm Mobile Database to synchronize data bi-directionally between the Realm Mobile Database on the client and MongoDB Atlas on the back end. Find out more by reading “ Unlocking Value in Mobile Apps: 3 Best Practices for Mobile Data Sync .”

November 3, 2021

Next →

Hanabi Technologies Uses MongoDB to Power AI Assistant, Hana

For all the hype surrounding generative AI, cynics tend to view the few real-world implementations as little more than “fancy chatbots.” But for Abhinav Aggarwal, CEO of Hanabi Technologies, the idea of a generative AI-powered bot that is more than just an assistant was intriguing. “I’d been using ChatGPT since it launched,” said Aggarwal. “That got me thinking: How could we make a chatbot that was like a team member?” And with that concept, Hana was born. The problem with bots “Most generative AI chatbots do not act like people; they wait for a command and give a response,” said Aggarwal. “We wanted to create a human-like chatbot that would proactively help people based on what they wanted—automating reminders, for example, or fetching time zones from your calendar to correctly schedule meetings.” Hanabi’s flagship product, Hana, is an AI assistant designed to enhance team collaboration within Google Chat, working in concert with Google Workspace and its suite of products. “Our target customers are smaller companies of between 10 and 50 people. At this size you’re not going to build your own agent from scratch,” he said. Hana integrates with Google APIs to deliver a human-like assistant that chimes in with helpful interventions, such as automatically setting reminders and making sure meetings are booked in the right time zone for each participant. “Hana is designed to bring AI to smaller companies and help them collaborate in a space where they are already working—Google Workspace,” Aggarwal explained. The MongoDB Atlas solution For Hana to act like a member of the team, Hanabi needed to process massive amounts of data to support advanced features like retrieval-augmented generation (RAG) for better information retrieval across Google Docs and many other sources. And with a rapidly growing user base of over 600 organizations and 17,000+ installs, Hanabi also required a secure, scalable, and high-performing data storage solution. MongoDB Atlas provided a flexible document model, built-in vector database, and scalable cloud-based infrastructure, freeing Hanabi engineers to build new features for Hana rather than focusing on rote tasks like data extract, transform, and load processes or manual scaling and provisioning. Now, MongoDB Atlas handles a variety of responsibilities: Scalability and security: MongoDB Atlas’s auto-scaling and automatic backup features have enabled Hanabi to seamlessly grow its user base without the need for manual database management. RAG: MongoDB Atlas plays a critical role in Hana’s RAG functionality. The platform enables Hanabi to split Google Docs into small sections, create embeddings, and store these sections in Atlas’s vector database. Development Processes: According to Aggarwal, MongoDB’s flexibility in managing changing schemas has been essential to the company’s fast-paced development cycle. Data Visualization: Using MongoDB Atlas Charts has enabled Hanabi to create comprehensive dashboards for real-time data visualization. This has helped the team track usage, set reminders, and optimize performance without needing to build a manual dashboard. Impact and results With MongoDB Atlas, Hanabi can successfully scale Hana to meet the demands of its rapidly expanding user base. The integration is also enabling Hana to offer powerful features like automatic interactions with customers, advanced information retrieval from Google Docs, and manually added memory snippets, making it an essential tool for teams around the world. Next steps Hanabi plans to continue integrating more tools into Hana while expanding its reach to personal Gmail users. The company is also rolling out a new automatic-interaction feature, further enhancing Hana’s ability to proactively assist users without direct commands. MongoDB Atlas remains a key component of Hanabi’s stack, alongside Google Kubernetes Engine, NestJS, and LangChain, enabling Hanabi to focus on innovating to improve the customer experience. Tech Stack MongoDB Atlas Google Kubernetes Engine NestJS LangChain Are you building AI apps? Join the MongoDB AI Innovators Program today! Successful participants gain access to free MongoDB Atlas credits, technical enablement, and invaluable connections within the broader AI ecosystem. If your company is interested in being featured, we’d love to hear from you. Connect with us at ai_adopters@mongodb.com.

November 21, 2024