5 New Analytics Features to Accelerate Insights and Automate Decision-Making

Adam Hughes
June 7, 2022 | Updated: September 9, 2024
>> Announcement: Some features mentioned below will be deprecated on Sep. 30, 2025. Learn more.

The applications we use every day are continually delivering richer experiences and working more efficiently. One of the driving forces of this progress is analytics. As organizations ingest and use ever increasing layers of data, they are able to derive more timely insights about their users’ preferences, patterns, and needs to deliver just-in-time information and choices within their applications.

The next generation of applications will take a huge leap in intelligence by integrating real-time analytics into their app experiences. Such analytics will increasingly be automated, developer-driven, and incorporated seamlessly within data platforms alongside transactional — or application workloads.

As announced at MongoDB World 2022, MongoDB will introduce five new features this year that will help businesses modernize their analytics: Column Store indexes, MongoDB Atlas Data Federation, MongoDB Atlas Data Lake, MongoDB Atlas SQL Interface, and distinct tiering for analytics nodes.

Using these features will automate decision-making and drastically decrease the time it takes to get application insights in front of users.

Modernizing analytics around operational data

Today, in order to create dynamic in-app experiences, businesses need to take multiple steps — collecting application data, sending it to a data warehouse or data lake to run analytics on it, deriving insights, coding new experiences, and releasing the app back to users.

Modern applications must be able to automate this process by capturing and processing the data at the source — that is, in the application. The data inside your application is the most valuable and current picture of what is happening with your business.

Combining real-time, operational, and embedded analytics — what some call translytics, HTAP, or augmented transaction databases — now enables analytics driven by application data to help determine, influence, and automate decision-making for the app and provide real-time insights for the user.

Real-time analytics is, as the name implies, done nearly instantly, usually on data that resides in an application. Examples include fraud detection for banks and personalized offers or recommendations on an e-commerce site. The analytics can range from basic aggregations to machine learning models that provide insight and automate an action, such as sending an offer. One example is Ticketek, an Australia-based event ticketing company, which uses real-time analytics to make critical decisions, such as whether to open up more sections of a venue or put on more shows.

Operational analytics is the process of finding insights from your data sources to improve decision-making for the daily operations of a business. Use cases include real-time reporting, improving overall operations, and product analytics. Online grocery Boxed, for example, was able to manage inventory levels during peak demand thanks to real-time data and insights directly from MongoDB Atlas.

Embedded analytics enhances applications by embedding data visualizations and dashboards with MongoDB Atlas Charts, providing users with relevant insights when and where they need them.

What's New

Here are five advances announced at MongoDB World that can help businesses modernize their analytics:

Column Store indexes: This feature enhances analytical queries by allowing developers to deliver real-time analytics on live, operational data. It also improves the performance of common analytical queries by adding a structure on top of collections that groups similar fields together to speed up reads. This eliminates the need to offload analytics to disparate specialized systems and rely on complex and fragile ETL pipelines that ultimately slow down the time to gain insights.

Atlas Data Federation: Atlas Data Lake is relaunching as Atlas Data Federation to reflect our focus on the value of federation. MongoDB Atlas users have the ability to query several data sources at once.

Atlas Data Lake: The new Atlas Data Lake provides a cost-effective data store optimized for high-performance analytics on large volumes of data. Atlas Data Lake delivers analytical workload isolation, allowing you to perform complex, long-running, or large analytical queries without impacting your production application. Fully integrated as part of the MongoDB Atlas, Atlas Data Lake can be provisioned alongside your Atlas Database, making the ingestion and optimization of data simple, with no infrastructure to set up or manage.

Atlas SQL Interface, Connectors, and Drivers: Atlas’s new SQL capabilities allow people who mainly work in SQL tools, such as data analysts, to easily interact with Atlas data. Users can query Atlas data via a BI tool or SQL driver and are able to directly query live data and gain enhanced schema control.

Distinct tiering for analytics nodes: Users can choose an appropriately sized node tier dedicated to their analytics workload without needing to change the tier of the entire cluster. This can enhance the performance of your analytics workloads; you can provision only what you need if your analytical workload requirements are less than your transactional requirements.

Learn more about MongoDB World 2022 announcements at mongodb.com/new and in these stories:

← Previous

Announcing Atlas Data Federation and Atlas Data Lake

>> Announcement: Some features mentioned below will be deprecated on Sep. 30, 2025. Learn more . Two years ago, we released the first iteration of Atlas Data Lake . Since then, we’ve helped customers combine data from various storage layers to feed downstream systems. But after years spent studying our customers’ experiences, we realized we hadn’t gone far enough. To truly unleash the genius in all our developers, we needed to add an economical cloud object storage solution with a rich MQL query experience to the world of Atlas. Today, we’re thrilled to announce that our new Atlas Data Federation and Atlas Data Lake offerings do just that. We now offer two complementary services, Atlas Data Federation (our existing query service formerly known as Atlas Data Lake) and our new and improved Atlas Data Lake (a fully managed analytic-oriented storage service). Together, these services (both in preview) provide flexible and versatile options for querying and transforming data across storage services, as well as a MongoDB-native analytic storage solution. With these tools, you can query across multiple clusters, move data into self managed cloud object storage for consumption by downstream services, query a workload-isolated inexpensive copy of cluster data, compare your cluster data across different points in time, and much, much more. In hearing from our customers about their experiences with Atlas Data Lake, we learned where they have struggled, as well as the features they’ve been looking for us to provide. With this in mind, we decided to shift the name of our current query federation service to Atlas Data Federation to better align with how customers see the service and are getting value. We’ve seen many customers benefit from the flexibility of a federated query engine service, including querying data across multiple clusters, databases, and collections, as well as exporting data to third-party systems. We also saw where our customers were struggling with data lakes. We heard them ask for a fully managed storage solution so they could achieve all of their analytic goals within Atlas. Specifically, customers wanted scalable storage that would provide high query performance at a low cost. Our new Data Lake provides a high-performance analytic object storage solution, allowing customers to query historical data with no additional formatting or maintenance work needed on their end. How it works Atlas Data Federation encompasses our existing Data Lake functionality with several new affordances. It continues to deliver the same power that it always has, with increased performance and efficiency. The new Atlas Data Lake will now allow you to create Data Lake pipelines (based on your Atlas Cluster backup schedules) and fields on which you can optimize queries. The service takes the following steps: On the selected schedule, a copy of your collection will be extracted from your Atlas backup with no impact to your cluster. During extraction, we build partition indexes based on the contents of your documents and the fields you’ve selected for optimization. These indexes allow your queries to be optimized by capturing the minimums and maximums (and other stats) of the records in each partition, letting you quickly find the relevant data for your queries. Finally, the underlying data lands in an analytic-oriented format inside of cloud object storage. This minimizes data scanned when you execute a query. Once a pipeline has run and a Data Lake dataset has been created, you can select it as a data source in our new Data Federation query experience. You can either set it as the source for a specific virtual collection in a Federated Database Instance or you can have your Federated Database Instance generate a collection name for each dataset that your pipeline has created. Amazingly, no part of this process will consume compute resources from your cluster — neither the export nor the querying of datasets. These datasets provide workload isolation and consistency for long-running analytic queries, a target for ETL jobs using the powerful $out to S3. This makes it easy to compare the state of your data over time. Advanced though this is, it’s only the beginning of the story. We’re committing to evolving the service, improving performance, adding more sources of data, and building new features. All of this will be based on the feedback you, the user, gives us. We can’t wait to see how you’ll use this powerful new tool and can’t wait to hear what you’d like to see next. Try Atlas Data Lake Today

June 7, 2022
Next →

MongoDB Database Observability: Integrating with Monitoring Tools

This post is the final in a three-part series on leveraging database observability. Welcome back to our series on Leveraging Database Observability! Our previous post showcased a real-world use case highlighting how MongoDB Atlas’s observability tools effectively tackle database performance challenges. Whether you’re a developer, DBA, or DevOps engineer, our mission is to empower you to harness the full potential of your data through our observability suite . Integrating Atlas metrics with your central enterprise observability tools can simplify your operations. By seamlessly working with popular observability tools, our approach helps teams streamline workflows and enhance visibility across systems. Integrating MongoDB Atlas with third-party monitoring tools MongoDB’s developer data platform combines all essential data services for building modern applications within a unified experience. Our purpose-built observability tools for Atlas environments offer automatic monitoring and optimization, guiding diagnostics tailored specifically for MongoDB. Additionally, we extend Atlas metrics into your existing enterprise observability stack, enabling seamless integration without replacing your current tools. This creates a consolidated, single-pane view that unifies Atlas telemetry with other tech and application metrics, ensuring comprehensive visibility into both database and full-stack performance. This integration empowers you to monitor, receive alerts, and make data-driven decisions within your existing workflows, driving greater efficiency. Below is a quick guide to modifying integration settings through the Atlas UI and the popular integrations we support: Navigate to the Project Integrations page in Atlas. Choose the organization and project you want to configure from the navigation bar. On the Project Integrations page, select the third-party services you’d like to integrate. Configure the chosen services with the required API keys and regions. Critical integrations for your observability platform With Atlas’s Datadog and Prometheus integrations, you can send critical MongoDB metrics to these platforms, empowering detailed, real-time monitoring. Through Datadog , you can track database operation counts, query efficiency, and resource usage, ideal for pinpointing bottlenecks and managing resources. Similarly, Prometheus enables you to monitor essential metrics like query times, connection rates, and memory usage, supporting flexible tracking of database health and performance. Both integrations facilitate proactive detection of issues, alert configuration for resource thresholds, and a cohesive view of Atlas data when visualized in Grafana. Atlas’s integration with PagerDuty streamlines incident management by sending metrics like performance alerts, billing anomalies, and security events directly to PagerDuty. This integration records incidents automatically, notifies teams upon alerts, and supports two-way syncing, ensuring resolved alerts in Atlas are reflected in PagerDuty. It enables efficient incident response and resource allocation to maintain system stability. With Atlas integrations for Microsoft Teams and Slack, you can route key metrics—such as query latency, disk usage, and throughput—to these channels for timely updates. Teams can use these insights for real-time performance monitoring, incident response, and collaboration. Notifications through these platforms ensure your team stays informed on database performance, storage health, and user activity changes as they occur. Use case: Centralized observability with MongoDB Atlas, Datadog, and Slack Let’s walk through a hypothetical scenario for ShopSmart, an e-commerce company that leverages MongoDB Atlas to manage its product catalog and customer data. As traffic surges, the DevOps team faces challenges in monitoring application performance and database health effectively. To tackle these challenges, the team leverages MongoDB Atlas’ integration with Datadog and Slack, creating a powerful observability ecosystem. Integrating MongoDB Atlas with Datadog: The team pushes key MongoDB Atlas metrics into Datadog, such as query performance, connection counts, and Atlas Vector Search metrics. With Datadog, they can visualize these metrics and correlate overall MongoDB performance with their other applications. Out-of-the-box monitors and dedicated dashboards allow the team to track metrics like throughput, average read/write latency, and current connections. This visibility helps pinpoint bottlenecks in real time, ensuring optimal database performance and improving overall application responsiveness. Setting up alerts in Datadog: The team configures alerts for critical metrics like high query latency and increased error rates. When thresholds are breached, Datadog instantly notifies the team. This proactive approach allows the team to address potential performance issues before they impact customers. Integrating Datadog with Slack: To ensure fast communication, alerts are sent directly to the dedicated Slack channel, “ShopSmart-Alerts.” This integration fosters seamless collaboration, enabling the team to discuss and resolve issues in real-time. With these integrations, ShopSmart’s engineering team can monitor performance quickly and address issues efficiently. The unified observability approach enhances operational efficiency, improves the customer experience, and supports ShopSmart’s competitive edge in the e-commerce industry. By leveraging MongoDB Atlas, Datadog, and Slack, the team ensures scalable performance and drives continuous innovation. Conclusion MongoDB Atlas empowers developers and organizations to achieve unparalleled observability and control over their database environments. By seamlessly integrating with central enterprise observability tools, Atlas enhances your ability to monitor performance metrics and ensures you can do so within your existing workflows. This means you can focus on building modern applications confidently, knowing you have the insights and alerts necessary to maintain optimal performance. Embrace the power of MongoDB Atlas and transform your approach to database management—because your applications can thrive when your data is observable. And that wraps up our Leveraging Database Observability series! We hope you learned something new and found value in these discussions. Sign up for MongoDB Atlas , our cloud database service, to see database observability in action. To dive deeper and expand your knowledge, check out this learning byte for more insights on the MongoDB observability suite and how it can enhance your database performance.

November 14, 2024