Closing the Developer Experience Gap: MongoDB World Announcements

Steve Jurczak
June 7, 2022 | Updated: September 9, 2024

>> Announcement: Some features mentioned below will be deprecated on Sep. 30, 2025. Learn more.

Now is a great time to be a software developer or architect. Never have there been so many solutions, vendors, and architectural patterns to choose from as you build new applications and features. But the sheer number of choices creates another puzzle for developers to solve before they can begin to build.

Many of MongoDB’s efforts over the past year have been to help address the needs of the developer communities we serve, and one of the greatest needs we’ve seen in developer communities is improving the experience of being a developer.

At MongoDB World 2022, we announced several tools to help improve that experience and to boost developer velocity:

  • Atlas Data API — A serverless API that lets you easily access your Atlas data from any environment that supports HTTPS requests, including services like AWS Lambda and Google App Services. The Atlas Data API is fully functional upon generation, language-agnostic, and secure from the start.

  • Serverless instances — With MongoDB serverless instances, developers don’t have to worry about scaling up to meet increasing workloads or paying for resources they’re not using if their workload is idle. The serverless model dynamically uses only what it needs — and only charges for what it uses.

  • Atlas CLI — The MongoDB Atlas CLI is a completely new way to access Atlas in a non-GUI-centered environment. CLIs are often the interaction method of choice by developers, especially advanced developers who prefer control and speed over a more visual interface. Our new CLI gives these developers an easier registration experience with nearly instant free tier deployments in Atlas.

  • Time series — We have expanded our data platform so developers can work more easily with time series data in support of IoT use cases, financial analytics, logistics, and more. MongoDB time series makes it faster and lower cost to build and run time series applications by natively supporting the entire time series data lifecycle.

  • Facets in Atlas Search — Categorize data with facets for fast, filtered search results. With facets in Atlas Search, you can index your data to map fields to categories, then quickly update query results based on the ones relevant to your users.

  • Verified Solutions — The MongoDB Verified Solutions program gives developers the confidence to use third-party tools, such as Mongoose, by guaranteeing comprehensive testing of the tools as well as a base level of support from MongoDB Technical Services.

  • Change streams — Change streams enable developers to build real-time, event-driven applications that react to data changes as they happen. This allows them to build more complex features and better end-user experiences.

The paradox of choice for developers

Developers today have no shortage of tools to work with, but the abundance of options is itself a problem. And when there’s little or no central decision-making, developers are forced to figure out how to stitch together a patchwork of technology solutions to create the seamless user experiences that consumers have come to expect.

Developers had fewer choices when applications were built on a three-tier framework composed of a relational database, a J2EE stack, and an app or web server. Since then, however, application development has fragmented into different architectures, SDKs, and cloud services, leaving developers many more patterns to figure out. On top of that, the rise of DevOps has increased the pressure on developers to build and maintain the tools they’re working with, and serious development shops often take pride in building their own toolchains, backends, and databases.

Put it all together — the abundance of choices, the patchwork nature of solutions, the pressure to build and maintain toolchains, and the glue code keeping it all together — and it adds up to more cognitive load, elevated stress levels, and a lengthening of time to value. As Stephen O’Grady from analyst firm RedMonk explains, “Developers are forced to borrow time from writing code and redirect it toward managing the issues associated with highly complex, multifactor developer toolchains held together in places by duct tape and baling wire. This, then, is the developer experience gap.”

Having a lot of options is a good thing — until it’s not. One way we’re working to unwind the paradox of choice is by providing tools that exist in the same form whether in the cloud or on the client — that is, solutions that integrate with the way developers already work. This could mean plugging into a CLI first, abstracting provisioning, simplifying and securing the data layer so developers don’t have to worry about it, and unlocking the creativity of developers with a data model that maps to how data is actually going to be used. We’re also enabling developers to access the tools they need from within MongoDB without having to integrate myriad bolt-on tools (i.e., the paradox of choice).

Building at velocity

The key to unlocking developer productivity, as we see it, is giving developers the building blocks they need to create a whole workload from scratch, or to bring a new workload into their ecosystem — be it time-series, search, or analytics — and have them run on a single platform instead of having to stitch together disparate systems.

Our goal is to bring a modern data layer to modern applications. We want to bring that experience to more and more of what you work on. We know that modern applications have complicated data requirements, but that shouldn’t mean complicated data infrastructure. We want to serve most of your workloads with a single unified platform.

Learn more about MongoDB World 2022 announcements at mongodb.com/new and in these stories:

← Previous

Announcing Atlas Data Federation and Atlas Data Lake

>> Announcement: Some features mentioned below will be deprecated on Sep. 30, 2025. Learn more . Two years ago, we released the first iteration of Atlas Data Lake . Since then, we’ve helped customers combine data from various storage layers to feed downstream systems. But after years spent studying our customers’ experiences, we realized we hadn’t gone far enough. To truly unleash the genius in all our developers, we needed to add an economical cloud object storage solution with a rich MQL query experience to the world of Atlas. Today, we’re thrilled to announce that our new Atlas Data Federation and Atlas Data Lake offerings do just that. We now offer two complementary services, Atlas Data Federation (our existing query service formerly known as Atlas Data Lake) and our new and improved Atlas Data Lake (a fully managed analytic-oriented storage service). Together, these services (both in preview) provide flexible and versatile options for querying and transforming data across storage services, as well as a MongoDB-native analytic storage solution. With these tools, you can query across multiple clusters, move data into self managed cloud object storage for consumption by downstream services, query a workload-isolated inexpensive copy of cluster data, compare your cluster data across different points in time, and much, much more. In hearing from our customers about their experiences with Atlas Data Lake, we learned where they have struggled, as well as the features they’ve been looking for us to provide. With this in mind, we decided to shift the name of our current query federation service to Atlas Data Federation to better align with how customers see the service and are getting value. We’ve seen many customers benefit from the flexibility of a federated query engine service, including querying data across multiple clusters, databases, and collections, as well as exporting data to third-party systems. We also saw where our customers were struggling with data lakes. We heard them ask for a fully managed storage solution so they could achieve all of their analytic goals within Atlas. Specifically, customers wanted scalable storage that would provide high query performance at a low cost. Our new Data Lake provides a high-performance analytic object storage solution, allowing customers to query historical data with no additional formatting or maintenance work needed on their end. How it works Atlas Data Federation encompasses our existing Data Lake functionality with several new affordances. It continues to deliver the same power that it always has, with increased performance and efficiency. The new Atlas Data Lake will now allow you to create Data Lake pipelines (based on your Atlas Cluster backup schedules) and fields on which you can optimize queries. The service takes the following steps: On the selected schedule, a copy of your collection will be extracted from your Atlas backup with no impact to your cluster. During extraction, we build partition indexes based on the contents of your documents and the fields you’ve selected for optimization. These indexes allow your queries to be optimized by capturing the minimums and maximums (and other stats) of the records in each partition, letting you quickly find the relevant data for your queries. Finally, the underlying data lands in an analytic-oriented format inside of cloud object storage. This minimizes data scanned when you execute a query. Once a pipeline has run and a Data Lake dataset has been created, you can select it as a data source in our new Data Federation query experience. You can either set it as the source for a specific virtual collection in a Federated Database Instance or you can have your Federated Database Instance generate a collection name for each dataset that your pipeline has created. Amazingly, no part of this process will consume compute resources from your cluster — neither the export nor the querying of datasets. These datasets provide workload isolation and consistency for long-running analytic queries, a target for ETL jobs using the powerful $out to S3. This makes it easy to compare the state of your data over time. Advanced though this is, it’s only the beginning of the story. We’re committing to evolving the service, improving performance, adding more sources of data, and building new features. All of this will be based on the feedback you, the user, gives us. We can’t wait to see how you’ll use this powerful new tool and can’t wait to hear what you’d like to see next. Try Atlas Data Lake Today

June 7, 2022
Next →

MongoDB Database Observability: Integrating with Monitoring Tools

This post is the final in a three-part series on leveraging database observability. Welcome back to our series on Leveraging Database Observability! Our previous post showcased a real-world use case highlighting how MongoDB Atlas’s observability tools effectively tackle database performance challenges. Whether you’re a developer, DBA, or DevOps engineer, our mission is to empower you to harness the full potential of your data through our observability suite . Integrating Atlas metrics with your central enterprise observability tools can simplify your operations. By seamlessly working with popular observability tools, our approach helps teams streamline workflows and enhance visibility across systems. Integrating MongoDB Atlas with third-party monitoring tools MongoDB’s developer data platform combines all essential data services for building modern applications within a unified experience. Our purpose-built observability tools for Atlas environments offer automatic monitoring and optimization, guiding diagnostics tailored specifically for MongoDB. Additionally, we extend Atlas metrics into your existing enterprise observability stack, enabling seamless integration without replacing your current tools. This creates a consolidated, single-pane view that unifies Atlas telemetry with other tech and application metrics, ensuring comprehensive visibility into both database and full-stack performance. This integration empowers you to monitor, receive alerts, and make data-driven decisions within your existing workflows, driving greater efficiency. Below is a quick guide to modifying integration settings through the Atlas UI and the popular integrations we support: Navigate to the Project Integrations page in Atlas. Choose the organization and project you want to configure from the navigation bar. On the Project Integrations page, select the third-party services you’d like to integrate. Configure the chosen services with the required API keys and regions. Critical integrations for your observability platform With Atlas’s Datadog and Prometheus integrations, you can send critical MongoDB metrics to these platforms, empowering detailed, real-time monitoring. Through Datadog , you can track database operation counts, query efficiency, and resource usage, ideal for pinpointing bottlenecks and managing resources. Similarly, Prometheus enables you to monitor essential metrics like query times, connection rates, and memory usage, supporting flexible tracking of database health and performance. Both integrations facilitate proactive detection of issues, alert configuration for resource thresholds, and a cohesive view of Atlas data when visualized in Grafana. Atlas’s integration with PagerDuty streamlines incident management by sending metrics like performance alerts, billing anomalies, and security events directly to PagerDuty. This integration records incidents automatically, notifies teams upon alerts, and supports two-way syncing, ensuring resolved alerts in Atlas are reflected in PagerDuty. It enables efficient incident response and resource allocation to maintain system stability. With Atlas integrations for Microsoft Teams and Slack, you can route key metrics—such as query latency, disk usage, and throughput—to these channels for timely updates. Teams can use these insights for real-time performance monitoring, incident response, and collaboration. Notifications through these platforms ensure your team stays informed on database performance, storage health, and user activity changes as they occur. Use case: Centralized observability with MongoDB Atlas, Datadog, and Slack Let’s walk through a hypothetical scenario for ShopSmart, an e-commerce company that leverages MongoDB Atlas to manage its product catalog and customer data. As traffic surges, the DevOps team faces challenges in monitoring application performance and database health effectively. To tackle these challenges, the team leverages MongoDB Atlas’ integration with Datadog and Slack, creating a powerful observability ecosystem. Integrating MongoDB Atlas with Datadog: The team pushes key MongoDB Atlas metrics into Datadog, such as query performance, connection counts, and Atlas Vector Search metrics. With Datadog, they can visualize these metrics and correlate overall MongoDB performance with their other applications. Out-of-the-box monitors and dedicated dashboards allow the team to track metrics like throughput, average read/write latency, and current connections. This visibility helps pinpoint bottlenecks in real time, ensuring optimal database performance and improving overall application responsiveness. Setting up alerts in Datadog: The team configures alerts for critical metrics like high query latency and increased error rates. When thresholds are breached, Datadog instantly notifies the team. This proactive approach allows the team to address potential performance issues before they impact customers. Integrating Datadog with Slack: To ensure fast communication, alerts are sent directly to the dedicated Slack channel, “ShopSmart-Alerts.” This integration fosters seamless collaboration, enabling the team to discuss and resolve issues in real-time. With these integrations, ShopSmart’s engineering team can monitor performance quickly and address issues efficiently. The unified observability approach enhances operational efficiency, improves the customer experience, and supports ShopSmart’s competitive edge in the e-commerce industry. By leveraging MongoDB Atlas, Datadog, and Slack, the team ensures scalable performance and drives continuous innovation. Conclusion MongoDB Atlas empowers developers and organizations to achieve unparalleled observability and control over their database environments. By seamlessly integrating with central enterprise observability tools, Atlas enhances your ability to monitor performance metrics and ensures you can do so within your existing workflows. This means you can focus on building modern applications confidently, knowing you have the insights and alerts necessary to maintain optimal performance. Embrace the power of MongoDB Atlas and transform your approach to database management—because your applications can thrive when your data is observable. And that wraps up our Leveraging Database Observability series! We hope you learned something new and found value in these discussions. Sign up for MongoDB Atlas , our cloud database service, to see database observability in action. To dive deeper and expand your knowledge, check out this learning byte for more insights on the MongoDB observability suite and how it can enhance your database performance.

November 14, 2024