Introducing MongoDB Realm’s Flexible Sync – Now Available in Preview

Emma Lullo and Ian Ward
January 24, 2022 | Updated: October 11, 2024

>> Announcement: Some features mentioned below will be deprecated on Sep. 30, 2025. Learn more.

Twelve months ago, we made MongoDB’s edge-to-cloud data synchronization service, Realm Sync, generally available. Since then, Sync has helped hundreds of our customers build reliable, offline-first mobile apps that serve data to millions of end users – from leading telematics providers to chart-topping consumer apps.

Historically, Realm Sync has worked well for apps where data is compartmentalized and permissions rarely change, but dynamic use cases with evolving permissions required workarounds. We knew we could do more, so today we are excited to announce the next iteration of Realm Sync – Flexible Sync. With the introduction of Flexible Sync, we are redefining the sync experience by enabling even the most complex use cases out-of-the-box without requiring any custom code.

Intuitive query-based sync

Distinctly different from how Realm Sync operates today, Flexible Sync lets you use language-native queries to define the data synced to user applications. This more closely mirrors how you are used to building applications today – using GET requests with query parameters – making it easy to learn and fast to build to MVP.

Flexible Sync also supports dynamic, overlapping queries based on user inputs. Picture a retail app that allows users to search available inventory. As users define inputs – show all jeans that are size 8 and less than $40 – the query parameters can be combined with logical ANDs and ORs to produce increasingly complex queries, and narrow down the search result even further.

In the same application, employees can quickly limit inventory results to only their store’s stock, pulling from the same set of documents as the customer, without worrying about overlap.

Document-level permissions

Whether it’s a company’s internal application or an app on the App Store, permissions are required in almost every application. That’s why we are excited by how seamless Flexible Sync makes applying a document-level permission model when syncing data – meaning synced documents can be limited based on a user’s role.

Consider how an emergency room team would use their hospital’s application. A resident should only be able to access her patients’ charts while her fellow needs to be able to see the entire care team’s charts. In Flexible Sync, a user’s role will be combined with the client-side query to determine the appropriate result set. For example, when the resident above filters to view all patient charts the permission system will automatically limit the results to only her patients.

Real-time collaboration optimizations

Flexible Sync also enhances query performance and optimizes for real-time collaboration by treating a single object or document as the smallest entity for synchronization. This means synced data is shared between client devices more efficiently and conflict resolution incorporates changes faster and with less data transfer than before.

Getting started

Flexible Sync is available now. Simply sign up or log in to your cloud account, deploy a Realm app, select your sync type, and dive right in.

Flexible Sync is compatible with MongoDB 5.0, which is available with dedicated Atlas database clusters (M10 and higher). Shared-tier cluster support for 5.0 and Flexible Sync will be made available mid-February.

Have questions? Check out our documentation or the more detailed announcement post on the Developer Hub.

Looking ahead

Our goal with Flexible Sync is to deliver a sync service that can fit any use case or schema design pattern imaginable without custom code or workarounds. And while we are excited that Flexible Sync is now in preview, we’re nowhere near done.

The Realm Sync team is planning to bring you more query operators, permissions integrations, and enhancements over the course of 2022. We look to you, our users, to help us drive the roadmap. Submit your ideas and feature requests to our feedback portal and ask questions in our Community forums. Happy building!

← Previous

10 Signs Your Data Architecture Is Limiting Your Innovation: Part 3

When it comes to your database architecture, complexity can quickly lead to a drag on your productivity, frustration for your developers, and less time to focus on innovation while your team maintains the status quo. New feature rollouts take longer than they should, while your resources are consumed up by tedious tasks that allow your app to survive, but not truly thrive. This complexity manifests in many different ways; as they accumulate, they can become a serious hindrance to your ability to bring innovative ideas to market. We think of the effect as a kind of tax — a tax that is directly rooted in the complexity of your data architecture. We call it DIRT — the Data and Innovation Recurring Tax . We have identified ten symptoms that can indicate your business is paying DIRT. For an in-depth view, read our white paper 10 Signs Your Data Infrastructure is Holding You Back . Sign #5: New features are rolled out in months, not days With a complex data architecture, your developers have to switch constantly between languages and think in different frameworks. They may use one language to work directly with a database, another to use the object-relational mapping (ORM) layer built on top of it, and yet another to access search functionality. That becomes a major drag on productivity. That slows down your individual developers, but it also has consequences for how they work as a team. If every application architecture is bespoke, it’s almost impossible for developers’ skills to be shared and put to use across an organization. Development slows down. When a key person leaves, there is no one who can effectively fill in and you end up hiring for very specific skills. That’s hard enough, but you also don’t know if you’ll still need those skills in a year or three. Sign #6: It takes longer to roll out schema changes than to build new features If you’re rolling out application changes frequently — or trying to — and you’re using a relational database, then schema changes are hard to avoid. One survey found that 60% of application changes require modifications to existing schema, and, worse, those database changes take longer to deploy than the application changes they are supposed to support. Legacy relational databases require developers to choose a schema at the outset of a project, before they understand the entirety of the data they need or the ways in which their applications will be used. Over time, and with user feedback, the application takes shape — but often it’s not the shape that was originally anticipated. At that point, a fixed schema makes it very hard to iterate, leaving teams with a tough choice: try to achieve your new goals within the context of a schema that isn’t really suitable or go through the painful process of changing it. Learn more about the innovation tax and how to lessen it in our white paper DIRT and the High Cost of Complexity .

January 21, 2022

Next →

MongoDB Database Observability: Integrating with Monitoring Tools

This post is the final in a three-part series on leveraging database observability. Welcome back to our series on Leveraging Database Observability! Our previous post showcased a real-world use case highlighting how MongoDB Atlas’s observability tools effectively tackle database performance challenges. Whether you’re a developer, DBA, or DevOps engineer, our mission is to empower you to harness the full potential of your data through our observability suite . Integrating Atlas metrics with your central enterprise observability tools can simplify your operations. By seamlessly working with popular observability tools, our approach helps teams streamline workflows and enhance visibility across systems. Integrating MongoDB Atlas with third-party monitoring tools MongoDB’s developer data platform combines all essential data services for building modern applications within a unified experience. Our purpose-built observability tools for Atlas environments offer automatic monitoring and optimization, guiding diagnostics tailored specifically for MongoDB. Additionally, we extend Atlas metrics into your existing enterprise observability stack, enabling seamless integration without replacing your current tools. This creates a consolidated, single-pane view that unifies Atlas telemetry with other tech and application metrics, ensuring comprehensive visibility into both database and full-stack performance. This integration empowers you to monitor, receive alerts, and make data-driven decisions within your existing workflows, driving greater efficiency. Below is a quick guide to modifying integration settings through the Atlas UI and the popular integrations we support: Navigate to the Project Integrations page in Atlas. Choose the organization and project you want to configure from the navigation bar. On the Project Integrations page, select the third-party services you’d like to integrate. Configure the chosen services with the required API keys and regions. Critical integrations for your observability platform With Atlas’s Datadog and Prometheus integrations, you can send critical MongoDB metrics to these platforms, empowering detailed, real-time monitoring. Through Datadog , you can track database operation counts, query efficiency, and resource usage, ideal for pinpointing bottlenecks and managing resources. Similarly, Prometheus enables you to monitor essential metrics like query times, connection rates, and memory usage, supporting flexible tracking of database health and performance. Both integrations facilitate proactive detection of issues, alert configuration for resource thresholds, and a cohesive view of Atlas data when visualized in Grafana. Atlas’s integration with PagerDuty streamlines incident management by sending metrics like performance alerts, billing anomalies, and security events directly to PagerDuty. This integration records incidents automatically, notifies teams upon alerts, and supports two-way syncing, ensuring resolved alerts in Atlas are reflected in PagerDuty. It enables efficient incident response and resource allocation to maintain system stability. With Atlas integrations for Microsoft Teams and Slack, you can route key metrics—such as query latency, disk usage, and throughput—to these channels for timely updates. Teams can use these insights for real-time performance monitoring, incident response, and collaboration. Notifications through these platforms ensure your team stays informed on database performance, storage health, and user activity changes as they occur. Use case: Centralized observability with MongoDB Atlas, Datadog, and Slack Let’s walk through a hypothetical scenario for ShopSmart, an e-commerce company that leverages MongoDB Atlas to manage its product catalog and customer data. As traffic surges, the DevOps team faces challenges in monitoring application performance and database health effectively. To tackle these challenges, the team leverages MongoDB Atlas’ integration with Datadog and Slack, creating a powerful observability ecosystem. Integrating MongoDB Atlas with Datadog: The team pushes key MongoDB Atlas metrics into Datadog, such as query performance, connection counts, and Atlas Vector Search metrics. With Datadog, they can visualize these metrics and correlate overall MongoDB performance with their other applications. Out-of-the-box monitors and dedicated dashboards allow the team to track metrics like throughput, average read/write latency, and current connections. This visibility helps pinpoint bottlenecks in real time, ensuring optimal database performance and improving overall application responsiveness. Setting up alerts in Datadog: The team configures alerts for critical metrics like high query latency and increased error rates. When thresholds are breached, Datadog instantly notifies the team. This proactive approach allows the team to address potential performance issues before they impact customers. Integrating Datadog with Slack: To ensure fast communication, alerts are sent directly to the dedicated Slack channel, “ShopSmart-Alerts.” This integration fosters seamless collaboration, enabling the team to discuss and resolve issues in real-time. With these integrations, ShopSmart’s engineering team can monitor performance quickly and address issues efficiently. The unified observability approach enhances operational efficiency, improves the customer experience, and supports ShopSmart’s competitive edge in the e-commerce industry. By leveraging MongoDB Atlas, Datadog, and Slack, the team ensures scalable performance and drives continuous innovation. Conclusion MongoDB Atlas empowers developers and organizations to achieve unparalleled observability and control over their database environments. By seamlessly integrating with central enterprise observability tools, Atlas enhances your ability to monitor performance metrics and ensures you can do so within your existing workflows. This means you can focus on building modern applications confidently, knowing you have the insights and alerts necessary to maintain optimal performance. Embrace the power of MongoDB Atlas and transform your approach to database management—because your applications can thrive when your data is observable. And that wraps up our Leveraging Database Observability series! We hope you learned something new and found value in these discussions. Sign up for MongoDB Atlas , our cloud database service, to see database observability in action. To dive deeper and expand your knowledge, check out this learning byte for more insights on the MongoDB observability suite and how it can enhance your database performance.

November 14, 2024