Data Resilience with MongoDB Atlas

Steve Jurczak
October 3, 2023 | Updated: January 9, 2024
#Atlas

Data is the central currency in today's digital economy. Studies have shown that 43% of companies that experience major data loss incidents are unable to resume business operations. A range of scenarios can lead to data loss, yet within the realm of database technology, they typically fall under three main categories: catastrophic technical malfunctions, human error, and cyber attacks.

A data loss event due to a catastrophic breakdown, human error, or cyber attack is not a matter of if, but a matter of when it will occur. Hence, businesses need to focus on how to avoid and minimize the effects as much as possible. Failure to effectively address these risks can lead to extended periods of downtime of a few hours or even a few weeks following an incident. The average cost of cyberattacks is a surprising $4.45 million, with some attacks costing in the hundreds of millions. Reputational harm is harder to quantify but no doubt real and substantial.

The specific industry you're in might be subject to regulatory frameworks designed to counter cyber attacks. Businesses that are subject to regulatory regimes must maintain compliance with these requirements. This can determine the configuration of your disaster recovery approach.

In this blog post, we'll explain the key disaster recovery (DR) capabilities available with MongoDB Atlas. We'll also cover the core responsibilities and strategies for data resilience including remediation, and recovery objectives (RTO/RPO).

Planning for data resilience in Atlas

Data resilience is not a one-size-fits-all proposition, which is why we offer a range of choices in Atlas for a comprehensive strategy. Our sensible defaults ensure you're automatically safeguarded, while also offering a variety of choices to precisely align with the needs of each individual application.

When formulating a disaster recovery plan, organizations commonly begin by assessing their recovery point objective (RPO) and recovery time objective (RTO). The RPO specifies the amount of data the business can tolerate losing during an incident, while the RTO indicates the speed of recovery. Since not all data carries the same urgency, analyzing the RPO and RTO on a per-application basis is important. For instance, critical customer data might have specific demands compared to clickstream analytics. The criteria for RTO, RPO, and the length of time you need to retain backups will influence the financial and performance implications of maintaining backups.

With MongoDB Atlas, we provide standard protective measures by default, with customizable options for tailoring protection to the service level agreements specified by the RPO and RTO in your DR plan. These are enhanced by additional features that can be leveraged to achieve greater levels of availability and durability for your most vital tasks. These features can be grouped into two main categories: prevention and recovery.

Backup, granular recovery, and resilience

There are many built-in features that are designed to prevent disasters from ever happening in the first place. Some key features and capabilities that enable a comprehensive prevention strategy include multi-region and multi-cloud clusters, encryption at rest, Queryable Encryption, cluster termination safeguards, backup compliance protocols, and the capability to test resilience. (We will discuss the features in-depth in part two of this series.)

While prevention might satisfy the resilience needs of certain applications, different applications may demand greater resilience against failures based on the business requirements of data protection and disaster recovery.

MongoDB provides comprehensive management of data backups, including the geographic distribution of backups across multiple regions, and the ability to prevent backups from being deleted, all through an automated retention schedule. Recovery capabilities are aimed at supporting RTO and minimizing data loss and include continuous cloud backups with point-in-time recovery.

Atlas cloud backups utilize the native snapshot feature of your cluster's cloud service provider, ensuring backup storage is kept separate from your MongoDB Atlas instances. Backups are essentially snapshots that capture the condition of your database cluster at a specific moment. They serve as a safeguard in case data is lost or becomes corrupted. For M10+ clusters, you have the option of utilizing Atlas Cloud Backups, which leverage the cluster's cloud service provider for storing backups in a localized manner.

Atlas comes with strong default backup retention of 12 months out of the box. You also have the option to customize snapshot and retention schedules, including the time of day for snapshots, the frequency at which snapshots are taken over time, and retention duration. Another important feature is continuous cloud backup with point-in-time recovery, which enables you to restore data to the moment just before any incident or disruption, such as a cyber attack.

To ensure your backups are regionally redundant and you can still restore even if the primary region that your backups are in is down, MongoDB Atlas offers the ability to copy these critical backups, with the point-in-time data, to any secondary region available from your cloud provider in Atlas.

For the most stringent regulations, or for businesses that want to ensure backups are available even after a bad actor or cyber attack, MongoDB Atlas can ensure that no user, regardless of role, can ever delete a backup before a predefined protected retention period with the Backup Compliance Policy.

Whatever your regulatory obligations or business needs are, MongoDB Atlas provides the flexibility to tailor your backup settings for requirements. Crucially, this ensures you can recover quickly, minimizing data loss and meeting your RPO in the event of a disaster recovery scenario.

When properly configured, testing has shown that Atlas can quickly recover to the exact timestamp before a disaster or failure event, giving you a one-minute RPO and RTO of less than 15 minutes when utilizing optimized restores. Recovery times can vary due to cloud provider disk warming and which point in time you are restoring to. So, it is important to also test this regularly.

This means that regardless of your regulatory or business requirements, MongoDB Atlas allows you to configure your backups to ensure that you can meet your recovery requirements and, most importantly, recover with precision and speed to ensure that your data loss is minimal and your recovery point objectives are met should you experience a recovery event.

Conclusion

As regulations and business needs continue to evolve, and cyber-attacks become more sophisticated and varied, creating and implementing a data resilience strategy can be simple and manageable. MongoDB Atlas comes equipped with built-in measures that deliver robust data resilience at the database layer, ensuring your ability to both avoid incidents and promptly restore operations with minimal data loss if an incident does occur. Furthermore, setting up and overseeing additional advanced data resilience features is straightforward, with automation driven by a pre-configured policy that operates seamlessly at any scale. This streamlined approach supports compliance without the need for manual interventions, all within the MongoDB Atlas platform.

For more information on the data resilience and disaster recovery features in MongoDB Atlas, download the Data Resilience Strategy with MongoDB Atlas whitepaper.

To get started on Atlas today, we invite you to launch a free tier today.

← Previous
Building AI with MongoDB: Cultivating Trust with Data
“Trust is like the air we breathe – when it’s present, nobody really notices; when it’s absent, everybody notices.” - Warren Buffett The issue of trust is one that dominates discussions around the safe and responsible adoption of AI across business and society. It was another Warren - this time Warren Bennis, a pioneer in modern leadership principles – who was attributed as saying "Trust is the lubrication that makes it possible for organizations to work." Particularly relevant when we think about how organizations are starting to embed AI into the very fabric of their businesses. On one hand, we have governments around the world that are at varying stages of regulating their way to trustworthy AI. However, this will not be a quick process, and enterprises can’t afford to wait. Businesses need to make progress now if they are going to unlock the opportunities presented by AI. In our latest roundup of AI innovators building with MongoDB, we’re going to focus on three companies tackling trust from different angles. We feature Nomic who are working to make AI more explainable. Robust Intelligence is focused on securing AI models against prompt injections, data poisoning, bias, PII leakage, and more. Finally, VISO TRUST comes at this issue from a totally different perspective. They use AI to help their customers reduce cybersecurity risks and improve trust across the supply chain. Let's dig in. Check out our AI resource page to learn more about building AI-powered apps with MongoDB. Making AI explainable and accessible Despite the huge advances in AI and its use in almost every industry, very little is known about how the most popular models actually work. What data are they trained on? What are they learning? How can we compare accuracy between different models? These are the questions Nomic AI is seeking to help us answer through its Atlas and GPT4All products. Nomic Atlas is a data engine that allows users to explore, label, search, share, and build on massive datasets using their web browser. With Atlas, users can begin to understand what data their chosen AI models are learning from and the associations they are making during the training phase. Atlas can be used for exploratory data analysis, data labeling and cleansing, and visualizations of vector embeddings. To see Nomic Atlas in action, take a look at the recent blog post with Hugging Face announcing IDEFICS , an open-access reproduction of the visual language model based on Flamingo. The model takes image and text inputs and produces text outputs from them. For example, it can answer questions about images, describe visual content, and create stories grounded in multiple images. Nomic allows users to visually explore the content of the training data, as illustrated in the image below. Atlas can be used to curate high-quality training and instruction-tuned datasets for the GPT4All models. Nomic GPT4All is an ecosystem for training and deploying powerful and customized large language models that run locally on consumer-grade CPUs in Windows, Mac, and Ubuntu Linux clients. With GPT4All, users have access to a free-to-use, locally running, privacy-aware chatbot that doesn’t require expensive and scarce GPUs to train and infer on, or an internet connection. It can power question-answering systems, personal writing assistants, document summarization, and code generation. Demand for GPT4All has been explosive, accruing more than 20,000 GitHub stars within its first week of launch. “Every month MongoDB is adding hundreds of organizations and thousands of developers who are building AI-enabled apps on its multi-cloud developer data platform ,” said Brandon Duderstadt, CEO of Nomic. “It makes sense for us to partner with MongoDB Ventures . They are helping us accelerate our vision of making AI explainable and accessible to everyone.” Update, February 6th 2024: On February 1, 2024, Nomic released its Nomic Embed open-source embedding model and a fully managed inference endpoint. This allows anyone to build their own powerful RAG applications for generative AI using a text embedding model with a 8,192 context-length that outperforms proprietary alternatives on a variety of benchmarks. To demonstrate its new endpoint and model in action, the Nomic engineers created the Building a RAG LLM with Nomic Embed and MongoDB . By following the blog post, you will learn: How to use Nomic to generate embeddings for your data sources. Add them to MongoDB Atlas Vector Search . (Note that this runs in the Atlas free tier, so there is no cost to you!) Use an open-source LLM to generate text from your retrieved documents. Because you have access to the code and data behind the Nomic Embed model, you can easily customize it for even better performance. Securing generative AI, supercharged by your data Robust Intelligence delivers end-to-end AI risk management to protect organizations from security, ethical, and operational risks. The company’s platform automates testing and compliance across the AI lifecycle through continuous validation and protects models in real-time with AI Firewall. This combined approach enables Robust Intelligence to proactively manage risk for any model type, including generative AI and gives organizations the confidence to unleash the true potential of AI. Robust Intelligence is trusted by leading companies including ADP, JPMorgan Chase, Expedia, Deloitte, PwC, and the U.S. Department of Defense. Recent advancements in generative AI have motivated companies to experiment with potential applications, but a lack of security controls has exposed companies to unmanaged risks. This challenge is exacerbated when sensitive company information is used to enrich pre-trained models, such as connecting vector databases, in order to increase the relevance to the end user. Robust Intelligence’s AI Firewall protects large language models (LLMs) in production by validating inputs and outputs in real-time. It assesses and mitigates operational risks such as hallucinations; ethical risks, including model bias and toxic outputs; and security risks such as prompt injections and PII extraction. AI Firewall stops bad or malicious inputs from reaching AI models and prevents undesired AI-generated results from reaching the application. Customers can confidently connect MongoDB Atlas Vector Search to any commercial or open-source LLM for secure retrieval-augmented generation with the AI Firewall integration. Atlas Vector Search serves as the memory and fact database for AI Firewall, ensuring the AI model provides enriched responses without hallucinating. Additionally, it serves as the memory and database to store historical data points. This is important in the context of identifying more advanced security attacks, such as data poisoning and model extraction, which often manifest across a cluster of data points as opposed to a single data point. Yaron Singer, CEO and co-founder at Robust Intelligence commented “By incorporating MongoDB’s Atlas Vector Search into the AI validation process, customers can confidently use their databases to enhance LLM responses knowing that sensitive information will remain secure. The integration provides seamless protection against a comprehensive set of security, ethical, and operational risks.” Being part of the MongoDB Partner Program provides Robust Intelligence with access to specialist technical support to optimize product integrations and provides visibility to the MongoDB customer base. Transforming cyber risk intelligence VISO TRUST is an AI-powered third-party cyber risk and trust platform that enables any company to access actionable vendor security information in minutes. VISO TRUST delivers fast and accurate intelligence needed to make informed cybersecurity risk decisions at scale. Today VISO TRUST has many great enterprise customers like InstaCart, Gusto, and Upwork and they all say the same thing: 90% less work, 80% reduction in time to assess risk, and near 100% vendor adoption. How does VISO TRUST achieve these results? Pierce Lamb, Senior Software Engineer on the Data and Machine Learning team at VISO TRUST provides more detail: “VISO TRUST Platform easily engages third parties, saving everyone time and resources. In a 5-minute web-based session, third parties are prompted to upload relevant artifacts of the security program that already exists, and our supervised AI – which we call Artifact Intelligence – does the rest. First, VISO TRUST deploys discriminator models that produce high-confidence predictions about features of the artifact. Secondly, artifacts have text content parsed out of them which we embed and store in MongoDB Atlas to become part of our dense retrieval system. This dense retrieval system performs Retrieval-Augmented Generation (RAG) using MongoDB features like Atlas Vector Search to provide ranked context to large language model (LLM) prompts. Thirdly, we use RAG results to seed LLM prompts and chain together their outputs to produce extremely accurate factual information about the artifact in the pipeline. This information is able to provide instant intelligence to customers that previously took weeks to produce.” VISO TRUST is the only SaaS third-party cyber risk management platform that delivers the rapid security intelligence needed for modern companies to make critical risk decisions early in the procurement process VISO TRUST uses state-of-the-art models from OpenAI, Hugging Face, Anthropic, Google, and AWS, augmented by vector search and retrieval from MongoDB Atlas. Read our interview blog post with VISO TRUST to learn more. What's next? If you are getting started with building AI-enabled apps on MongoDB, sign up for our AI Innovators Program . Successful applicants get access to expert technical advice, free MongoDB Atlas credits, co-marketing opportunities, and – for eligible startups, introductions to potential venture investors. In the spirit of "Trust, but verify" (Ronald Reagan), if you’re not sure how the program or indeed, MongoDB, could deliver value to you, take a look at earlier blog posts in this series: Building AI with MongoDB: first qualifiers include AI at the network edge for computer vision and augmented reality; risk modeling for public safety; and predictive maintenance paired with Question-answer generation for maritime operators. Building AI with MongoDB: compliance to copilots features AI in healthcare along with intelligent assistants that help product managers specify better products and help sales teams compose emails that convert 2x higher. Building AI with MongoDB: unlocking value from multimodal data showcases open source libraries that transform unstructured data into a usable JSON format; entity extraction for contracts management; and making sense of “dark data” to build customer service apps. You should look at the MongoDB for Artificial Intelligence resources page for the latest best practices that get you started in turning your idea into an AI-driven reality.
October 3, 2023

Next →
MongoDB and Partners: Building the AI Future, Together
If you’re like me, over the past year you’ve closely watched AI’s developments—and the world’s reactions to them. From infectious excitement about AI’s capabilities, to impatience with its cost and return on investment, every day has been filled with AI twists and turns. It’s been quite the roller coaster. During the ride, from time to time I’ve wondered where AI falls on the Gartner hype cycle, which gives "a view of how a technology or application will evolve over time." Have we hit the "peak of inflated expectations" only to fall into the "trough of disillusionment?" Or is the hype cycle an imperfect guide, as The Economist argues? The reality is that it takes time for any new technology—even transformative ones like AI—to take hold. And every advance, no matter how big, has had its detractors. A famous example is that of Picasso (!), who in 1968 said, “Computers are useless. They can only give you answers.” (!!) For our part, MongoDB is convinced that AI is a once-in-a-generation technology that will enhance every future application—a belief that has been reinforced by the incredible work our partners have shared at MongoDB’s 2024 events. Speeding AI development MongoDB is committed to helping organizations of all sizes succeed with AI, and one way we’re doing that is by collaborating with the MongoDB partner ecosystem to create powerful, user-friendly AI development tools and solutions. For example, Fireworks.ai —which is a member of the MongoDB AI Applications Program ecosystem —created an inference solution that hosts gen AI models and supports containerized deployments. This tool makes it easier for developers to build and deploy powerful applications with a range of easy-to-use tools and customization options. They can choose to use state-of-the-art, open-source language, image, and multimodal foundation models off the shelf, or they can customize and fine-tune models to their needs. Jointly, Fireworks.ai and MongoDB provide a solution for developers who want to leverage highly curated and optimized open-source models and combine these with their organization’s own proprietary data—and to do so with unparalleled speed and security. “MongoDB is one of the most sophisticated database providers, and it’s very easy to use,” said Benny Chen , cofounder of Fireworks.ai. "We want developers to be able to use these tools, and we want to work with providers who enable and empower developers." Nomic , another MAAP ecosystem member, also enables developers with best-in-class solutions across the entire unstructured data workflow. Their Embed offering, available through the Nomic API , allows users to vectorize large-scale datasets for use in text, image, and multimodal retrieval applications, including retrieval-augmented generation (RAG), using only their web browser. The Nomic-MongoDB solution is a highly efficient, open-weight model that developers can use to visualize the unstructured datasets they store in MongoDB Atlas . These insights help users quickly discover trends and articulate data-driven value propositions. Nomic also supported the recently announced vector quantization in MongoDB Atlas Vector Search , which reduces vector sizes while preserving performance. Last—but hardly least!—there’s our new reference architecture with MAAP partners AWS and Anthropic. Announced at MongoDB.local London , the reference architecture supports building memory-enhanced AI agents, and is designed to streamline complex processes and develop smarter, more responsive applications. For more—including a link to the code on Github— check out the MongoDB Developer Center . Making AI work for anyone and everyone The companies MongoDB partners with aren’t just making gen AI easier for developers—they’re building tools for everyone. For example, Capgemini has invested $2 billion in gen AI and is training 100,000 of its employees in the technology. GenYoda, a solution that helps insurance professionals with their daily work, is a product of this investment. GenYoda leverages MongoDB Atlas Vector Search to analyze large amounts of customer data, like policy statements, premiums, claims history, and health information. Using GenYoda, insurance professionals can quickly analyze underwriters’ reports to make informed decisions, create longitudinal health summaries, and streamline customer interactions to improve contact center efficiency. GenYoda can ingest 100,000 documents in just a few hours and respond to users’ queries in two to three seconds—a metric on par with the most widely used gen AI models. And it produces results: in one example, by using Capgemini’s solution an insurer was able to increase productivity by 15%, add new reports 25% faster (thus speeding decision-making), and reduce the manual effort of searching PDFs, increasing efficiency by 10%. Building the future of AI together So, what’s next? Honestly, I’m as curious as you are. But I’m also incredibly excited. At MongoDB, we’re active participants in the AI revolution, working to embrace the possibilities that lie ahead. The future of gen AI is bright, and I can’t wait to see what we’ll build together. To learn more about how MongoDB can accelerate your AI journey, explore the MongoDB AI Applications Program .
November 4, 2024