Ways to Integrate MongoDB Atlas in Your DevOps Processes

Darshana Paithankar
October 11, 2023 | Updated: September 27, 2024
#AtlasCLI#DevOps

MongoDB Atlas - the industry-leading developer data platform integrates all of the data services you need to build modern applications in a unified developer experience. We want to meet you where you are and offer various ways to begin with Atlas and make the most of its features. Starting with the Atlas User Interface is a good initial step. Now what if your requirement is to automate the deployment of Atlas clusters at scale, and by leveraging tools that are already integral to your application ecosystem? Through this blog, we will address this question.

We will discuss the programmatic methods for starting with Atlas and deploying Atlas resources and infrastructure according to your specific needs. The tools we provide cover most of the control plane management ( easily deploy, manage, scale) Atlas clusters. Thereby, enabling you to create the building blocks of Atlas such as clusters, database users, projects, Atlas Search indexes, backups, alerts, and more. You can interact directly with the data plane through the command line or programmatically using tools like MongoDB Shell, Compass, language drivers, etc., to work with data and perform CRUD operations.

1. Atlas Administration API: Leveraging your choice of client

No matter what your choice of client: be it cURL commands, Postman, Insomnia etc. You can use a client of your choice to directly interact with Atlas through the Atlas Administration API. The Atlas Administration API gives you a RESTful interface to interact with Atlas resources and perform various actions within MongoDB Atlas. Each endpoint represents a specific resource in Atlas ( e.g. cluster). You can programmatically deploy and manage all of the Atlas resources from an administrative standpoint such as creating a cluster, database user, projects, advanced clusters, backups, monitoring etc.

All of the tools that we will cover below are built on top of the Atlas Administration API and abstract away the complexities of using the Atlas Atlas Administration API directly.

2. GoSDK Client: A simplified way to get started with the Atlas Admin API

One of the mechanisms that simplify the interaction with the API is the availability of SDK (Software Development Kit). If you are a Go developer, the GoSDK client gives you a much simpler experience of getting started with the Atlas Administration API. It also has full endpoint coverage of the Administration API and improves the speed of getting started with the Atlas Administration API. Getting started with the Admin API through the Go SDK client only takes a few lines of code since the SDK includes pre-built functions, structs, and methods that encapsulate the complexity of HTTP requests, authentication, error handling, versioning, and other low-level details. We will be adding SDK support for more languages in the future. Feel free to add feedback on what other languages you would like to see supported.

3. MongoDB Atlas CLI: Simple command-line tool to easily deploy Atlas resources

If you prefer to manage your Atlas resources using simple commands in the terminal, the MongoDB Atlas Command-Line Interface (CLI) is your answer. With the Atlas CLI, you can seamlessly manage your clusters, automate database user creation, control network access, and perform various other administrative tasks, all from the command line. You can also script these actions out with the CLI for even easier repeatability. It is available for multiple operating systems, including Windows, Linux, and macOS. It is often used in conjunction with other command-line tools to automate workflows and integrate with CI/CD pipelines. An easy way to get started with the Atlas CLI is through the quickstart.

4. Infrastructure as Code (IaC) integrations: Automating deployment of Atlas using Infrastructure as Code tools

There are several advantages of using Infrastructure as Code ( IaC) tools to provision application infrastructure. By treating your infrastructure code similar to your application code, IaC tools offer benefits such as version control, scalability, security, repeatability etc. What if you could easily deploy your Atlas resources using your preferred IaC tools? MongoDB Atlas gives you that flexibility. Whether you're an AWS CloudFormation or a HashiCorp Terraform enthusiast, you can provision and manage Atlas resources using these tools with ease through the integrations we offer:

AWS CloudFormation integration: MongoDB Atlas supports three ways to provision resources using AWS CloudFormation

  • The first is by leveraging Atlas resources directly from the CloudFormation Public Registry. We have 33+ resources available today. The configurations are defined in JSON/ YAML and can be executed using the AWS CLI/ AWS management console.
  • If you want a faster way to get started, you can explore our AWS Partner Solutions ( formerly known as quickstarts), which have pre-built Cloudformation templates to help you provision a group of Atlas resources for specific use cases instead of deploying them one by one.
  • And if you are passionate about using languages you are comfortable with such as JavaScript/TypeScript, Python, Java, C#, and Go instead of learning YAML/JSON, you can leverage the AWS Cloud Development Kit (CDK) to deploy Atlas resources. Under the hood, when your AWS CDK applications are run, they translate your code to CloudFormation templates which use the CloudFormation service for provisioning.

HashiCorp Terraform integration: If you are already using Terraform as your IaC tool of choice, we have integrations with Hashicorp Terraform as well. There are two easy ways to get started with Hashicorp Terraform

  • By directly provisioning Atlas resources on AWS, Azure, and Google Cloud using the Terraform MongoDB Atlas provider.
  • If you prefer to use your favorite language, e.g. Typescript, Python, C#, Java, Go, you can use CDKTF (Cloud Development Kit for Terraform), which will allow you to deploy Atlas using Terraform under the hood without knowing the specifics of Terraform’s configuration language

5. Atlas Kubernetes Operator: Use your existing Kubernetes tooling to manage Atlas resources

For organizations leveraging Kubernetes for container orchestration, the Atlas Kubernetes Operator provides seamless integration between Kubernetes and MongoDB Atlas. This operator allows you to deploy and manage Atlas resources using your existing Kubernetes tooling, streamlining the process of spinning up and scaling databases alongside your applications. You can manage Atlas in exactly the same way you manage your applications running in Kubernetes. This is done by managing Atlas directly via custom resources in Kubernetes. These custom resources can be created, managed, and stored as the source of truth in your repository. You can then leverage a Continuous Deployment tool of your choice such as ArgoCD to apply them into Kubernetes.

Whether you prefer working with the command line, a RESTful API, Kubernetes, or IaC tools, MongoDB Atlas provides a diverse set of tools to help you achieve your automation goals. By embracing these methods, you can streamline your operations, improve efficiency, and pave the way for a more agile and responsive development process.

Learn more from our Atlas Programmatic Access documentation page

← Previous

Boost the Accuracy of E-commerce Search Results with Atlas Vector Search

Artificial Intelligence’s (AI) growth has led to transformative advancements in the retail industry, including natural language processing, image recognition, and data analysis. These capabilities are pivotal to enhancing the efficiency and accuracy of e-commerce search results. E-commerce, characterized by its vast product catalogs and diverse customer base, generates enormous amounts of data every day. From user preferences and search histories to product reviews and purchase patterns — and add to that images, video, and audio associated with product campaigns and user search — the data is both a goldmine and a challenge. Traditional search mechanisms, which rely on exact keyword matches, are inadequate at handling such nuanced and voluminous data. This is where vector search comes into play as the perfect data mining tool . As a sophisticated search mechanism, it leverages AI-driven algorithms to understand the intrinsic relationships between data points. This enables it to discern complex patterns, similarities, and contexts that conventional keyword-based searches might overlook. Let’s dig deeper into the differences between traditional keyword matching search and vector search, and answer questions like: What type of queries does vector search improve in the retail search landscape? What are the challenges associated with it? And how can your business tap into the competitive advantage it represents? Check out our AI resource page to learn more about building AI-powered apps with MongoDB. Traditional Keyword Matching vs. Vector Search Traditional search functionalities for e-commerce platforms — keyword matching, typo tolerance, autocomplete, highlighting, facets, and scoring — are often built in-house or implemented on top of typical search engines like Apache Lucene, AtlasSearch, or ElasticSearch, relying heavily on metadata textual descriptions. While this has served the industry well for years, it often falls short of understanding the nuanced needs of modern consumers. For instance, a customer might be looking for a "blue floral summer dress," but if the product description lacks these terms, it might not appear in the search results, even if it perfectly matches the visual description. Figure 1: As embeddings encode numerically the meaning of documents, semantically close documents will be geometrically close as well. Vector search is a method that finds similar items in a dataset based on their vector representations, and offers a more efficient and accurate way to sift through large datasets. Instead of relying on exact matches, it uses mathematical techniques to measure the similarity between vectors, allowing it to retrieve items that are semantically similar to the user's query, even if the query and the item descriptions don't contain exact keyword matches. Figure 2: Data flow diagram showcasing how applications, vector embedding algorithms, and search engines work together at a high level. One great thing about Vector search is that by encoding any type of data, i.e. text, images or sound, you can perform queries on top of that, creating a much more comprehensive way of improving the relevance of your search results. Let’s explore examples of queries that involve context, intent, and similarity. Visual similarity queries Query: "Find lipsticks in shades similar to this coral lipstick." Vector Search Benefit: Vector search can recognize the color tone and undertones of the specified lipstick and suggest similar shades from the same or different brands. Data type: image or text Contextual queries Query: "Affordable running shoes for beginners." Vector Search Benefit: Vector search can consider both the price range and the context of "beginners," leading to relevant shoe suggestions tailored to the user's experience level and budget. Data type: text, audio (voice) Natural language queries Query: "Show me wireless noise-canceling headphones under $100." Vector Search Benefit: Capture intent. Vector search can parse the query's intent to filter headphones with specific features (wireless, noise-canceling) and a price constraint, offering products that precisely match the request. Data type: text, audio (voice) Complementary product queries Query: "Match this dress with elegant heels and a clutch." Vector Search Benefit: Vector search can comprehend the user's request to create a coordinated outfit by suggesting shoes and accessories that complement the selected dress. Data type: text, audio (voice), image Challenging landscape, flexible stack Now that we've explored different queries and their associated data types that could be used in vector embeddings for search, we can see how much more information can be used to deliver more accurate results and fuel growth. Let’s consider some of the challenges associated with a vector search solution data workflow and how MongoDB Atlas Vector Search helps bridge the gap between challenges and opportunities. Data overload The sheer volume of products and user-generated data can be overwhelming, making it challenging to offer relevant search results. By embedding different types of data inputs like images, audio (voice), and text queries for later use with vector search, we can simplify this workload. Storing your vector encoding in the same shared operational data layer your applications are built on top of, but also generating search indexes based on those vectors, makes it simple to add context to your application search functionalities. Using Atlas Vector Search combined with MongoDB App Services , you can reduce operational overhead by creating a trigger that could “see” when a new document is created in your collections and automatically make the call to the embedding API of your preference, pushing the document to it and storing the retrieved embedding data in the same document stored in your collection. Figure 3: Storing vectors with the data simplifies the overall architecture of your application. As the number of documents or vectors grows, efficient indexing structures ensure that search performance remains reasonable. By simply creating an index based on the embedded data field, you can leverage the optimized retrieval of the data, reduce the computational load, and accelerate its performance, especially for nearest neighbor search tasks, where the goal is to find items that are most similar to a given query. Altogether, the combination of MongoDB Vector Search capabilities with App Services and indexing provides a robust and scalable solution to achieve real-time responsiveness. An indexed vector search database can provide rapid query results, making it suitable for applications like recommendation engines or live search interfaces. Changing consumer behavior Developing an effective vector search solution involves understanding the nuances of the retail domain. Retailers must consider factors like seasonality, trends, and user behavior to improve the accuracy of search results. To overcome this challenge, retailers will need to be able to adjust their business model by categorizing their product catalogs and user data according to different criteria, for example: So as you can see all this vast amount of information can be embedded to build more comprehensive criteria for relevance, but first it needs to be properly captured and organized. This is where the value of the flexible document model comes into play. The document model allows you to define different fields and attributes for each category of data. This can be used to capture the various categorization criteria. Retailers could also utilize embedded subdocuments to associate relevant information with products or customers. For instance, you can embed a subdocument containing marketing campaign data, engagement channels, and geographic location within products to track their performance. As categorization criteria evolve, dynamic schema evolution allows you to add or modify fields without disrupting existing data. This flexibility easily accommodates changing business needs. Retailers may also use embedded arrays to record purchase history for customers. Each array element can represent a transaction, including product details and purchase date, facilitating segmentation based on recency and frequency. By embedding all these different data types, and leveraging the flexible capabilities of the document model, retailers can create a comprehensive and dynamic system that effectively categorizes data according to diverse criteria in a fast and resilient way. This enables personalized search experiences and enhanced customer engagement in the e-commerce space. Sitting on a goldmine Every retailer worldwide now realizes that with their customer data, they are sitting on a goldmine. Using the proper enabling technologies would allow them to build better experiences for their customers while infusing their applications with automated, data-driven decision-making. Retailers offering more intuitive and contextual search results can ensure their customers find what they're looking for by personalizing the relevance of their search results, enhancing satisfaction, and increasing the likelihood of successful transactions. The future of e-commerce search lies in harnessing the power of technologies like Atlas Vector Search , as it’s not only another vector search database, but also an extended product for the developer data platform , providing them with an integrated set of data and application services. For retailers, the message is clear: to offer unparalleled shopping experiences, embracing and integrating vector search functionalities with a performant and reliant platform that simplifies your data organization and storage is not just beneficial, it's essential. Learn more and discover How to Implement Databricks Workflows and Atlas Vector Search for Enhanced E-commerce Search Accuracy with our developer guide, and check out our GitHub repository explaining the full code for deploying an AI-Enhanced e-commerce search solution Head over to our quick-start guide to get started with Atlas Vector Search today.

October 11, 2023
Next →

MongoDB’s 2024 Year in Review

It’s hard to believe that another year is almost over! 2024 was a transformative year for MongoDB, and it was marked by both innovation and releases that further our commitment to empowering customers, developers, and partners worldwide. So without further ado, let’s dive into MongoDB’s 2024 highlights. We’ll also share our executive team’s predictions of what 2025 might have in store. A look back at 2024 MongoDB 8.0: The most performant version of MongoDB ever In October we released MongoDB 8.0 , the fastest, most resilient, secure, and reliable version of MongoDB yet. Architectural optimizations in MongoDB 8.0 have significantly improved the database’s performance, with 36% faster reads and 59% higher throughput for updates. Our new architecture also makes horizontal scaling cheaper and faster. Finally, working with encrypted data is easier than ever, thanks to the addition of range queries in Queryable Encryption (which allows customers to encrypt, store, and perform queries directly on data). Whether you’re a startup building your first app, or you’re a global enterprise managing mission-critical workloads, MongoDB 8.0 offers unmatched power and flexibility, solidifying MongoDB’s place as the world’s most popular document database. Learn more about what makes 8.0 the best version of MongoDB ever on the MongoDB 8.0 page . Delivering customer value with the MongoDB AI Applications Program AI applications have become a cornerstone of modern software, and MongoDB is committed to equipping customers with the technology, tools, and support they need to succeed on their AI journey. That’s why we launched the MongoDB AI Applications Program (MAAP) in 2024, a comprehensive program designed to accelerate the development of AI applications. By offering customers resources like access to AI specialists, an ecosystem of leading AI and tech companies, and AI architectural best practices supported by integrated services, MAAP helps solve customers’ most pressing business challenges, unlocks competitive advantages, and accelerates time to value for AI investments. Overall, MAAP’s aim is to set customers on the path to AI success. Visit the MongoDB AI Applications Program page or watch our session from AWS re:Invent to learn more! Advancing AI with MongoDB Atlas Vector Search In 2024, MongoDB further cemented its role in the AI space with enhancements to MongoDB Atlas Vector Search . Recognized in 2024 (for the second consecutive year!) as one of the most loved vector databases , MongoDB continues to provide a scalable, unified, and secure platform for building cutting-edge AI use cases. Recent advancements like vector quantization in Atlas Vector Search help deliver even more value to our customers, enabling them to scale applications to billions of vectors at a lower cost. Head over to our Atlas Vector Search quick start guide to get started with Atlas Vector Search today, or visit our AI resources hub to learn more about how MongoDB can power AI applications. Search Nodes: Performance at scale Search functionality is indispensable in modern applications, and with Atlas Search Nodes, organizations can now optimize their search workloads like never before. By providing dedicated infrastructure for Atlas Search and Vector Search workloads, Search Nodes ensure high performance (e.g., a 40–60% decrease in query times), scalability, and reliability, even for the most demanding use cases. As of this year , Search Nodes are generally available across AWS, Google Cloud, and Microsoft Azure. This milestone underscores MongoDB’s commitment to delivering powerful solutions that scale alongside our customers’ needs. To learn more about Search Nodes, check out our documentation or watch our tutorial . Looking ahead: MongoDB’s 2025 predictions After the excitement of the past few years, 2025 will be defined by ensuring that technology investments deliver tangible value. Organizations remain excited about the potential AI and emerging technologies hold to solve real business challenges, but are increasingly focused on maintaining a return on investment. “Enterprises need to innovate faster than ever, but speed is no longer the only measure of success. Increasingly, organizations are laser-focused on ensuring that their technology investments directly address critical business challenges and provide clear ROI and competitive advantage—whether it’s optimizing supply chains, delivering hyper-personalized customer experiences, or scaling operations efficiently,” said Sahir Azam, Chief Product Officer at MongoDB. “In 2025, I expect to see organizations make significant strides in driving this innovation and efficiency by applying AI to more production use cases and by maturing the way they leverage their data to build compelling and differentiated customer experiences.” Indeed, we expect to see organizations make more strategic investments in emerging technologies like gen AI—innovating with a sharp focus on solving business challenges. “In 2025, we can expect the focus to shift from ‘what AI can do’ to ‘what AI should do,’ moving beyond the hype to a clearer understanding of where AI can provide real value and where human judgment is still irreplaceable,” said Tara Hernandez, VP of Developer Productivity at MongoDB. “As we advance, I think we’ll see organizations begin to adopt more selective, careful applications of AI, particularly in areas where stakes are high, such as healthcare, finance, and public safety. A refined approach to AI development will be essential—not only for producing quality results but also to build trust, ensuring these tools genuinely support human goals rather than undermining them.” With more capable, accessible application development tools and customer-focused programs like MAAP at developers’ fingertips, 2025 is an opportunity to make a data-driven impact faster than ever before. "Right now, organizations have an opportunity to leverage their data to reimagine how they do business, to more effectively adapt to a changing world, and to revolutionize our quality of life,” said Andrew Davidson, SVP of Products at MongoDB. “By harnessing our latest technologies, developers can build a foundation for a transformative future." Head over to our updates page to learn more about the new releases and updates from MongoDB in 2024. Keep an eye on our events page to learn what's to come from MongoDB in 2025!

December 19, 2024