PeerIslands Cosmos DB Migrator Tool to MongoDB Atlas on Google Cloud

When you’re in the midst of innovating, the last thing you want to worry about is infrastructure. Whether you’re looking to streamline inventory management or reimagine marketing, you need applications that can scale fast and maintain high availability. That’s where MongoDB Atlas on Google Cloud comes in.

With MongoDB Atlas’ general-purpose, document-based database, users can free themselves from the hassle of database management, and give back precious time to developers to focus on innovation. Combine these benefits with Google Cloud’s cloud computing power, high availability, and ability to integrate with tools like BigQuery, Dataflow, Dataproc and more, and it’s hard to find a comparable joint solution. In fact, many current Microsoft Azure Cosmos DB users are now considering making the move to MongoDB.

Microsoft’s Cosmos DB only supports single partition transactions, has no schema governance and forces developers to work with five different APIs to deliver full application functionality. Conversely, MongoDB Atlas on Google Cloud supports distributed multi-document ACID transactions, includes schema governance, and offers integrated full-text search, auto-archiving, data lakes, and edge-to-cloud data sync.

The following blog illustrates how PeerIslands’ Cosmos DB Migrator tool can help users move from Cosmos DB to MongoDB Atlas on Google Cloud.

Why PeerIslands

PeerIslands is an enterprise-class digital transformation company composed of a team of polyglots who are comfortable across multiple technologies and cloud platforms. As a services firm, PeerIslands is focused on helping customers with both cloud-native development and application transformation.

With best-in-the-industry talent, PeerIslands has been working with the MongoDB team to build a suite of solutions around two key objectives:

  1. For a customer evaluating MongoDB, how can we rapidly address common questions?

  2. Once a customer has chosen MongoDB, how can we reduce time to value by rapidly migrating workloads to MongoDB?

With this in mind, PeerIslands developed a suite of tools around schema generation, understanding MongoDB query performance, as well as helping customers understand code changes required for upgrading MongoDB versions. In terms of workload migrations, PeerIslands developed solutions for both homogenous and heterogenous migrations. The company is also contributing to the open source community with a mobile app for enabling MongoDB admins to manage Atlas on the go.

PeerIslands' Cosmos DB migration use case

The current approach for migrating data from Cosmos DB to MongoDB is to use MongoDB dump and restore. But there are several problems with this approach.

  1. It’s fully manual and CLI-based which creates a poor user experience and requires technical resources even for simple migrations.

  2. There’s a lack of change capture capability which requires downtime during the duration of migration. For large Cosmos DB migrations, this causes significant issues. The team is also under pressure to deliver the entire migration in a short period of time.

  3. Migrations often get delayed as customers have difficulty identifying the right migration window.

The Cosmos to MongoDB tool is a “Live Migrate” like tool that helps perform one-time migrations and change data capture from Cosmos DB (MongoDB model) to MongoDB Atlas and minimizes downtime requirements associated with migrations. The tool is fully GUI-based and nearly everything is automated. All the tasks for infrastructure provisioning, dump & restore, change stream listeners and processors have all been automated with a graphical user interface (GUI).

The Cosmos to Mongo migration tool uses native MongoDB tools and the performance is similar to native tools. For change capture, we leverage the native MongoDB change stream APIs.

A high level view of the solution is provided in figure 1 below:

Solution Map
Figure 1: Solution Map

Migration steps:

  1. Migration configuration: Provide the name of the migration task, source Cosmos DB details, and target MongoDB details. The tool supports key vault integration as well.

  2. Migration infrastructure provisioning: Provide migration infrastructure details required for creating the VM (Virtual Machine) including location, type of VM instance, etc.

  3. Migration execution: Allow for automation of the migration once the configuration is complete. The migration is executed in 3 steps: backup, restore and change event processing. As a user, you can initiate the backup process. The change event listener is started in parallel with the backup process and captures all the changes. Once the backup is complete, the user can restore the initial data and then perform change event processing to apply all the changes to MongoDB.

  4. Migration validation: The tool also provides facilities for validating the migration. Users can view the total number of documents on both source Cosmos DB collection and target MongoDB collection. They can also compare random documents picked up from Cosmos DB and MongoDB side by side and validate whether the data elements have been loaded correctly.

For a more detailed demo and description of events, watch the following video:

Migrating to a new database can feel daunting at first, but PeerIslands Cosmos DB migrator makes it easy. Major concerns like delays and downtime are eliminated from the process, helping you run your business smoothly and reap the benefits of MongoDB more quickly. And with PeerIslands suite of tools, you can rapidly address MongoDB-specific questions and accelerate time to value.

Reach out today to get started

← Previous

Uncover hidden truths with Interactive Filtering

Atlas Charts has been updated with greater dashboard interactivity! With Interactive Filtering, users can click on charts to highlight and filter their dashboard data to their heart’s content. Interactive dashboards support better decision making Data visualisations are meant to be more than just a pretty collection of charts. They are a valuable decision making tool. At MongoDB, we believe that users should not only be able to manipulate their data but explore the significance of data changes in their dashboards to aid in making faster decisions. Last year, we added the ability to interact with charts and dashboards using Dashboard Filtering . This year, we are taking it to the next level. We are very excited to release ‘Interactive Filtering’, allowing users to interact with a particular chart to filter data in other charts. How does Interactive Filtering work? So let’s use an example to understand how this feature works: We have built a dashboard with multiple charts using the ‘Movies’ sample data source and added dashboard filters for the ‘languages’ and ‘rated’ fields. Previously, you could only interact with these filters by checking or unchecking options in the filter pane. While that option remains available, Interactive Filtering allows you to explore your data directly from the charts. So when you click on chart elements, such as bars, category labels or legend entries, it is a faster way to filter your dashboard. Chart elements are clickable if they are built with fields that are linked to dashboard filters. When filter fields are used on a chart’s category channels, we now highlight the requested data point but continue to show the other chart data, making it easy to see how the values compare. Looking at the dashboard below, if you want to know how many Spanish movies are released or the average metacritic score of Spanish movies, then you can highlight ‘Spanish’ by clicking the ‘Spanish’ bar or the ‘Spanish’ label in the ‘Movies by Language’ chart. The other charts in the dashboard get filtered to show only data based on the Spanish language. You can also use the ‘languages’ dashboard filter to achieve the same result. Interact by clicking directly in your dashboard Interact by modifying your dashboard filters How do you know what Charts are highlighted vs filtered? We now show a handy filter and highlight icon to indicate whether a chart is being filtered, highlighted, or both. What if you want to go back to the previous state before you started clicking? Simple, just click anywhere on the empty area of a chart and the changes are rolled back. If you were to use the dashboard filter pane to highlight or filter the dashboards, we save the state. Think of the dashboard filter pane changes as a persistent state and click changes on a chart as a temporary state. Depending on the dashboard filters selected, you also have the option to make a chart in the dashboard filter or highlight, filter only or ignore the dashboard filters completely. Try Interactive Filtering today With ‘Interactive Filtering’ the possibilities to manipulate and explore your data are limitless. Try the new highlighting functionality today by creating dashboard filters for your key fields and uncover the hidden answers in your data. New to Charts? You can start now for free by signing up for MongoDB Atlas , deploying a free tier cluster and activating Charts. Have an idea on how we can make MongoDB Charts better? Feel free to leave an idea at the MongoDB Feedback Engine .

December 9, 2021
Next →

Customer Service Expert Wati.io Scales Up on MongoDB

Wati.io is a software-as-a-service (SaaS) platform that empowers businesses to develop conversation-driven strategies to boost growth. Founded by CEO Ken Yeung in 2019, Wati started as a chatbot solution for large enterprises, such as banks and insurance companies. However, over time, Yeung and his team noticed a growing need among small and medium-sized businesses (SMBs) to manage customer conversations more effectively. To address this need, Wati used MongoDB Atlas and built a solution based on the WhatsApp Business API. It enables businesses to manage and personalize conversations with customers, automate responses, improve commerce functions, and enhance customer engagement. Speaking at MongoDB.local Hong Kong in September 2024, Yeung said, “The current solutions on the market today are not good enough. Especially for SMBs [that] don’t have the same level of resources as enterprises to deal with the number of conversations and messages that need to be handled every day.” Supporting scale: From MongoDB Community Edition to MongoDB Atlas “From the beginning, we relied on MongoDB to handle high volumes of messaging data and enable businesses to manage and scale their customer interactions efficiently,” said Yeung. Wati originally used MongoDB Community Edition , as the company saw the benefits of a NoSQL model from the beginning. As the company grew, it realized it needed a scalable infrastructure, so Wati transitioned to MongoDB Atlas. “When we started reaching the 2 billion record threshold, we started having some issues. Our system slowed down, and we were not able to scale it,” said Yeung. Atlas has now become an essential part of Wati’s infrastructure, helping the company store and process millions of messages each month for over 10,000 customers in 165 countries. “Transitioning to a new platform—MongoDB Atlas—seamlessly was critical because our messaging system needs to be on 24/7,” said Yeung. Wati collaborated closely with the MongoDB Professional Services and MongoDB Support teams, and in a few months it was able to rearchitect the deployment and data model for future growth and demand. The work included optimizing Wati’s database by breaking it down into clusters. Wati then focused on extracting connections, such as conversations, and dividing and categorizing data within the clusters—for example, qualifying data as cold or hot based on the read and write frequencies. This architecture underpins the platform’s core features, including automated customer engagement, lead qualification, and sales management. Deepening search capabilities with MongoDB Atlas Search For Wati’s customers, the ability to search through conversation histories and company documents to retrieve valuable information is a key function. This often requires searching through millions of records to rapidly find answers so that they can respond to customers in real-time. By using MongoDB Atlas Search , Wati improved its search capabilities, ultimately helping its business customers perform more advanced analytics and improve their customer service agents’ efficiency and customer reporting. “[MongoDB] Atlas Search is really helpful because we don’t have to do a lot of technical integration, and minimal programming is required,” said Yeung. Looking ahead: Using AI and integrating more channels Wati expects to continue collaborating with MongoDB to add more features to its platform and keep innovating at speed. The company is currently exploring to build more AI capabilities of Wati KnowBot , as well as how it can expand its integration with other conversation platforms and channels such as Instagram and Facebook. To learn more about MongoDB Atlas, visit our product page . To get started with MongoDB Atlas Search, visit the Atlas Search product page .

November 25, 2024