Real-time Data Architectures With MongoDB Cloud Manager and Verizon 5G Edge
.png&w=3840&q=75)
Ado Kukic, Robert Belson8 min read • Published Aug 12, 2021 • Updated Aug 28, 2024
Rate this tutorial
The network edge has been one of the most explosive cloud computing opportunities in recent years. As mobile contactless experiences become the norm and as businesses move ever-faster to digital platforms and services, edge computing is positioned as a faster, cheaper, and more reliable alternative for data processing and compute at scale.
While mobile devices continue to increase their hardware capabilities with built-in GPUs, custom chipsets, and more storage, even the most cutting-edge devices will suffer the same fundamental problem: each device serves as a single point of failure and, thus, cannot effectively serve as a persistent data storage layer. Said differently, wouldn’t it be nice to have the high-availability of the cloud but with the topological distance to your end users of the smartphone?
Mobile edge computing promises to precisely address this problem—bringing low latency compute to the edge of networks with the high-availability and scale of cloud computing. Through Verizon 5G Edge with AWS Wavelength, we saw the opportunity to explore how to take existing compute-intensive workflows and overlay a data persistence layer with MongoDB, utilizing the MongoDB Atlas management platform, to enable ultra-immersive experiences with personalized experience—reliant on existing database structures in the parent region with the seamlessness to extend to the network edge.
In this article, learn how Verizon and MongoDB teamed up to deliver on this vision, a quick Getting Started guide to build your first MongoDB application at the edge, and advanced architectures for those proficient with MongoDB.
Let’s get started!
Through Verizon 5G Edge, AWS developers can now deploy parts of their application that require low latency at the edge of 4G and 5G networks using the same AWS APIs, tools, and functionality they use today, while seamlessly connecting back to the rest of their application and the full range of cloud services running in an AWS Region. By embedding AWS compute and storage services at the edge of the network, use cases such as ML inference, real-time video streaming, remote video production, and game streaming can be rapidly accelerated.
However, for many of these use cases, a persistent storage layer is required that extends beyond the native storage capabilities of AWS Wavelength—namely Elastic Block Storage (EBS) volumes. However, using MongoDB Enterprise, developers can leverage the underlying compute (i.e.,. EC2 instances) at the edge to deploy MongoDB clusters either a) as standalone clusters or b) highly available replica sets that can synchronize data seamlessly.
MongoDB is a general purpose, document-based, distributed database built for modern application developers. With MongoDB Atlas, developers can get up and running even faster with fully managed MongoDB databases deployed across all major cloud providers.
While MongoDB Atlas today does not support deployments within Wavelength Zones, MongoDB Cloud Manager can automate, monitor, and back up your MongoDB infrastructure. Cloud Manager Automation enables you to configure and maintain MongoDB nodes and clusters, whereby MongoDB Agents running on each MongoDB host can maintain your MongoDB deployments. In this example, we’ll start with a fairly simple architecture highlighting the relationship between Wavelength Zones (the edge) and the Parent Region (core cloud):
Just like any other architecture, we’ll begin with a VPC consisting of two subnets. Instead of one public subnet and one private subnet, we’ll have one public subnet and one carrier subnet —a new way to describe subnets exposed within Wavelength Zones to the mobile network only.
- Public Subnet: Within the us-west-2 Oregon region, we launched a subnet in us-west-2a availability zone consisting of a single EC2 instance with a public IP address. From a routing perspective, we attached an Internet Gateway to the VPC to provide outbound connectivity and attached the Internet Gateway as the default route (0.0.0.0/0) to the subnet’s associated route table.
- Carrier Subnet: Also within the us-west-2 Oregon region, our second subnet is in the San Francisco Wavelength Zone (us-west-2-wl1-sfo-wlz-1) —an edge data center within the Verizon carrier network but part of the us-west-2 region. In this subnet, we also deploy a single EC2 instance, this time with a carrier IP address—a carrier network-facing IP address exposed to Verizon mobile devices. From a routing perspective, we attached a Carrier Gateway to the VPC to provide outbound connectivity and attached the Carrier Gateway as the default route (0.0.0.0/0) to the subnet’s associated route table.
Next, let’s configure the EC2 instance in the parent region. Once you get the IP address (54.68.26.68) of the launched EC2 instance, SSH into the instance itself and begin to download the MongoDB agent.
1 ssh -i "mongovz.pem" ec2-user@ec2-54-68-26-68.us-west-2.compute.amazonaws.com
Once you are in, download and install the packages required for the MongoDB MMS Automation Agent. Run the following command:
1 sudo yum install cyrus-sasl cyrus-sasl-gssapi \ 2 cyrus-sasl-plain krb5-libs libcurl \ 3 lm_sensors-libs net-snmp net-snmp-agent-libs \ 4 openldap openssl tcp_wrappers-libs xz-libs
Once within the instance, download the MongoDB MMS Automation Agent, and install the agent using the RPM package manager.
1 curl -OL https://cloud.mongodb.com/download/agent/automation/mongodb-mms-automation-agent-manager-10.30.1.6889-1.x86_64.rhel7.rpm 2 3 sudo rpm -U mongodb-mms-automation-agent-manager-10.30.1.6889-1.x86_64.rhel7.rpm
Next, navigate to the /etc/mongodb-mms/ and edit the automation-agent.config file to include your MongoDB Cloud Manager API Key. To create a key, head over to MongoDB Atlas at https://mongodb.prakticum-team.ru/atlas and either login to an existing account, or sign up for a new free account.
Once you are logged in, create a new organization, and for the cloud service, be sure to select Cloud Manager.
With your organization created, next we’ll create a new Project. When creating a new project, you may be asked to select a cloud service, and you’ll choose Cloud Manager again.
Next, you’ll name your project. You can select any name you like, we’ll go with Verizon for our project name. After you give your project a name, you will be given a prompt to invite others to the project. You can skip this step for now as you can always add additional users in the future.
Finally, you are ready to deploy MongoDB to your environment using Cloud Manager. With Cloud Manager, you can deploy both standalone instances as well as Replica Sets of MongoDB. Since we want high availability, we’ll deploy a replica set.
Clicking on the New Replica Set button will bring us to the user interface to configure our replica set. At this point, we’ll probably get a message saying that no servers were detected, and that’s fine since we haven’t started our MongoDB Agents yet.
Click on the “see instructions” link to get more details on how to install the MongoDB Agent. On the modal that pops up, it will have familiar instructions that we’re already following, but it will also have two pieces of information that we’ll need. The mmsApiKey and mmsGroupId will be displayed here and you’ll likely have to click the Generate Key button to generate a new mmsAPIKey which will be automatically populated. Make note of these mmsGroupId and mmsApiKey values as we’ll need when configuring our MongoDB Agents next.
Head back to your terminal for the EC2 instance and navigate to the /etc/mongodb-mms/ and edit the automation-agent.config file to include your MongoDB Cloud Manager API Key.
In this example, we edited the mmsApiKey and mmsGroupId variables. From there, we’ll create the data directory and start our MongoDB agent!
1 sudo mkdir -p /data 2 sudo chown mongod:mongod /data 3 sudo systemctl start mongodb-mms-automation-agent.service
Once you’ve completed these configuration steps, go ahead and do the same for your Wavelength Zone instance. Note that you will not be able to SSH directly to the instance’s Carrier IP (155.146.16.178/). Instead, you must use the parent region instance as a bastion host to “jump” onto the edge instance itself. To do so, find the private IP address of the edge instance (10.0.0.54) and, from the parent region instance, SSH into the second instance using the same key pair you used.
1 ssh -i "mongovz.pem" ec2-user@10.0.0.54
After completing configuration of the second instance, which follows the same instructions from above, it’s time for the fun part —launching the ReplicaSet on the Cloud Manager Console! The one thing to note for the replica set, since we’ll have three nodes, on the edge instance we’ll create a /data and /data2 directories to allow for two separate directories to host the individual nodes data. Head back over to https://mongodb.prakticum-team.ru/atlas and the Cloud Manager to complete setup.
Refresh the Create New Replica Set page and now since the MongoDB Agents are running you should see a lot of information pre-populated for you. Make sure that it matches what you’d expect and when you’re satisfied hit the Create Replica Set button.
Click on the “Create Replica Set” button to finalize the process.
Within a few minutes the replica set cluster will be deployed to the servers and your MongoDB cluster will be up and running.
With the replica set deployed, you should now be able to connect to your MongoDB cluster hosted on either the standard Us-West or Wavelength zone. To do this, you’ll need the public address for the cluster and the port as well as Authentication enabled in Cloud Manager. To enable Authentication, simply click on the Enabled/Disabled button underneath the Auth section of your replica set and you’ll be given a number of options to connect to the client. We’ll select Username/password.
Click Next, and the subsequent modal will have your username and password to connect to the cluster with.
You are all set. Next, let’s see how the MongoDB performs at the edge. We’ll test this by reading data from both our standard US-West node as well as the Wavelength zone and compare our results.
After laying out the architecture, we wanted to see the power of 5G Edge in action. To that end, we designed a very simple “race.” Over 1,000 trials we would read data from our MongoDB database, and timestamp each operation both from the client to the edge and to the parent region.
1 from pymongo import MongoClient 2 import time 3 client = MongoClient('155.146.144.134', 27017) 4 mydb = client["mydatabase"] 5 mycol = mydb["customers"] 6 mydict = { "name": "John", "address": "Highway 37" } 7 8 # Load dataset 9 for i in range(1000): 10 x = mycol.insert(mydict) 11 12 # Measure reads from Parent Region 13 edge_latency=[] 14 for i in range(1000): 15 t1=time.time() 16 y = mycol.find_one({"name":"John"}) 17 t2=time.time() 18 edge_latency.append(t2-t1) 19 20 print(sum(edge_latency)/len(edge_latency)) 21 22 client = MongoClient('52.42.129.138', 27017) 23 mydb = client["mydatabase"] 24 mycol = mydb["customers"] 25 mydict = { "name": "John", "address": "Highway 37" } 26 27 # Measure reads from Wavelength Region 28 edge_latency=[] 29 for i in range(1000): 30 t1=time.time() 31 y = mycol.find_one({"name":"John"}) 32 t2=time.time() 33 edge_latency.append(t2-t1) 34 35 print(sum(edge_latency)/len(edge_latency))
After running this experiment, we found that our MongoDB node at the edge performed over 40% faster than the parent region! But why was that the case?
Given that the Wavelength Zone nodes were deployed within the mobile network, packets never had to leave the Verizon network and incur the latency penalty of traversing through the public internet—prone to incremental jitter, loss, and latency. In our example, our 5G Ultra Wideband connected device in San Francisco had two options: connect to a local endpoint within the San Francisco mobile network or travel 500+ miles to a data center in Oregon. Thus, we validated the significant performance savings of MongoDB on Verizon 5G Edge relative to the next best alternative: deploying the same architecture in the core cloud.
While Verizon 5G Edge alone enables developers to build ultra-immersive applications, how can immersive applications become personalized and localized?
Enter MongoDB.
From real-time transaction processing, telemetry capture for your IoT application, or personalization using profile data for localized venue experiences, bringing MongoDB ReplicaSets to the edge allows you to maintain the low latency characteristics of your application without sacrificing access to user profile data, product catalogues, IoT telemetry, and more.
There’s no better time to start your edge enablement journey with Verizon 5G Edge and MongoDB. To learn more about Verizon 5G Edge, you can visit our developer resources page. If you have any questions about this blog post, find us in the MongoDB community.
In our next post, we will demonstrate how to build your first image classifier on 5G Edge using MongoDB to identify VIPs at your next sporting event, developer conference, or large-scale event.