2

Curious about your motivation to move from Atlas to EC2 self managed… managing your own instance can come with hidden costs… and management headaches… but it is definitely possible. Here’s how I’d start to think about doing this…

  1. Getting the Data Over: First, you’ll want to use mongodump to create a backup of your data from Atlas. Then, you can restore it onto your EC2 instance with mongorestore. Here’s how that looks:
  • Dump from Atlas:
mongodump --uri "<your Atlas connection string>" --gzip --archive=backup.gz
  • Restore to EC2: After transferring the dump file to your EC2 instance, you can use mongorestore to load it into your self-hosted setup.
mongorestore --gzip --archive=backup.gz --uri "mongodb://<EC2 instance connection string>"
  1. Keeping Things in Sync: Depending on your update frequency… and the app in place… you can use MongoDB Change Streams. Change Streams allow you to watch your collections in real time and capture any inserts, updates, or deletes. You could set up a process that listens for changes in Atlas and applies those changes to your EC2 instance. Another option to explore is Atlas Triggers, which can fire off scripts/actions based on changes in your data (for example, docs inserted or modified). You could use a trigger to send updates to your EC2 instance. Depending on your needs, you could build a simple process that continuously listens for changes and updates the EC2 instance accordingly.

Not sure what language or frameworks you’re using but here’s some pseudo-code in python leveraging changestreams.

from pymongo import MongoClient
from pymongo.errors import ConnectionFailure
import pprint

# Replace these connection strings with your actual Atlas and EC2 MongoDB connection strings
ATLAS_URI = "mongodb+srv://<atlas-username>:<password>@<atlas-cluster-url>/<database>?retryWrites=true&w=majority"
EC2_URI = "mongodb://<ec2-username>:<password>@<ec2-instance-url>:27017/<database>"

# Set up MongoDB clients for Atlas and EC2 instances
atlas_client = MongoClient(ATLAS_URI)
ec2_client = MongoClient(EC2_URI)

# Specify the database and collection to sync
db_name = "<database>"
collection_name = "<collection>"

atlas_db = atlas_client[db_name]
ec2_db = ec2_client[db_name]

atlas_collection = atlas_db[collection_name]
ec2_collection = ec2_db[collection_name]

# Function to sync changes from Atlas to EC2
def sync_changes(change):
    operation_type = change["operationType"]
    document = change["fullDocument"]

    if operation_type == "insert":
        # Insert the document into the EC2 collection
        ec2_collection.insert_one(document)
        print(f"Document inserted in EC2: {document}")

    elif operation_type == "update":
        # Update the document in the EC2 collection
        document_id = change["documentKey"]["_id"]
        updated_fields = change["updateDescription"]["updatedFields"]
        ec2_collection.update_one({"_id": document_id}, {"$set": updated_fields})
        print(f"Document updated in EC2: {updated_fields}")

    elif operation_type == "delete":
        # Delete the document from the EC2 collection
        document_id = change["documentKey"]["_id"]
        ec2_collection.delete_one({"_id": document_id})
        print(f"Document deleted from EC2: {document_id}")

# Watch the Atlas collection for changes
try:
    with atlas_collection.watch() as stream:
        print("Listening for changes in Atlas...")
        for change in stream:
            pprint.pprint(change)
            sync_changes(change)

except ConnectionFailure as e:
    print(f"Error connecting to MongoDB: {e}")

Hope this helps… let us know how you make out.

3 Likes