Explore Developer Center's New Chatbot! MongoDB AI Chatbot can be accessed at the top of your navigation to answer all your MongoDB questions.

Join us at AWS re:Invent 2024! Learn how to use MongoDB for AI use cases.
MongoDB Developer
MongoDB
plus
Sign in to follow topics
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Productschevron-right
MongoDBchevron-right

How to Migrate Your Flask App From SQL To MongoDB

Muhammed Ali9 min read • Published Jun 28, 2024 • Updated Jun 28, 2024
MongoDBPython
FULL APPLICATION
Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Migrating a Flask application from SQL to MongoDB involves several steps, as the two database systems have different paradigms. This article will guide you through converting an existing Flask blog application that uses SQLAlchemy to Flask-PyMongo to integrate MongoDB with the Flask application. We will explain the changes made to each file and the reasons behind them.
Understanding MongoDB
MongoDB is a NoSQL database known for its document-style kind of structure and ease of use.
Key concepts
  • Collections: Analogous to tables in SQL databases, collections in MongoDB store documents. However, unlike tables, collections do not enforce a schema, meaning each document can have different fields.
  • Documents: This is the primary data structure in MongoDB, similar to rows in SQL databases, but more complex. Documents are JSON-like objects that can contain arrays and nested subdocuments.
  • BSON: Binary representation of JSON is used internally by MongoDB. It extends JSON with additional data types, such as Date and Binary.

Differences between MongoDB and SQL databases

FeatureSQLMongoDB
SchemaEnforces a fixed schema with predefined tables and columnsSchema-less, allowing for flexible and dynamic data models
Data StructureStores data in tables with rows and columnsStores data in collections of documents
RelationshipsUses foreign keys and joins to establish relationships between tablesEmbeds documents within other documents or references them, but lacks joins
Query LanguageUses Structured Query Language (SQL)Uses a rich, JSON-based query language
Planning the migration
Before migrating the Flask application from SQL to MongoDB, thorough planning is necessary to ensure a smooth transition and preserve data integrity.
Proper planning helps identify potential blockers and develop strategies to address them.

Steps required for migration

  • Set up MongoDB environment
  • Assess current SQL schema and relationships: Review the existing database schema, including tables, relationships, and data types. Identify complex queries and relationships that might require special handling in MongoDB.
  • Define the MongoDB schema: Design the MongoDB schema considering the document-oriented nature of the database. Map SQL tables and relationships to MongoDB collections and documents.
  • Backup existing data: Create a complete backup of the current SQL database to prevent data loss. Ensure backup processes are reliable and can be restored if needed.
  • Data transformation and migration strategy: Choose a migration tool or write custom scripts to handle data migration, as we will do in this article.
  • Update Flask application code: Modify Flask models to use MongoDB instead of SQLAlchemy. Update routes, forms, and templates to work with the new data structure. Test each module of the application to ensure compatibility with MongoDB.
  • Post-migration testing: Thoroughly test the application to ensure all functionalities are working as expected. Monitor performance and optimize queries and indexes in MongoDB.

Application features and structure

The application to be migrated is a Flask blog application with the following features:
  • Creating, editing, and viewing blog posts
  • Searching for blog posts
Setting up MongoDB for a Flask application
Before integrating MongoDB with a Flask application, the first step is to install MongoDB on your local machine. MongoDB provides detailed installation guides for various operating system versions.
Configuring MongoDB for Flask
After installing MongoDB, the next step is to configure it to work on your Flask application. This involves installing the Flask-PyMongo package and setting up the connection with your local mongo db server.
Flask-PyMongo is a Flask extension that makes it easy to use MongoDB instances in your Flask app.
Install it using pip:
1pip install Flask-PyMongo Flask
You can also install Flask if you haven’t already.

Original SQL-based Flask application

Here’s a brief overview of the files in our original Flask application:
  • models.py: Defines the Post model using SQLAlchemy with attributes for id, title, content, and date_posted
  • __init__.py: Initializes the Flask application and SQLAlchemy
  • forms.py: Contains form definitions using Flask-WTF
  • manage.py: Manages database migrations using Flask-Script and Flask-Migrate
  • routes.py: Contains the route definitions and logic for handling requests
What you will do is make a copy of the folder containing the old project. We will use the current file structure and just replace the current code with some new code.

Update models.py

Go to models.py and paste in the following code:
1from datetime import datetime
2from bson.objectid import ObjectId
3class Post:
4    def __init__(self, title, content, date_posted=None, _id=None):
5        self.title = title
6        self.content = content
7        self.date_posted = date_posted if date_posted else datetime.utcnow()
8        self._id = _id if _id else ObjectId()
9
10    def to_dict(self):
11        return {
12            "title": self.title,
13            "content": self.content,
14            "date_posted": self.date_posted,
15            "_id": self._id
16        }
17
18    @staticmethod
19    def from_dict(data):
20        return Post(
21            title=data.get('title'),
22            content=data.get('content'),
23            date_posted=data.get('date_posted'),
24            _id=data.get('_id')
25        )
26
27    def __repr__(self):
28        return f"Post('{self.title}', '{self.date_posted}')"
This code defines a Post class that models a blog post for use with a MongoDB database, utilizing PyMongo's BSON types. The class includes an initializer (__init__ method) that sets the title, content, date posted, and a unique identifier for each post. If the date posted or identifier is not provided during instantiation, it defaults to the current UTC time and a new ObjectId, respectively. The class also provides a to_dict method to convert an instance into a dictionary format suitable for MongoDB storage, and a from_dict static method to create a Post instance from a dictionary. Additionally, the __repr__ method is defined to offer a clear string representation of the post, useful for debugging and logging.
The use of this is for the handling of blog post data within a Flask web application that uses MongoDB for its database. Converting the Post instances to and from dictionary format allows easy interaction with MongoDB's document-oriented storage.

Update __init__.py

Paste the following code into your __init__.py file. The Flask application is initialized to use MongoDB through the Flask-PyMongo extension. The PyMongo instance (mongo) is set up with the Flask application which enables MongoDB support.  The application routes are imported to define the URL endpoints and their associated handlers.
1from flask import Flask
2from flask_pymongo import PyMongo
3from config import Config
4
5
6app = Flask(__name__)
7app.config.from_object(Config)
8mongo = PyMongo(app)
9
10
11from app import routes
Additionally, update the Config class in your config.py to include MongoDB settings:
1import os
2class Config:
3    SECRET_KEY = os.urandom(24)
4    MONGO_URI = 'mongodb://localhost:27017/blogdb'

Update manage.py

manage.py is used for database migrations in SQL, which isn't directly applicable to MongoDB so you can delete the file. If you need migrations, the Beanie ODM provides some support for migrations with MongoDB.

Update routes.py

1from flask import render_template, url_for, flash, redirect, request
2from app import app, mongo
3from bson.objectid import ObjectId
4from app.models import Post
5from app.forms import PostForm
6
7@app.route("/")
8@app.route("/home")
9def home():
10    """
11    Render the home page with a list of all posts.
12    """
13    posts_data = mongo.db.posts.find()
14    posts = [Post.from_dict(post) for post in posts_data]
15    return render_template("index.html", posts=posts)
16
17
18@app.route("/post/new", methods=["GET", "POST"])
19def new_post():
20    """
21    Handle the creation of a new post. Render the new post form and process the form submission.
22
23    """
24    form = PostForm()
25    if form.validate_on_submit():
26        post = Post(title=form.title.data, content=form.content.data)
27        mongo.db.posts.insert_one(post.to_dict())
28        flash("Your post has been created!", "success")
29        return redirect(url_for("home"))
30    return render_template("post.html", title="New Post", form=form)
31
32
33@app.route("/post/<post_id>")
34def post(post_id):
35    """
36    Display a single post by its ID.
37    """
38    post_data = mongo.db.posts.find_one_or_404({"_id": ObjectId(post_id)})
39    post = Post.from_dict(post_data)
40    return render_template("detail.html", post=post)
41
42
43@app.route("/post/<post_id>/edit", methods=["GET", "POST"])
44def edit_post(post_id):
45    """
46    Handle the editing of an existing post. Render the edit post form and process the form submission.
47    """
48    post_data = mongo.db.posts.find_one_or_404({"_id": ObjectId(post_id)})
49    post = Post.from_dict(post_data)
50    form = PostForm()
51    if form.validate_on_submit():
52        updated_post = {
53            "$set": {"title": form.title.data, "content": form.content.data}
54        }
55        mongo.db.posts.update_one({"_id": ObjectId(post_id)}, updated_post)
56        flash("Your post has been updated!", "success")
57        return redirect(url_for("post", post_id=post_id))
58    elif request.method == "GET":
59        form.title.data = post.title
60        form.content.data = post.content
61    return render_template("edit_post.html", title="Edit Post", form=form)
62
63
64@app.route("/post/<post_id>/delete", methods=["POST"])
65def delete_post(post_id):
66    """
67    Handle the deletion of a post by its ID.
68    """
69    mongo.db.posts.delete_one({"_id": ObjectId(post_id)})
70    flash("Your post has been deleted!", "success")
71    return redirect(url_for("home"))
72
73
74@app.route("/search", methods=["GET", "POST"])
75def search():
76    """
77    Handle the search functionality for posts. Search for posts by title or content.
78    """
79    query = request.args.get("query")
80    posts_data = mongo.db.posts.find(
81        {
82            "$or": [
83                {"title": {"$regex": query, "$options": "i"}},
84                {"content": {"$regex": query, "$options": "i"}},
85            ]
86        }
87    )
88    posts = [Post.from_dict(post) for post in posts_data]
89    return render_template("index.html", posts=posts)
To make the Flask application compatible with MongoDB, several modifications were made to the routes to interact with MongoDB instead of SQLAlchemy. Firstly, the Post class instances are now converted to dictionaries using the to_dict method for storage in MongoDB. The home route retrieves all posts from the MongoDB collection using mongo.db.posts.find(), converts them to Post instances using the from_dict method, and then renders them.
For creating new posts in the new_post route, the post data is inserted into MongoDB with mongo.db.posts.insert_one(post.to_dict()). For retrieving, editing, and deleting posts, routes, post, edit_post, and delete_post use MongoDB's query methods (find_one_or_404, update_one, and delete_one) with ObjectId to handle the unique identifiers. The search route performs a text search on titles and content using MongoDB's regex capabilities, ensuring posts matching the query are fetched and rendered.
Migrating the data
The migration process involves exporting data from the existing SQL database and importing it into MongoDB. We will start by exporting the data from SQLite. On GitHub, you will find the db file we will use in this tutorial.
If you are using my db file, run the following code to query the database to list all tables and determine their names.
1import sqlite3
2
3
4# Connect to the SQLite database
5db_path = 'relative/path/to/site.db'
6conn = sqlite3.connect(db_path)
7cursor = conn.cursor()
8
9# Query to get all table names
10cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
11tables = cursor.fetchall()
12
13# Close the connection
14conn.close()
15
16print(tables)
This Python code above connects to your existing SQLite database specified in the path db_path and retrieves the names of all tables within it, then closes the connection. The SQL query SELECT name FROM sqlite_master WHERE type='table'; is executed to fetch the names of all tables in the database. You should have post as your output.
After that, run the following code to export the data to a JSON file.
1import sqlite3
2import json
3
4# Connect to the SQLite database
5db_path = 'relative/path/to/site.db'
6conn = sqlite3.connect(db_path)
7cursor = conn.cursor()
8
9# Export data from the 'post' table
10cursor.execute("SELECT * FROM post")
11rows = cursor.fetchall()
12
13# Get the column names
14column_names = [description[0] for description in cursor.description]
15
16# Convert to list of dictionaries
17data = [dict(zip(column_names, row)) for row in rows]
18
19# Save to a JSON file
20with open('post.json', 'w') as f:
21    json.dump(data, f, indent=4)
22
23# Close the connection
24conn.close()
This Python code connects to your SQLite database, extracts all data from the post table, converts it to a JSON format, and saves it to a file.  A cursor object is created from this connection to execute SQL commands. The SQL query SELECT * FROM post is used to fetch all rows from the post table, and the results are stored in the rows variable. The variable column_names is used to extract the column names from the cursor's description attribute. Each row is then combined with the column names to create a list of dictionaries, where each dictionary represents a row of the table. And finally, the list is saved as a JSON file named post.json using the json.dump() function.
After running, you will then be provided with a JSON file (post.json) that looks like this:
1[
2    {
3        "id": 1,
4        "title": " Superworm (Zophobas morio)",
5        "content": "Superworms, scientifically known as Zophobas morio, are the larval stage of a darkling beetle species...",
6        "date_posted": "2024-05-14 18:03:43.219562"
7    },
8    {
9        "id": 2,
10        "title": "MongoDB article",
11        "content": "MongoDB is a widely-used NoSQL database known for its flexibility...",
12        "date_posted": "2024-05-14 18:04:57.884181"
13    },
14    {
15        "id": 3,
16        "title": "Unique Techniques in Farming",
17        "content": "Modern farming has evolved in various ways…",
18        "date_posted": "2024-05-14 18:07:12.665001"
19    },
20    {
21        "id": 4,
22        "title": " Examples of NoSQL Databases",
23        "content": "NoSQL databases have improved the way...",
24        "date_posted": "2024-05-14 18:08:28.759601"
25    },
26    {
27        "id": 5,
28        "title": "d",
29        "content": "d",
30        "date_posted": "2024-05-18 18:06:59.767954"
31    }
32]

Importing data into MongoDB

You can now import this JSON file into your MongoDB database. Here's a script to help you with that:
1from pymongo import MongoClient
2import json
3from datetime import datetime
4
5def import_json_to_mongo(db_name, collection_name, json_path):
6    client = MongoClient("mongodb://localhost:27017/")
7    db = client[db_name]
8
9    def convert_date(data):
10        # Update 'date_posted' field to datetime object
11        for item in data:
12            if 'date_posted' in item:
13                item['date_posted'] = datetime.strptime(item['date_posted'], '%Y-%m-%d %H:%M:%S.%f')
14        return data
15
16    with open(json_path, 'r') as f:
17        data = json.load(f)
18        data = convert_date(data)
19        db[collection_name].insert_many(data)
20
21    client.close()
22
23if __name__ == "__main__":
24    db_name = 'blogdb'
25    collection_name = 'posts'
26    json_path = 'flask-mongo/post.json'
27    import_json_to_mongo(db_name, collection_name, json_path)
The connection string mongodb://localhost:27017/blogdb connects to your MongoDB instance running on localhost at port 27017 and uses the "blogdb" database. If the specified database does not exist, MongoDB will create it automatically when you first store data.

Update the templates

For the templates, you only need to change the variable for id. You will change all instances of post_id=post.id to post_id=post._id.
You can now run the app with the following command:
1python run.py
Your application will be running on http://127.0.0.1:5000/home.
the new app with MongoDB
You can find the complete application on GitHub.
Conclusion
Migrating a Flask application from SQL to MongoDB involves a deep understanding of the differences between both database systems and then planning to ensure a smooth transition. This article provided a step-by-step guide on how to convert an existing Flask blog application from using SQLAlchemy to using Flask-PyMongo to integrate MongoDB with the Flask application. We focused mainly on covering the necessary changes to each file. In a real-world scenario, there would be a need to construct a document schema from a tabular schema with joins and several queries. Even though this is a simple application, this tutorial covers the necessary steps to have a successful migration.
Want to continue the conversation? Head to the MongoDB Developer Community next.
Top Comments in Forums
There are no comments on this article yet.
Start the Conversation

Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Article

Entangled: A Story of Data Re-modeling and 10x Storage Reduction


Dec 14, 2023 | 5 min read
Article

Building a Quarkus Application to Perform MongoDB Vector Search


Oct 07, 2024 | 9 min read
Article

The Cost of Not Knowing MongoDB


Nov 11, 2024 | 23 min read
Tutorial

Trader Joe's Fall Faves Party Planner With Playwright, LlamaIndex, and MongoDB Atlas Vector Search


Nov 12, 2024 | 11 min read
Table of Contents