Community

Karen Zhang

Community author

Nowadays, choosing the right database and deployment platform can make a significant difference in your application's performance and scalability. This tutorial will guide you through creating a Flask application that uses MongoDB Atlas and deploying it on Fly.io, a platform that makes deployment and scaling a breeze.

In this tutorial, we'll build a location-based discovery API using Flask and MongoDB Atlas. By combining Flask's simplicity with MongoDB's powerful geospatial features, we'll create a RESTful API that allows users to store, search, and discover locations. Whether you're building a travel app, a local business finder, or any location-aware service, this tutorial will show you how to leverage MongoDB's GIS capabilities for spatial queries, implement text search, and use aggregation pipelines for insights. We'll use MongoDB Atlas, a fully managed cloud database service, making it easy to deploy and scale our application, and Fly.io for hosting our Flask application in the cloud. All the code can be found on [GitHub](https://github.com/Karenzhang7717/flask-mongodb-fly/tree/main).

## 1. Set up the development environment

Before we dive into coding, let's set up our development environment:

1. Install Python (if you haven't already).
2. Create a new directory for your project:

   ```sh
   mkdir flask-mongodb-fly
   cd flask-mongodb-fly
   ```

3. Create and activate a virtual environment:

   ```sh
   python -m venv venv
   source venv/bin/activate  # On Windows, use `venv\Scripts\activate`
   ```

4. Install the required packages:

   ```sh
   pip install Flask pymongo gunicorn
   ```

   ![Installing required packages][image1]

5. Create a `requirements.txt` file:

   ```sh
   pip freeze > requirements.txt
   ```

## 2. Create a basic Flask application

Let's create a simple Flask application with routes for CRUD operations and a geospatial query. Create a file named `app.py` with the following content:

```python
import os
from flask import Flask, request, jsonify
from pymongo import MongoClient
from bson import json_util
import json

app = Flask(__name__)

# MongoDB Atlas connection string
mongodb_uri = os.environ.get("MONGODB_URI", "mongodb://localhost:27017/geoapp")
client = MongoClient(mongodb_uri)
db = client.get_database()

@app.route("/")
def hello_world():
    return "Hello, World\! This is a Flask & MongoDB app deployed on Fly.io"

@app.route("/locations", methods=["POST"])
def add_location():
    data = request.json
    result = db.locations.insert_one(data)
    return (jsonify({"message": "Location added successfully", "id": str(result.inserted_id)}), 201,)

@app.route("/locations", methods=["GET"])
def get_locations():
    locations = list(db.locations.find())
    return json.loads(json_util.dumps(locations))

@app.route("/nearby", methods=["GET"])
def find_nearby():
    lat = float(request.args.get("lat"))
    lon = float(request.args.get("lon"))
    max_distance = int(request.args.get("distance", 1000))  # Default to 1000 meters

    nearby = db.locations.find({
        "location": {
            "$near": {
                "$geometry": {
                    "type": "Point",
                    "coordinates": [lon, lat]
                },
                "$maxDistance": max_distance
            }
        }
    })

    return json.loads(json_util.dumps(list(nearby)))

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080) # change to your ip address
```

This application includes:

- A home route ("/") that returns a simple greeting.
- A route to add new locations ("/locations" - POST).
- A route to retrieve all locations ("/locations" - GET).
- A route to find nearby locations based on coordinates and distance ("/nearby" - GET).

## 3. Integrate MongoDB with Flask

To use MongoDB with our Flask application, we need to set up a MongoDB Atlas account and create a cluster:

1. Sign up for a [free MongoDB Atlas account](https://www.mongodb.com/cloud/atlas).
2. Create a new cluster (choose the free tier for testing).
3. In the Security tab, create a database user and whitelist your IP address.
   ![][image2]
   ![][image3]
4. In the Clusters tab, click "Connect" and choose "Connect your application." You need to use the Python driver. Since we already installed PyMongo in Step 4 while  setting up the development environment, we just need to add your connection string into the application code. Create an .env file in your project and add MONGODB_URI
as an environment variable. Then, copy and paste the connection string.
   ![][image4]
   ![][image5]
5. Replace `<password>` with your database user's password. This can be found in Security -> Database Access. The user should at least have read and write permission to the database.
   ![][image6]

## 4. Test the application locally

Before deploying, let's test our application locally:

1. Set the `MONGODB_URI` environment variable:

   ```sh
   export MONGODB_URI="your_mongodb_atlas_connection_string"
   ```
   Replace `your_mongodb_atlas_connection_string` with your actual connection string.

2. Run the Flask application:

   ```sh
   python app.py
   ```

3. Test the endpoints using URL:

   ```sh
   # Add a location
   curl -X POST http://localhost:8080/locations -H "Content-Type: application/json" -d '{"name": "Eiffel Tower", "location": {"type": "Point", "coordinates": [2.2945, 48.8584.}}'

   # Get all locations
   curl http://localhost:8080/locations

   # Find nearby locations
   curl "http://localhost:8080/nearby?lat=48.8584.lon=2.2945.distance=5000"
   ```

## 5. Deploy to Fly.io

Now, let's deploy our application to Fly.io:

1. Install the Fly CLI:

   ```sh
   curl -L https://fly.io/install.sh | sh
   ```

2. Sign up and log in to Fly.io:

   ```sh
   fly auth signup
   fly auth login
   ```

3. Set the MongoDB URI as a secret:

   ```sh
   fly secrets set MONGODB_URI="your_mongodb_atlas_connection_string"
   ```

   Replace `your_mongodb_atlas_connection_string` with your actual connection string.

4. Set up the configuration:

   ```sh
   fly launch
   ```

   `fly launch` is a command that initializes your application on Fly.io by creating a `fly.toml` configuration file and setting up the necessary infrastructure.

   The file.toml file will look something like this:

   ```toml
   # fly.toml app configuration file generated for flask-mongodb-fly-late-dew-6702 on 2024-11-18T15:45:10-07:00
   #
   # See https://fly.io/docs/reference/configuration/ for information about how to use this file.
   #

   app = 'your-app-name’'
   primary_region = 'sea'

   [build]

   [http_service]
   internal_port = 8080
   force_https = true
   auto_stop_machines = 'stop'
   auto_start_machines = true
   min_machines_running = 0
   processes = ['app']

   [[vm]]
   memory = '1gb'
   cpu_kind = 'shared'
   cpus = 1
   ```

5. Deploy your app:

   ```sh
   fly deploy
   ```

   ![][image7]

   After the app is successfully deployed, you can see it on Fly.io with the details:

   ![][image8]

   ![][image9]

   If we go to the URL, this is the “Hello, World\!” app that we just deployed.

   ![][image10]

## 6. Post-deployment steps

After deployment, let's verify that our application is working correctly:

1. Open your app in a browser:

   ```sh
   fly open
   ```

3. Test the endpoints using cURL:

   ```sh
   # Add a location

   curl -X POST https://flask-mongodb-fly-wild-sky-7177.fly.dev/locations -H 'Content-Type: application/json' -d '{"name": "Central Park", "location": {"type": "Point", "coordinates": [-73.965355, 40.782865.}, "description": "Famous park in Manhattan"}'
   ```

   ![][image11]

   ```sh
   # Get all locations
   curl https:flask-mongodb-fly-wild-sky-7177.fly.dev/locations
   ```

   ![][image12]

   ```sh
   # Find nearby locations
   curl "https://flask-mongodb-fly-wild-sky-7177.fly.dev/nearby?lat=40.782865.lon=-73.965355.distance=1000"

   ```

   It shows the Manhattan park that we just POST:
   `[{"_id":{"$oid":"6727b03c8fb4fcebcf186af9"},"description":"Famous park in Manhattan","location":{"coordinates":[-73.965355,40.782865.,"type":"Point"},"name":"Central Park"}]`
   
   ![][image13]

4. Monitor your app's logs:

   ```sh
   fly logs
   ```

## 7. Troubleshooting common deployment issues

If you encounter a 500 Internal Server Error or other issues:

1. Check your app's logs:

   ```sh
   fly logs
   ```

2. Verify your MongoDB connection string:

   ```sh
   fly secrets list
   ```

3. If needed, update your secrets:

   ```sh
   fly secrets set MONGODB_URI="your_updated_mongodb_uri"
   ```

4. Redeploy your application:

   ```sh
   fly deploy
   ```

## Conclusion

Through this tutorial, we've built a robust, location-based API that demonstrates the power of MongoDB Atlas's geospatial features combined with Flask's web framework capabilities. We've implemented spatial queries for finding nearby locations, text search for discovering places, and aggregation pipelines for generating insights. The application showcases real-world GIS features that can be used for location tracking.

By deploying on Fly.io and using MongoDB Atlas, we've created a scalable solution that can serve as a foundation for various location-based services. Whether you're building a travel application, a local business directory, or any location-aware service, you can extend this foundation by adding authentication, more complex queries, or additional features specific to your use case.

For questions, comments, or to see what other developers are building using MongoDB, join us in the [MongoDB Developer Community](https://www.mongodb.com/community/forums/).

[image1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt6d44d2f489e4d929/67489732799856185de22d96/image4.png
[image2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt78795bdda4b9fd29/674897332452f94fb20a6b0e/image5.png
[image3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltc8e63d25cee1513f/674897349457b56dd043894c/image10.png
[image4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blta218018a40179f86/6748973464eb634a08d7309f/image9.png
[image5]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltd6f2a9e8a89e9230/674897754d7c436a19f62f03/image11.png
[image6]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltdf53b2a9d688fd6b/67489733ef6d5a63b82281b5/image7.png
[image7]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blte560e307b92a0a99/674897334652421c82d34ef8/image2.png
[image8]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt9a3786607b32f50d/674897330da03c32e2b543f8/image6.png
[image9]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltc52114aad3043876/674897342eb39e4571afb8cf/image3.png
[image10]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt9286debce106c879/67489732a1696b5f2d7a0b2f/image1.png
[image11]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt7524449b74929004/6748973348603b65cc49ac9d/image8.png
[image12]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt54ca6a4e6e8a4c62/67489774aafcfcd481addd09/image13.png
[image13]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt75a12d0a478ba7fb/674897742781189bae5d6daf/image12.png

Learn how to deploy a Flask Application with MongoDB on Fly.io

How to Deploy a Flask Application With MongoDB on Fly.io

Anaiya Raisinghani

Associate Developer Advocate

Mark Smith

Senior Developer Advocate

In this article, we will be going through some of the best practices for getting started with Flask, so you too can build complex, customizable applications – easier! 

From using the right driver to encoding an ObjectID correctly in JSON, this article will set you up for success when using Flask and MongoDB. Read on to learn our suggested best practices!

## First, use the right driver
One of the most crucial steps is to ensure you’re using the correct driver. You have two choices of PyMongo or Motor. Both of these are recommended and developed by MongoDB. Pick [Motor](https://www.mongodb.com/docs/drivers/motor/) if you’re building a Flask application with asyncio routes, and choose [PyMongo](https://www.mongodb.com/docs/languages/python/pymongo-driver/current/) if you are building a more traditional non-async Flask application. 
 
To download PyMongo, just use this simple command:
```
pip install pymongo
```

Or you can install Motor using:
```
pip install motor
```
## Flask!
Flask is one of the most popular frameworks out there for Python developers. It integrates fantastically with MongoDB. Here are some tips and tricks to help you out when building anything from simple projects to highly complex web applications with the two platforms. 
## Keep your secrets a secret
We want to keep our sensitive information properly protected when we’re using Flask. While there are various ways of doing this, let’s go over two of the most secure methods for saving your environment variables and API keys. 

## Dotenv
The first way is through the Python library `dotenv`. This will include downloading the `python-dotenv` package via pip, ensuring that you’re in an activated virtual environment:

```
pip install python-dotenv
```
From here, you’ll create a `.env` file in the root of your project directory to contain any and all environment variables, such as your MongoDB connection string:

```
# MongoDB connection_string [help](https://www.mongodb.com/docs/manual/reference/connection-string/)
MYAPP_CONNECTION_STRING=mongodb_connection_string_here
```

It’s crucial to not commit this file! To ensure the file isn’t committed, you can add your `.env` file to your project’s `.gitignore` file or your global `.gitignore` file. Learn more about [keeping your `.env` file private](https://sebastiandedeyne.com/setting-up-a-global-gitignore-file/).

You can import `load_dotenv` from the `dotenv` package, using `from dotenv import load_dotenv`. Calling the `load_dotenv` function will then copy all of the configuration from your `.env` file into your Flask application's process. You can then use `app.config.from_prefixed_env()` to import any environments that start with a certain prefix into your Flask application. Some sample skeleton code is shown below:

```
from dotenv import load_dotenv


load_dotenv()
app = Flask(__name__)
app.config.from_prefixed_env(“MYAPP”)
client = MongoClient(app.config['CONNECTION_STRING'])
```

You can access your secrets using the name of the variable without the “MYAPP_” prefix. This is where the app loads configuration from any environment variables that begin with “MYAPP_”. In this case, a single configuration option is provided, CONNECTION_STRING. The `dotenv` library doesn't override any existing environment variables, so you now have the choice of storing config values in the `.env` file or overriding them by setting environment variables directly.

Flask's configuration is very flexible, and you can find out more about it from the [official documentation](https://flask.palletsprojects.com/en/2.3.x/config/).

## How do I connect to MongoDB?!
Once you’ve properly loaded your connection string, you can add in some code in the heart of your application, which is sometimes called the `app.py` file, to properly connect to the database and collection of your choosing, or to create a database and collection in the cluster of your choice. Let’s run through how to do this step with some skeleton code:


```
mongo_client = MongoClient(app.config[“CONNECTION_STRING”])


# access your database and collection from Atlas
database = mongo_client.get_database(“anaiyamovies”)
collection = database.get_collection(“movies”)


# instantiating new object with `__name__`
app = Flask(__name__)
```

What you need to know about the code above is that it doesn’t contact MongoDB at all. It won’t try to connect to your database cluster until you modify data in your cluster or obtain some data from your cluster. 

The database is named “anaiyamovies” while the collection is named “movies.”
Now, using either CRUD operations inside of our Flask application or simple MongoDB operators, we can write to our collection to add in documents, delete documents, change documents, or even update our docs! 


## How can I know if I’m actually connected to my MongoDB cluster?
Because PyMongo is designed to be lazy, it doesn't attempt to connect to your cluster when MongoClient is initialized. This can lead to surprising behavior when you eventually try to query your database, if it turns out that you can't connect. For this reason, it's good practice to validate your database connection when you create a MongoClient. There are various ways to check if you are properly connected to your MongoDB cluster when in the process of creating your Flask application, but one of the simplest ways is to send over a `ping` command and just check the response! Let’s go through how to do this. 



```
# our imports go here
from pymongo import MongoClient
app = Flask(__name__)
app.config.from_prefixed_env(“MYAPP”)


def init_mongo_client(app: Flask):
  try:
      connection_string = "connection string here with user + pass"
      mongo_client = MongoClient(connection_string)


      database = mongo_client['anaiyamovies']
      result = mongo_client.admin.command('ping')
    
      if int(result.get('ok')) == 1:
          print("Connected")
      else:
          raise Exception("Cluster ping returned OK != 1")
      app.mongo_client = mongo_client


init_mongo_client(app)


```
> If this function can't connect to your MongoDB cluster, this will raise an exception and fail, which is good because we want to make sure we are starting up Flask correctly. If everything's okay, it will add a `mongo_client` attribute to your Flask app that your routes can use to access your MongoDB cluster.

It’s always good practice to validate your database connection when your application starts up, so that you are aware of any problems with your deployment configuration at the first opportunity. 

By running a `ping` test, you can easily see if you’re properly connected to the cluster you want to work with!


## Encode an ObjectID correctly in JSON

By default, every document contains an ObjectId which is a type unique to BSON. This means they cannot be directly converted to JSON, so we need a workaround, such as encoding them as a string.

```
from bson import json_util
json_util.dumps(object_with_objectid)
```


```
from flask import Flask, Response
from bson import json_util
from typing import Mapping, Any

from dotenv import load_dotenv
from pymongo import MongoClient

load_dotenv()

app = Flask(__name__)
app.config.from_prefixed_env("MYAPP")

def init_mongodb_client(app: Flask):
	client = MongoClient(app.config [connection_string])
	app.db = client.get_database()

init_mongodb_client(app)


def bson_to_json(bson: Mapping[str, Any]) -> Response:
    return Response(
        json_util.dumps(bson),
        content_type="application/json",
    )


@app.get("/")
def index():
    return bson_to_json(app.db.find_one())
```



This code will handle your ObjectIds and convert them into a string format that is easier to use and deal with in your project. Bear in mind that [BSON contains other types](https://www.mongodb.com/docs/manual/reference/bson-types/) that cannot be directly converted to JSON. You may also want to consider those when writing your Flask application.  


## Remember to model your data correctly
When you’re using MongoDB, generally, it’s crucial to model your data correctly so it’s as efficient as possible when using various operations. To learn more about this, please check out our [Advanced Schema Design Patterns](https://learn.mongodb.com/courses/advanced-schema-design-patterns?_ga=2.184415077.259239060.1723227814-326144593.1661803453&_gac=1.58795743.1720620648.CjwKCAjw4ri0BhAvEiwA8oo6Fyn1eGEGRsezHwH9CHTNhhuvvQ-_BXnxlaWtvFh7tlYP4ah1REro-hoCC2QQAvD_BwE) course available in MongoDB University or our [Building With Patterns](https://www.mongodb.com/blog/post/building-with-patterns-a-summary) series.

## Conclusion
Hopefully, these tips and tricks are helpful for using Flask with MongoDB. Through this article, you’ve learned a variety of best practices that will set you up for success while integrating Flask with your MongoDB Atlas cluster and creating some fantastic applications. 

Questions? Want to share what you’re building? Head to the [MongoDB Developer Community](https://www.mongodb.com/community/forums/) next!


Learn some of the best practices for using Flask and MongoDB so you can set yourself up for success.

Best Practices for Using Flask and MongoDB

>This is the first part of a short series of blog posts called "Rewrite it in Rust (RiiR)." It's a tongue-in-cheek title for some posts that will investigate the similarities and differences between the same service written in Python with Flask, and Rust with Actix-Web.

This post will show how I built a RESTful API for a collection of cocktail recipes I just happen to have lying around. The aim is to show an API server with some complexity, so although it's a small example, it will cover important factors such as:

-   Data transformation between the database and a JSON representation.
-   Data validation.
-   Pagination.
-   Error-handling.

## Prerequisites

-   Python 3.8 or above
-   A MongoDB Atlas cluster. Follow the "[Get Started with Atlas](https://docs.atlas.mongodb.com/getting-started/)" guide to create your account and MongoDB cluster. Keep a note of your database username, password, and [connection string](https://docs.atlas.mongodb.com/tutorial/connect-to-your-cluster/#connect-to-your-atlas-cluster) as you will need those later.

This is an *advanced* guide, so it'll cover a whole bunch of different libraries which can be brought together to build a declarative Restful API server on top of MongoDB. I won't cover repeating patterns in the codebase, so if you want to build the whole thing, I recommend checking out the source code, which is all [on GitHub](https://github.com/mongodb-developer/rewrite-it-in-rust).

It won't cover the basics of Python, Flask, or MongoDB, so if that's what you're looking for, I recommend checking out the following resources before tackling this post:

-   [Think Python](https://greenteapress.com/wp/think-python-2e/)
-   [The Python & MongoDB Quickstart Series](https://developer.mongodb.com/quickstart/python-quickstart-crud/)
-   [Flask Tutorial](https://flask.palletsprojects.com/en/1.1.x/tutorial/)
-   [Pydantic Documentation](https://pydantic-docs.helpmanual.io/)

## Getting Started

Begin by cloning the sample code source [from GitHub](https://github.com/mongodb-developer/rewrite-it-in-rust). There are four top-level directories:

-   [actix-cocktail-api](https://github.com/mongodb-developer/rewrite-it-in-rust/tree/master/actix-cocktail-api): You can ignore this for now.
-   [data](https://github.com/mongodb-developer/rewrite-it-in-rust/tree/master/data): This contains an export of my cocktail data. You'll import this into your cluster in a moment.
-   [flask-cocktail-api](https://github.com/mongodb-developer/rewrite-it-in-rust/tree/master/flask-cocktail-api): The code for this blog post.
-   [test_scripts](https://github.com/mongodb-developer/rewrite-it-in-rust/tree/master/test_scripts): A few shell scripts that use curl to test the HTTP interface of the API server.

There are more details in the [GitHub repo](https://github.com/mongodb-developer/rewrite-it-in-rust/tree/master/flask-cocktail-api), but the basics are: Install the project with your virtualenv active:

``` shell
pip install -e .
```

Next, you should import the data into your cluster. Set the environment variable `$MONGO_URI` to your cluster URI. This environment variable will be used in a moment to import your data, and also by the Flask app. I use `direnv` to configure this, and put the following line in my `.envrc` file in my project's directory:

``` shell
export MONGO_URI="mongodb+srv://USERNAME:PASSW0RD@cluster0-abcde.azure.mongodb.net/cocktails?retryWrites=true&w=majority"
```

Note that your database must be called "cocktails," and the import will create a collection called "recipes." After checking that `$MONGO_URI` is set correctly, run the following command:

``` shell
mongoimport --uri "$MONGO_URI" --file ./recipes.json
```

Now you should be able to run the Flask app from the
`flask-cocktail-api` directory:

``` shell
FLASK_DEBUG=true FLASK_APP=cocktailapi flask run
```

(You can run `make run` if you prefer.)

Check the output to ensure it is happy with the configuration, and then in a different terminal window, run the `list_cocktails.sh` script in the `test_scripts` directory. It should print something like this:

``` json
{
    "_links": {
        "last": {
            "href": "http://localhost:5000/cocktails/?page=5"
        }, 
        "next": {
            "href": "http://localhost:5000/cocktails/?page=5"
        }, 
        "prev": {
            "href": "http://localhost:5000/cocktails/?page=3"
        }, 
        "self": {
            "href": "http://localhost:5000/cocktails/?page=4"
        }
    }, 
    "recipes": [
        {
            "_id": "5f7daa198ec9dfb536781b0d", 
            "date_added": null, 
            "date_updated": null, 
            "ingredients": [
            {
                "name": "Light rum", 
                "quantity": {
                "unit": "oz", 
                }
            }, 
            {
                "name": "Grapefruit juice", 
                "quantity": {
                "unit": "oz", 
                }
            }, 
            {
                "name": "Bitters", 
                "quantity": {
                "unit": "dash", 
                }
            }
            ], 
            "instructions": [
            "Pour all of the ingredients into an old-fashioned glass almost filled with ice cubes", 
            "Stir well."
            ], 
            "name": "Monkey Wrench", 
            "slug": "monkey-wrench"
        },
    ]
    ...
```

## Breaking it All Down

The code is divided into three submodules.

-   `__init__.py` contains all the Flask setup code, and defines all the HTTP routes.
-   `model.py` contains all the Pydantic model definitions.
-   `objectid.py` contains a Pydantic field definition that I stole from the [Beanie](https://github.com/roman-right/beanie) object-data mapper for MongoDB.

I mentioned earlier that this code makes use of several libraries:

-   [PyMongo](https://pymongo.readthedocs.io/en/stable/index.html) and [Flask-PyMongo](https://flask-pymongo.readthedocs.io/en/latest/) handle the connection to the database. Flask-PyMongo specifically wraps the database collection object to provide a convenient`find_one_or_404` method.
-   [Pydantic](https://pydantic-docs.helpmanual.io/usage/exporting_models/) manages data validation, and some aspects of data transformation between the database and a JSON representations.
-   along with a single function from [FastAPI](https://fastapi.tiangolo.com/).

## Data Validation and Transformation

When building a robust API, it's important to validate all the data passing into the system. It would be possible to do this using a stack of `if/else` statements, but it's much more effective to define a schema declaratively, and to allow that to programmatically validate the data being input.

I used a technique that I learned from [Beanie](https://github.com/roman-right/beanie), a new and neat ODM that I unfortunately couldn't practically use on this project, because Beanie is async, and Flask is a blocking framework.

Beanie uses [Pydantic](https://pydantic-docs.helpmanual.io/usage/exporting_models/) to define a schema, and adds a custom Field type for ObjectId.

``` python
# model.py

class Cocktail(BaseModel):
    id: Optional[PydanticObjectId] = Field(None, alias="_id")
    slug: str
    name: str
    ingredients: List[Ingredient]
    instructions: List[str]
    date_added: Optional[datetime]
    date_updated: Optional[datetime]

    def to_json(self):
        return jsonable_encoder(self, exclude_none=True)

    def to_bson(self):
        data = self.dict(by_alias=True, exclude_none=True)
        if data["_id"] is None:
            data.pop("_id")
        return data
```

This `Cocktail` schema defines the structure of a `Cocktail` instance, which will be validated by Pydantic when instances are created. It includes another embedded schema for `Ingredient`, which is defined in a similar way.

I added convenience functions to export the data in the `Cocktail` instance to either a JSON-compatible `dict` or a BSON-compatible `dict`. The differences are subtle, but BSON supports native `ObjectId` and `datetime` types, for example, whereas when encoding as JSON, it's necessary to encode ObjectId instances in some other way (I prefer a string containing the hex value of the id), and datetime objects are encoded as ISO8601 strings.

The `to_json` method makes use of a function imported from FastAPI, which recurses through the instance data, encoding all values in a JSON-compatible form. It already handles `datetime` instances correctly, but to get it to handle ObjectId values, I extracted some [custom field](https://github.com/mongodb-developer/rewrite-it-in-rust/blob/master/flask-cocktail-api/src/cocktailapi/objectid.py) code from Beanie, which can be found in `objectid.py`.

The `to_bson` method doesn't need to pass the `dict` data through `jsonable_encoder`. All the types used in the schema can be directly saved with PyMongo. It's important to set `by_alias` to `True`, so that the key for `_id` is just that, `_id`, and not the schema's `id` without an underscore.

``` python
# objectid.py

class PydanticObjectId(ObjectId):
    """
    ObjectId field. Compatible with Pydantic.
    """

    @classmethod
    def __get_validators__(cls):
        yield cls.validate

    @classmethod
    def validate(cls, v):
        return PydanticObjectId(v)

    @classmethod
    def __modify_schema__(cls, field_schema: dict):
        field_schema.update(
            type="string",
            examples=["5eb7cf5a86d9755df3a6c593", "5eb7cfb05e32e07750a1756a"],
        )

ENCODERS_BY_TYPE[PydanticObjectId] = str
```

This approach is neat for this particular use-case, but I can't help feeling that it would be limiting in a more complex system. There are many [patterns for storing data in MongoDB](https://www.mongodb.com/blog/post/building-with-patterns-a-summary). These often result in storing data in a form that is optimal for writes or reads, but not necessarily the representation you would wish to export in an API.

>**What is a Slug?**
>
>Looking at the schema above, you may have wondered what a "slug" is ... well, apart from a slimy garden pest.
>
>A slug is a unique, URL-safe, mnemonic used for identifying a document. I picked up the terminology as a Django developer, where this term is part of the framework. A slug is usually derived from another field. In this case, the slug is derived from the name of the cocktail, so if a cocktail was called "Rye Whiskey Old-Fashioned," the slug would be "rye-whiskey-old-fashioned."
>
>In this API, that cocktail could be accessed by sending a `GET` request to the `/cocktails/rye-whiskey-old-fashioned` endpoint.
>
>I've kept the unique `slug` field separate from the auto-assigned `_id` field, but I've provided both because the slug could change if the name of the cocktail was tweaked, in which case the `_id` value would provide a constant identifier to look up an exact document.

In the Rust version of this code, I was nudged to use a different approach. It's a bit more verbose, but in the end I was convinced that it would be more powerful and flexible as the system grew.

## Creating a New Document

Now I'll show you what a single endpoint looks like, first focusing on the "Create" endpoint, that handles a POST request to `/cocktails` and creates a new document in the "recipes" collection. It then returns the document that was stored, including the newly unique ID that MongoDB assigned as `_id`, because this is a RESTful API, and that's what RESTful APIs do.

``` python
@app.route("/cocktails/", methods=["POST"])
def new_cocktail():
    raw_cocktail = request.get_json()
    raw_cocktail["date_added"] = datetime.utcnow()

    cocktail = Cocktail(**raw_cocktail)
    insert_result = recipes.insert_one(cocktail.to_bson())
    cocktail.id = PydanticObjectId(str(insert_result.inserted_id))
    print(cocktail)

    return cocktail.to_json()
```

This endpoint modifies the incoming JSON directly, to add a `date_added` item with the current time. It then passes it to the constructor for our Pydantic schema. At this point, if the schema failed to validate the data, an exception would be raised and displayed to the user.

After validating the data, `to_bson()` is called on the `Cocktail` to convert it to a BSON-compatible dict, and this is directly passed to PyMongo's `insert_one` method. There's no way to get PyMongo to return the document that was just inserted in a single operation (although an upsert using `find_one_and_update` is similar to just that).

After inserting the data, the code then updates the local object with the newly-assigned `id` and returns it to the client.

## Reading a Single Cocktail

Thanks to `Flask-PyMongo`, the endpoint for looking up a single cocktail is even more straightforward:

``` python
@app.route("/cocktails/<string:slug>", methods=["GET"])
def get_cocktail(slug):
    recipe = recipes.find_one_or_404({"slug": slug})
    return Cocktail(**recipe).to_json()
```

This endpoint will abort with a 404 if the slug can't be found in the collection. Otherwise, it simply instantiates a Cocktail with the document from the database, and calls `to_json` to convert it to a dict that Flask will automatically encode correctly as JSON.

## Listing All the Cocktails

This endpoint is a monster, and it's because of pagination, and the links for pagination. In the sample data above, you probably noticed the `_links` section:

``` json
"_links": {
    "last": {
        "href": "http://localhost:5000/cocktails/?page=5"
    }, 
    "next": {
        "href": "http://localhost:5000/cocktails/?page=5"
    }, 
    "prev": {
        "href": "http://localhost:5000/cocktails/?page=3"
    }, 
    "self": {
        "href": "http://localhost:5000/cocktails/?page=4"
    }
}, 
```

This `_links` section is specified as part of the [HAL (Hypertext Application
Language)](https://tools.ietf.org/html/draft-kelly-json-hal-00) specification. It's a good idea to follow a standard for pagination data, and I didn't feel like inventing something myself!

And here's the code to generate all this. Don't freak out.

``` python
@app.route("/cocktails/")
def list_cocktails():
    """
    GET a list of cocktail recipes.

    The results are paginated using the `page` parameter.
    """

    page = int(request.args.get("page", 1))
    per_page = 10  # A const value.

    # For pagination, it's necessary to sort by name,
    # then skip the number of docs that earlier pages would have displayed,
    # and then to limit to the fixed page size, ``per_page``.
    cursor = recipes.find().sort("name").skip(per_page * (page - 1)).limit(per_page)

    cocktail_count = recipes.count_documents({})

    links = {
        "self": {"href": url_for(".list_cocktails", page=page, _external=True)},
        "last": {
            "href": url_for(
                ".list_cocktails", page=(cocktail_count // per_page) + 1, _external=True
            )
        },
    }
    # Add a 'prev' link if it's not on the first page:
    if page > 1:
        links["prev"] = {
            "href": url_for(".list_cocktails", page=page - 1, _external=True)
        }
    # Add a 'next' link if it's not on the last page:
    if page - 1 < cocktail_count // per_page:
        links["next"] = {
            "href": url_for(".list_cocktails", page=page + 1, _external=True)
        }

    return {
        "recipes": [Cocktail(**doc).to_json() for doc in cursor],
        "_links": links,
    }
```

Although there's a lot of code there, it's not as complex as it may first appear. Two requests are made to MongoDB: one for a page-worth of cocktail recipes, and the other for the total number of cocktails in the collection. Various calculations are done to work out how many documents to skip, and how many pages of cocktails there are. Finally, some links are added for "prev" and "next" pages, if appropriate (i.e.: the current page isn't the first or last.) Serialization of the cocktail documents is done in the same way as the previous endpoint, but in a loop this time.

The update and delete endpoints are mainly repetitions of the code I've already included, so I'm not going to include them here. Check them out in the [GitHub repo](https://github.com/mongodb-developer/rewrite-it-in-rust/blob/master/flask-cocktail-api/src/cocktailapi/__init__.py) if you want to see how they work.

## Error Handling

Nothing irritates me more than using a JSON API which returns HTML when an error occurs, so I was keen to put in some reasonable error handling to avoid this happening.

After Flask set-up code, and before the endpoint definitions, the code registers two error-handlers:

``` python
@app.errorhandler(404)
def resource_not_found(e):
    """
    An error-handler to ensure that 404 errors are returned as JSON.
    """
    return jsonify(error=str(e)), 404


@app.errorhandler(DuplicateKeyError)
def resource_not_found(e):
    """
    An error-handler to ensure that MongoDB duplicate key errors are returned as JSON.
    """
    return jsonify(error=f"Duplicate key error."), 400
```

The first error-handler intercepts any endpoint that fails with a 404 status code and ensures that the error is returned as a JSON dict.

The second error-handler intercepts a `DuplicateKeyError` raised by any endpoint, and does the same thing as the first error-handler, but sets the HTTP status code to "400 Bad Request."

As I was writing this post, I realised that I've missed an error-handler to deal with invalid Cocktail data. I'll leave implementing that as an exercise for the reader! Indeed, this is one of the difficulties with writing robust Python applications: Because exceptions can be raised from deep in your stack of dependencies, it's very difficult to comprehensively predict what exceptions your application may raise in different circumstances.

This is something that's very different in Rust, and even though, as you'll see, error-handling in Rust can be verbose and tricky, I've started to love the language for its insistence on correctness.

## Wrapping Up

When I started writing this post, I though it would end up being relatively straightforward. As I added the requirement that the code should not just be a toy example, some of the inherent difficulties with building a robust API on top of any database became apparent.

In this case, Flask may not have been the right tool for the job. I recently wrote a blog post about [building an API with Beanie](https://developer.mongodb.com/article/beanie-odm-fastapi-cocktails/). Beanie and FastAPI are a match made in heaven for this kind of application and will handle validation, transformation, and pagination with much less code. On top of that, they're self-documenting and can provide the data's schema in open formats, including [OpenAPI Spec](https://swagger.io/specification/) and [JSON Schema](https://json-schema.org/)!

If you're about to build an API from scratch, I strongly recommend you check them out, and you may enjoy reading Aaron Bassett's posts on the [FARM (FastAPI, React, MongoDB) Stack](https://developer.mongodb.com/how-to/FARM-Stack-FastAPI-React-MongoDB/).

I will shortly publish the second post in this series, *Build a Cocktail API with Actix-Web, MongoDB, and Rust*, and then I'll conclude with a third post, *I Rewrote it in Rust—How Did it Go?*, where I'll evaluate the strengths and weaknesses of the two experiments.

Thank you for reading. Keep a look out for the upcoming posts!

>If you have questions, please head to our [developer community website](https://community.mongodb.com/) where the MongoDB engineers and the MongoDB community will help you build your next big idea with MongoDB.

Learn how I built a RESTful API for a collection of cocktail recipes. Although it's a small example, it covers important factors like data validation and pagination.

Build a RESTful API With Flask, MongoDB, and Python

## Creators
[Mehant Kammakomati](https://www.linkedin.com/in/mehant-kammakomati-1a0b41170/) and [Sai Vittal B](https://www.linkedin.com/in/saivittalb/) contributed this project.

## About the project

MergeURL is an instant URL shortening and merging service that lets you merge multiple URLs into a single short URL. You can merge up to 5 URLs within no time and share one single shortened URL. MergeURL lifts off the barriers of user registration and authentication, making it instant to use. It also provides two separate URLs to view the URLs' list and open all the browser URLs.

MergeURL is ranked #2 product of the day on ProductHunt. It is used by people across the world, with large numbers coming from the United States and India.


# Inspiration

We had this problem of sharing multiple URLs in a message or an email or via Twitter. We wanted to create a trustworthy service that can merge all those URLs in a single short one.  We tried finding out if there were already solutions to this problem, and most of the solutions we found, required an account creation or to put my credentials. We wanted to have something secure, trustworthy, but that doesn’t require user authentication. Sai Vittal worked mostly on the front end of the application, and I (Mehant) worked on the back end and the MongoDB database. It was a small problem that we encountered that led us to build MergeURL.  

We added our product to ProductHunt last August, and we became number #2 for a while; this gave us the kickstart to reach a wider audience. We currently have around 181.000 users and around 252.000 page views. The number of users motivates us to work a lot on updates and add more security layers to it. 

## Why MongoDB?
    
For MergeURL, MongoDB plays a crucial role in our URL shortening and merging algorithm, contributing to higher security and reducing data redundancy. MongoDB Atlas lifts off the burden to host and maintain databases that made our initial development of MergeURL 10X faster, and further maintaining and monitoring has become relatively easy. 

Firstly we discussed whether to go for a SQL or NoSQL database. According to the algorithms, our primary approach is that going with a NoSQL database would be the better option. MongoDB is at the top of the chart; it is the one that comes to mind when you think about NoSQL databases.  Client libraries like PyMongo make it so much easier to connect and use MongoDB. We use MongoDB Atlas itself because it’s already hosted. It made it much easier for us to work with it. We’ve been using the credits that we received from the GitHub Student Developer Pack offer. 

## How it works

The frontend is written using React, and it’s compiled into the optimal static assets.  As we know, the material is a relatively simple service; we don’t need a lot of complicated stuff in the back end. Therefore we used a microservice; we used Flask to write the backend server. And we use MongoDB. We have specific algorithms that work on the URLs, and MongoDB played a vital role in implementing those algorithms and taking control of the redundancy. 

It works relatively smoothly. You go to our website; you fill out the URLs you want to shorten, and it will give you a short URL that includes all the URLs. 

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/mergeurls_shown_along_with_QR_Code_6b33977a03.png)

## Challenges and lessons learned

One of the challenges lies in our experience. We both didn’t have any experience launching a product and getting it to users. Launching MergeURL was the first time we did this, and it went very well.

MongoDB specific, we didn’t have any problems. Specifically (Mehant),  I struggled a lot with SQL databases in my freshman and sophomore years. I’m pleased that I found MongoDB; it saves a lot of stress and frustration. Everything is relatively easy. Besides that, the documents are quite flexible; it’s not restricted as with SQL. We can create many more challenges with MongoDB. 

I’ve learned a lot about the process. Converting ideas into actual implementation was the most important thing. One can have many ideas, but turning them into life is essential. 

At the moment, the project merges the URLs. We are thinking of maybe adding a premium plan where people can get user-specific extensions. We use a counter variable to give those IDs to the shortened URL, but we would like to implement adding user specific extensions. 

And we would like to add analytics. How many users are clicking on your shortened URL? Where is the traffic coming from? 

We are thrilled with the product as it is, but there are plenty of future ideas. 





Shorten multiple URLs instantly while overcoming the barriers of user registration.

MergeURL - Python Example App

Shelby Carpenter

Senior Product Marketing Manager, Developer Tools and APIs

Learn step-by-step how to build a full-stack web application to track reports of unidentified flying objects (UFOs) in your area.

Build an App With Python, Flask, and MongoDB to Track UFOs

Tom McCarthy

Senior Solutions Architect

Pierre Petersson

Advisory Solutions Architect

The right to erasure, also known as the right to be forgotten, is a right granted to individuals under laws and regulations such as GDPR. This means that companies storing an individual's personal data must be able to delete it on request. Because this data can be spread across several systems, it can be technically challenging for these companies to identify and remove it from all places. Even if this is properly executed, there is also a risk that deleted data can be restored from backups in the future, potentially contributing to legal and financial risks.

This blog post addresses those challenges, demonstrating how you can make use of MongoDB's Client-Side Field Level Encryption to strengthen procedures for removing sensitive data.

>***Disclaimer**: We provide no guarantees that the solution and techniques described in this article will fulfill regulatory requirements around the right to erasure. Each organization needs to make their own determination on appropriate or sufficient measures to comply with various regulatory requirements such as GDPR.*

## What is crypto shredding?
Crypto shredding is a data destruction technique that consists of destroying the encryption keys that allow the data to be decrypted, thus making the data undecipherable. The example below gives a more in-depth explanation.

Imagine you are storing data for multiple users. You start by giving each user their own unique data encryption key (DEK), and mapping it to that customer. This is represented in the below diagram, where "User A" and "User B" each have their own key in the key store. This DEK can then be used to encrypt and decrypt any data related to the user in question.

![Diagram showing different data encryption keys for User A and User B](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/data_encryption_keys_602f207e54.png)

Let's assume that we want to remove all data for User B. If we remove User B's DEK, we can no longer decrypt any of the data that was encrypted with it; all we have left in our data store is "junk" cipher text. As the diagram below illustrates, User A's data is unaffected, but we can no longer read User B's data.

![Diagram showing what happens when user B's data encryption key is deleted from the key store](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/encryption_key_deleted_8c803d3e52.png)

## What is CSFLE?
With MongoDB’s [Client-Side Field Level Encryption](https://www.mongodb.com/docs/manual/core/csfle/) (CSFLE), applications can encrypt sensitive fields in documents prior to transmitting data to the server. This means that even when data is being used by the database in memory, it is never in plain text. The database only sees the encrypted data but still enables you to query it.

MongoDB CSFLE utilizes envelope encryption, which is the practice of encrypting plaintext data with a data key, which itself is in turn encrypted by a top level envelope key (also known as a "master key"). 

![Diagram showing an envelope key being used to encrypt a data key, which in turn encrypts data](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/envelop_key_6a599cfd2e.png)

Envelope keys are usually managed by a Key Management Service (KMS). MongoDB CSFLE supports multiple KMSs, such as AWS KMS, GCP KMS, Azure KeyVault, and Keystores supporting the KMIP standard (e.g., Hashicorp Keyvault).

CSFLE can be used in either [automatic](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/automatic-encryption/#std-label-csfle-fundamentals-automatic-encryption) mode or [explicit mode](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/manual-encryption/#std-label-csfle-fundamentals-manual-encryption) — or a combination of both. Automatic mode enables you to perform encrypted read and write operations based on a defined encryption schema, avoiding the need for application code to specify how to encrypt or decrypt fields. This [encryption schema](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/create-schema/#std-label-csfle-fundamentals-create-schema) is a JSON document that defines what fields need to be encrypted. [Explicit](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/manual-encryption/#std-label-csfle-fundamentals-manual-encryption) mode refers to using the MongoDB driver's encryption library to manually encrypt or decrypt fields in your application.

In this article, we are going to use the [explicit encryption](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/manual-encryption/#std-label-csfle-fundamentals-manual-encryption) technique to showcase how we can use crypto shredding techniques with CSFLE to implement (or augment) procedures to "forget" sensitive data. We'll be using AWS KMS to demonstrate this.

## Bringing it all together
With MongoDB as our database, we can use CSFLE to implement crypto shredding, so we can provide stronger guarantees around data privacy.

To demonstrate how you could implement this, we'll walk you through a demo application. The demo application is a python (Flask) web application with a front end, which exposes functionality for signup, login, and a data entry form. We have also added an "admin" page to showcase the crypto shredding related functionality. If you want to follow along, you can run the application yourself — you'll find the necessary code and instructions in [GitHub](https://github.com/mongodb-developer/mongodb-flask-cryptoshredding-example).

![demo application home page](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/demo_application_homepage_ad48ff5c3d.png)

When a user signs up, our application will generate a DEK for the user, then store the ID for the DEK along with other user details. Key generation is done via the  `create_data_key` method on the `ClientEncryption` class, which we initialized earlier as `app.mongodb_encryption_client`. This encryption client is responsible for generating a DEK, which in this case will be encrypted by the envelope key. In our case, the encryption client is configured to use an envelope key from AWS KMS.

```python
# flaskapp/db_queries.py

@aws_credential_handler
def create_key(userId):
    data_key_id = \
        app.mongodb_encryption_client.create_data_key(kms_provider,
            master_key, key_alt_names=[userId])
    return data_key_id
```

We can then use this method when saving the user.

```python
# flaskapp/user.py

def save(self):
        dek_id = db_queries.create_key(self.username)
        result = app.mongodb[db_name].user.insert_one(
            {
                "username": self.username,
                "password_hash": self.password_hash,
                "dek_id": dek_id,
                "createdAt": datetime.now(),
            }
        )
        if result:
            self.id = result.inserted_id
            return True
        else:
            return False
```

Once signed up, the user can then log in, after which they can enter data via a form shown in the screenshot below. This data has a "name" and a "value", allowing the user to store arbitrary key-value pairs.

![demo application showing a form to add data](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/form_to_add_data_364735b756.png)

In the database, we'll store this data in a MongoDB collection called “data,” in documents structured like this:

```json
{
   "name": "shoe size",
   "value": "10",
   "username": "tom"
}
```

For the sake of this demonstration, we have chosen to encrypt the value and username fields from this document. Those fields will be encrypted using the DEK created on signup belonging to the logged in user.

```python
# flaskapp/db_queries.py

# Fields to encrypt, and the algorithm to encrypt them with
ENCRYPTED_FIELDS = {
    # Deterministic encryption for username, because we need to search on it
    "username": Algorithm.AEAD_AES_256_CBC_HMAC_SHA_512_Deterministic,
    # Random encryption for value, as we don't need to search on it
    "value": Algorithm.AEAD_AES_256_CBC_HMAC_SHA_512_Random,
}
```

The insert_data function then loops over the fields we want to encrypt and the algorithm we're using for each.

```python
# flaskapp/db_queries.py

def insert_data(document):
    document["username"] = current_user.username
    # Loop over the field names (and associated algorithm) we want to encrypt
    for field, algo in ENCRYPTED_FIELDS.items():
        # if the field exists in the document, encrypt it
        if document.get(field):
            document[field] = encrypt_field(document[field], algo)
    # Insert document (now with encrypted fields) to the data collection
    app.data_collection.insert_one(document)
```

If the specified fields exist in the document, this will call our encrypt_field function to perform the encryption using the specified algorithm.

```python
# flaskapp/db_queries.py

# Encrypt a single field with the given algorithm
@aws_credential_handler
def encrypt_field(field, algorithm):
    try:
        field = app.mongodb_encryption_client.encrypt(
            field,
            algorithm,
            key_alt_name=current_user.username,
        )
        return field
    except pymongo.errors.EncryptionError as ex:
        # Catch this error in case the DEK doesn't exist. Log a warning and 
        # re-raise the exception
        if "not all keys requested were satisfied" in ex._message:
            app.logger.warn(
                f"Encryption failed: could not find data encryption key for user: {current_user.username}"
            )
        raise ex
```

Once data is added, it will be shown in the web app:

![demo application showing the data added in the previous step](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/demo_application_showing_data_12dfb4df80.png)

Now let's see what happens if we delete the DEK. To do this, we can head over to the admin page. This admin page should only be provided to individuals that have a need to manage keys, and we have some choices:

![demo application admin page, showing the "delete data encryption key" button](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/demo_application_admin_page_829d227a7f.png)

We're going to use the "Delete data encryption key" option, which will remove the DEK, but leave all data entered by the user intact. After that, the application will no longer be able to retrieve the data that was stored via the form. When trying to retrieve the data for the logged in user, an error will be thrown

![demo application showing an error message when trying to retrieve data](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/error_message_e04de7ff46.png)

**Note**: After we do perform the data key deletion, the web application may still be able to decrypt and show the data for a short period of time before its cache expires — this takes a maximum of 60 seconds. 

But what is actually left in the database? To get a view of this, you can go back to the Admin page and choose "Fetch data for all users." In this view, we won't throw an exception if we can't decrypt the data. We'll just show exactly what we have stored in the database. Even though we haven't actually deleted the user's data, because the data encryption key no longer exists, all we can see now is cipher text for the encrypted fields "username" and "value".

![demo application showing raw cipher text instead of decrypted data](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/raw_cipher_text_8312deece3.png)

And here is the code we're using to fetch the data in this view. As you can see, we use very similar logic to the encrypt method shown earlier. We perform a find operation without any filters to retrieve all the data from our data collection. We'll then loop over our ENCRYPTED_FIELDS dictionary to see which fields need to be decrypted.

```python
# flaskapp/db_queries.py

def fetch_all_data_unencrypted(decrypt=False):
    results = list(app.data_collection.find())

    if decrypt:
        for field in ENCRYPTED_FIELDS.keys():
            for result in results:
                if result.get(field):
                    result[field], result["encryption_succeeded"] = decrypt_field(result[field])
    return results
```

The decrypt_field function is called for each field to be decrypted, but in this case we'll catch the error if we cannot successfully decrypt it due to a missing DEK.

```python
# flaskapp/db_queries.py

# Try to decrypt a field, returning a tuple of (value, status). This will be either (decrypted_value, True), or (raw_cipher_text, False) if we couldn't decrypt
def decrypt_field(field):
    try:
        # We don't need to pass the DEK or algorithm to decrypt a field
        field = app.mongodb_encryption_client.decrypt(field)
        return field, True
    # Catch this error in case the DEK doesn't exist.
    except pymongo.errors.EncryptionError as ex:
        if "not all keys requested were satisfied" in ex._message:
            app.logger.warn(
                "Decryption failed: could not find data encryption key to decrypt the record."
            )
            # If we can't decrypt due to missing DEK, return the "raw" value.
            return field, False
        raise ex
```

We can also use the `mongosh` shell to check directly in the database, just to prove that there's nothing there we can read. 

![mongosh](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/mongosh_ccc9e4e3a7.png)

At this point, savvy readers may be asking the question, "But what if we restore the database from a backup?" If we want to prevent this, we can use two separate database clusters in our application — one for storing data and one for storing DEKs (the "key vault"). This theory is applied in the sample application, which requires you to specify two MongoDB connection strings — one for data and one for the key vault. If we use separate clusters, it decouples the restoration of backups for application data and the key vault; restoring a backup on the data cluster won't restore any DEKs which have been deleted from the key vault cluster.

## Conclusion
In this blog post, we've demonstrated how MongoDB's Client-Side Field Level Encryption can be used to simplify the task of "forgetting" certain data. With a single "delete data key" operation, we can effectively forget data which may be stored across different databases, collections, backups, and logs. In a real production application, we may wish to delete all the user's data we can find, on top of removing their DEK. This "defense in depth" approach helps us to ensure that the data is really gone. By implementing crypto shredding, the impact is much smaller if a delete operation fails, or misses some data that should have been wiped.

You can find more details about MongoDB's Client-Side Field Level Encryption in our [documentation](https://www.mongodb.com/docs/manual/core/csfle/). If you have questions, feel free to make a post on our [community forums](https://www.mongodb.com/community/forums/). 

Learn how to make use of MongoDB's Client-Side Field Level Encryption to strengthen procedures for removing sensitive data.

Flask

Featured

How to Deploy a Flask Application With MongoDB on Fly.io

Best Practices for Using Flask and MongoDB

Implementing Right to Erasure with CSFLE

All Flask Content