Partner

Abirami Sukumaran

Developer Advocate, Google

Stanimira Vlaeva

Developer Advocate

## Plea and the Pledge: Truly Serverless

As modern application developers, we’re juggling many priorities: performance, flexibility, usability, security, reliability, and maintainability. On top of that, we’re handling dependencies, configuration, and deployment of multiple components in multiple environments and sometimes multiple repositories as well. And then we have to keep things secure and simple. Ah, the nightmare!

This is the reason we love serverless computing. Serverless allows developers to focus on the thing they like to do the most—development—and leave the rest of the attributes, including infrastructure and maintenance, to the platform offerings. 

In this read, we’re going to see how Cloud Run and MongoDB come together to enable a completely serverless MEAN stack application development experience. We'll learn how to build a serverless MEAN application with [Cloud Run](https://cloud.google.com/run/) and [MongoDB Atlas](https://www.mongodb.com/atlas), the multi-cloud modern database by MongoDB.

### Containerized deployments with Cloud Run

All serverless platform offer exciting capabilities: 

* Event-driven function (not a hard requirement though)
* No-infrastructure maintenance
* Usage-based pricing
* Auto-scaling capabilities

Cloud Run stands out of the league by enabling us to: 

* Package code in multiple stateless containers that are request-aware and invoke it via HTTP requests
* Only be charged for the exact resources you use
* Support any programming language or any operating system library of your choice, or any binary

Check this link for more features in full [context](https://cloud.google.com/run/?utm_source=google&utm_medium=cpc&utm_campaign=japac-IN-all-en-dr-skws-all-pkws-trial-b-dr-1009882&utm_content=text-ad-none-none-DEV_c-CRE_529584319423-ADGP_Hybrid%20%7C%20SKWS%20-%20PHR%20%7C%20Txt%20~%20Compute%20~%20Cloud%20Run_Global%20gap%20analysis-KWID_43700065767178853-aud-970366092687%3Akwd-677335471899&userloc_1007768-network_g&utm_term=KW_cloud%20run&gclid=Cj0KCQjw1tGUBhDXARIsAIJx01lcrrpCE3MLqe4k3MiQdd8ibvGmXduL9kLXELPjodjSu7Gipc5ODYwaAlBkEALw_wcB&gclsrc=aw.ds).

However, many serverless models overlook the fact that traditional databases are not managed. You need to manually provision infrastructure (vertical scaling) or add more servers (horizontal scaling) to scale the database. This introduces a bottleneck in your serverless architecture and can lead to performance issues.

### Deploy a serverless database with MongoDB Atlas

MongoDB launched serverless instances, a new fully managed, serverless database deployment in Atlas to solve this problem. With serverless instances you never have to think about infrastructure — simply deploy your database and it will scale up and down seamlessly based on demand — requiring no hands-on management. And the best part, you will only be charged for the operations you run. To make our architecture truly serverless, we'll combine Cloud Run and MongoDB Atlas capabilities.

## What's the MEAN stack?

The [MEAN stack](https://www.mongodb.com/mean-stack) is a technology stack for building full-stack web applications entirely with JavaScript and JSON. The MEAN stack is composed of four main components—MongoDB, Express, Angular, and Node.js.

* **MongoDB** is responsible for data storage. 
* **Express.js** is a Node.js web application framework for building APIs.
* **Angular** is a client-side JavaScript platform.
* **Node.js** is a server-side JavaScript runtime environment. The server uses the MongoDB Node.js driver to connect to the database and retrieve and store data. 

![MEAN stack](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/mean_stack_e791a7354e.PNG)

## Steps for deploying truly serverless MEAN stack apps with Cloud Run and MongoDB

In the following sections, we’ll provision a new MongoDB serverless instance, connect a MEAN stack web application to it, and finally, deploy the application to Cloud Run.

### 1. Create the database

Before you begin, get started with [MongoDB Atlas on Google Cloud](https://console.cloud.google.com/marketplace/product/mongodb/mdb-atlas-self-service).

![MongoDB Atlas on Google Cloud subscription page](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/mongodb_atlas_on_google_cloud_e3bcc0b537.JPG)

Once you sign up, click the “Build a Database” button to create a new serverless instance. Select the following configuration:

![Create new serverless instance in MongoDB Atlas configuration page
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/create_new_serverless_instance_9134772726.JPG)

Once your serverless instance is provisioned, you should see it up and running.

![Database deployments page in MongoDB Atlas
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/database_deployments_page_5309d13fb4.JPG)

Click on the “Connect” button to add a connection IP address and a database user.

For this blog post, we’ll use the “Allow Access from Anywhere” setting. MongoDB Atlas comes with a set of security and access features. You can learn more about them in the [security features documentation article](https://www.mongodb.com/docs/atlas/setup-cluster-security/).

Use credentials of your choice for the database username and password. Once these steps are complete, you should see the following:

![Screen showing 'You are ready to connect' in MongoDB Atlas
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/ready_to_connect_e06352f53b.JPG)

Proceed by clicking on the “Choose a connection method” button and then selecting “Connect your application”.

![Screen showing the connection string for MongoDB Atlas
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/connection_string_mongodb_atlas_c28ea136ce.JPG)

Copy the connection string you see and replace the password with your own. We’ll use that string to connect to our database in the following sections.

### 2. Set up a Cloud Run project

First, sign in to [Cloud Console](http://console.cloud.google.com/), create a new project, or reuse an existing one.

Remember the Project Id for the project you created. Below is an image from [https://codelabs.developers.google.com/codelabs/cloud-run-hello#1](https://codelabs.developers.google.com/codelabs/cloud-run-hello#1) that shows how to create a new project in Google Cloud.

!['Create a new project' configuration page in Google Cloud. The project id is highlighted.
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/create_new_project_605bbeec53.JPG)

Then, enable Cloud Run API from [Cloud Shell](https://cloud.google.com/shell):

* Activate Cloud Shell from the Cloud Console. Simply click Activate Cloud Shell.

![Google Cloud Shell Editor loading page
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/google_cloud_shell_editor_18236232e8.JPG)

* Use the below command:

*gcloud services enable run.googleapis.com*

We will be using Cloud Shell and [Cloud Shell Editor](https://cloud.google.com/shell/docs/editor-overview#:~:text=Cloud%20Shell%20comes%20with%20a,and%20is%20based%20on%20Theia.) for code references. To access Cloud Shell Editor, click Open Editor from the Cloud Shell Terminal:

![Google Cloud Shell Code Editor
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/google_cloud_shell_code_editor_51ed2f16ca.JPG)

Finally, we need to clone the MEAN stack project we’ll be deploying. 

We’ll deploy an employee management web application. The REST API is built with Express and Node.js; the web interface, with Angular; and the data will be stored in the MongoDB Atlas instance we created earlier.

Clone the project repository by executing the following command in the Cloud Shell Terminal:

`git clone` [https://github.com/mongodb-developer/mean-stack-example.git](https://github.com/mongodb-developer/mean-stack-example.git)

In the following sections, we will deploy a couple of services—one for the Express REST API and one for the Angular web application. 

### 3. Deploy the Express and Node.js REST API

First, we’ll deploy a Cloud Run service for the Express REST API. 

The most important file for our deployment is the Docker configuration file. Let’s take a look at it:

**mean-stack-example/server/Dockerfile**

```
FROM node:17-slim
 
WORKDIR /usr/app
COPY ./ /usr/app
 
# Install dependencies and build the project.
RUN npm install
RUN npm run build
 
# Run the web service on container startup.
CMD ["node", "dist/server.js"]
```

The configuration sets up Node.js, and copies and builds the project. When the container starts, the command “node dist/server.js” starts the service.

To start a new Cloud Run deployment, click on the Cloud Run icon on the left sidebar:

![Select the 'Cloud Run' icon from the left sidebar
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/select_new_cloud_run_e2c48b457a.JPG)

Then, click on the Deploy to Cloud Run icon:

![Select the 'Deploy to Cloud Run' icon from the opened pane
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/select_deploy_to_cloud_run_c90b1dc8f6.JPG)

Fill in the service configuration as follows:

* Service name: node-express-api
* Deployment platform: Cloud Run (fully managed)
* Region: Select a region close to your database region to reduce latency
* Authentication: Allow unauthenticated invocations

Under Revision Settings, click on Show Advanced Settings to expand them:

* Container port: 5200
* Environment variables. Add the following key-value pair and make sure you add the connection string for your own MongoDB Atlas deployment:

`ATLAS_URI:mongodb+srv:/<username>:<password>@sandbox.pv0l7.mongodb.net/meanStackExample?retryWrites=true&w=majority`

For the Build environment, select Cloud Build.

Finally, in the Build Settings section, select:

* Builder: Docker
* Docker: mean-stack-example/server/Dockerfile

Click the Deploy button and then Show Detailed Logs to follow the deployment of your first Cloud Run service!

After the build has completed, you should see the URL of the deployed service:

![Screenshot displaying the message 'Deployment completed successfully!' and the deployment URL.
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/deployment_completed_successfully_31bcb13b6f.JPG)

Open the URL and append ‘/employees’ to the end. You should see an empty array because currently, there are no documents in the database. Let’s deploy the user interface so we can add some!

### 4. Deploy the Angular web application

Our Angular application is in the client directory. To deploy it, we’ll use the Nginx server and Docker.

> Just a thought, there is also an option to use Firebase Hosting for your [Angular application deployment](https://www.youtube.com/watch?v=jVPHsolXgLk) as you can serve your content to a CDN (content delivery network) directly.

Let’s take a look at the configuration files:

**mean-stack-example/client/nginx.conf**

```
events{}
 
http {
 
   include /etc/nginx/mime.types;
 
   server {
       listen 8080;
       server_name 0.0.0.0;
       root /usr/share/nginx/html;
       index index.html;
 
       location / {
           try_files $uri $uri/ /index.html;
       }
   }
}
```

In the Nginx configuration, we specify the default port—8080, and the starting file—`index.html`.

**mean-stack-example/client/Dockerfile**

```
FROM node:17-slim AS build
 
WORKDIR /usr/src/app
COPY package.json package-lock.json ./
 
# Install dependencies and copy them to the container
RUN npm install
COPY . .
 
# Build the Angular application for production
RUN npm run build --prod
 
# Configure the nginx web server
FROM nginx:1.17.1-alpine
COPY nginx.conf /etc/nginx/nginx.conf
COPY --from=build /usr/src/app/dist/client /usr/share/nginx/html
 
# Run the web service on container startup.
CMD ["nginx", "-g", "daemon off;"]
```

In the Docker configuration, we install Node.js dependencies and build the project. Then, we copy the built files to the container, configure, and start the Nginx service.

Finally, we need to configure the URL to the REST API so that our client application can send requests to it. Since we’re only using the URL in a single file in the project, we’ll hardcode the URL. Alternatively, you can attach the environment variable to the window object and access it from there.

**mean-stack-example/client/src/app/employee.service.ts**

```
@Injectable({
 providedIn: 'root'
})
export class EmployeeService {
 // Replace with the URL of your REST API
 private url = 'https://node-express-api-vsktparjta-uc.a.run.app'; 
…
```

We’re ready to deploy to Cloud Run! Start a new deployment with the following configuration settings:

* Service Settings: Create a service
* Service name: angular-web-app
* Deployment platform: Cloud Run (fully managed)
* Authentication: Allow unauthenticated invocations

For the Build environment, select Cloud Build.

Finally, in the Build Settings section, select:

* Builder: Docker
* Docker: mean-stack-example/client/Dockerfile

Click that Deploy button again and watch the logs as your app is shipped to the cloud! When the deployment is complete, you should see the URL for the client app:

![Screenshot displaying the message 'Deployment completed successfully!' and the deployment URL for the Angular service.
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/deployment_completed_successfully_notice_6602368f53.JPG)

Open the URL, and play with your application!

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/employees_demo_1_da0a6342ba.gif)

### Command shell alternative for build and deploy

The steps covered above can alternatively be implemented from Command Shell as below:

Step 1: Create the new project directory named “mean-stack-example” either from the Code Editor or Cloud Shell Command (Terminal):

*mkdir mean-stack-demo
cd mean-stack-demo*

Step 2: Clone project repo and make necessary changes in the configuration and variables, same as mentioned in the previous section.

Step 3: Build your container image using Cloud build by running the command in Cloud Shell:

*gcloud builds submit --tag gcr.io/$GOOGLECLOUDPROJECT/mean-stack-demo*

$GOOGLE_CLOUD_PROJECT is an environment variable containing your Google Cloud project ID when running in Cloud Shell.

Step 4: Test it locally by running: 
docker run -d -p 8080:8080 gcr.io/$GOOGLE_CLOUD_PROJECT/mean-stack-demo 
and by clicking Web Preview, Preview on port 8080.

Step 5: Run the following command to deploy your containerized app to Cloud Run:

*gcloud run deploy mean-stack-demo --image 
gcr.io/$GOOGLECLOUDPROJECT/mean-stack-demo --platform managed --region us-central1 --allow-unauthenticated --update-env-vars DBHOST=$DB_HOST*

a. –allow-unauthenticated will let the service be reached without authentication.

b. –platform-managed means you are requesting the fully managed environment and not the Kubernetes one via Anthos.
 
c. –update-env-vars expects the MongoDB Connection String to be passed on to the environment variable DBHOST.
Hang on until the section on Env variable and Docker for Continuous Deployment for Secrets and Connection URI management.

d. When the deployment is done, you should see the deployed service URL in the command line.

e. When you hit the service URL, you should see your web page on the browser and the logs in the Cloud Logging Logs Explorer page. 

![Service logs displayed in the Cloud Logging Logs Explorer page
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/services_logs_2085273e8a.JPG)

### 5. Environment variables and Docker for continuous deployment

If you’re looking to automate the process of building and deploying across multiple containers, services, or components, storing these configurations in the repo is not only cumbersome but also a security threat. 

1. For ease of cross-environment continuous deployment and to avoid security vulnerabilities caused by leaking credential information, we can choose to pass variables at build/deploy/up time.
    
    *--update-env-vars* allows you to set the environment variable to a value that is passed only at run time. In our example, the variable DBHOST is assigned the value of $DB_HOST. which is set as *DB_HOST = ‘<<ENCODED CONNECTION URI>>’*.

    Please note that unencoded symbols in Connection URI (username, password) will result in connection issues with MongoDB. For example, if you have a $ in the password or username, replace it with %24 in the encoded Connection URI.

2. Alternatively, you can also pass configuration variables as env variables at build time into docker-compose (*docker-compose.yml*). By passing configuration variables and credentials, we avoid credential leakage and automate deployment securely and continuously across multiple environments, users, and applications.

## Conclusion

MongoDB Atlas with Cloud Run makes for a truly serverless MEAN stack solution, and for those looking to build an application with a serverless option to run in a stateless container, Cloud Run is your best bet. 

## Before you go…

Now that you have learnt how to deploy a simple MEAN stack application on Cloud Run and MongoDB Atlas, why don’t you take it one step further with your favorite client-server use case? Reference the below resources for more inspiration:

* Cloud Run HelloWorld: [https://codelabs.developers.google.com/codelabs/cloud-run-hello#4](https://codelabs.developers.google.com/codelabs/cloud-run-hello#4)
* MongoDB - MEAN Stack: [https://www.mongodb.com/languages/mean-stack-tutorial](https://www.mongodb.com/languages/mean-stack-tutorial)

If you have any comments or questions, feel free to reach out to us online: [Abirami Sukumaran](http://www.linkedin.com/in/abiramisukumaran) and [Stanimira Vlaeva](https://twitter.com/StanimiraVlaeva).

In this blog, we'll see how Cloud Run and MongoDB come together to enable a completely serverless MEAN stack application development experience.

Serverless MEAN Stack Applications with Cloud Run and MongoDB Atlas

Joel Lord

Cedric Clyburn

Developer Advocate Intern

Kubernetes is now an industry-wide standard when it comes to all things containers, but when it comes to deploying a database, it can be a bit tricky! However, tasks like adding persistence, ensuring redundancy, and database maintenance can be easily handled with [MongoDB Atlas](https://www.mongodb.com/atlas/database). Fortunately, the [MongoDB Atlas Operator](https://www.mongodb.com/kubernetes/atlas-operator) gives you the full benefits of using MongoDB Atlas, while still managing everything from within your Kubernetes cluster. In this tutorial, we’ll deploy a [MERN](https://www.mongodb.com/languages/mern-stack-tutorial) stack application in Kubernetes, install the Atlas operator, and connect our back end to Atlas using a Kubernetes secret.

## Pre-requisites
* [`kubectl`](https://kubernetes.io/docs/tasks/tools/)
* [`minikube`](https://minikube.sigs.k8s.io/docs/)
* [`jq`](https://github.com/stedolan/jq/)

You can find the complete source code for this application on [Github](https://github.com/mongodb-developer/mern-k8s). It’s a mini travel planner application using MongoDB, Express, React, and Node ([MERN](https://www.mongodb.com/languages/mern-stack-tutorial)). While this tutorial should work for any Kubernetes cluster, we’ll be using Minikube for simplicity and consistency.

## Getting started

When it comes to deploying a database on Kubernetes, there’s no simple solution. Apart from persistence and redundancy challenges, you may need to move data to specific geolocated servers to ensure that you comply with GDPR policies. Thus, you’ll need a reliable, scalable, and resilient database once you launch your application into production. 

MongoDB Atlas is a full modern database that includes the database you love, which takes care of many of the database complexities you’re used to. But, there is a gap between MongoDB Atlas and your Kubernetes cluster. Let’s take a look at the MongoDB Atlas Operator by deploying the example MERN application with a back end and front end.

This application uses a three-tier application architecture, which will have the following layout within our Kubernetes cluster:

![Kubernetes architecture with the Atlas Operator](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/kubernetes_layout_885ae9b68e.jpg)

To briefly overview this layout, we’ve got a back end with a deployment that will ensure we have two pods running at any given time, and the same applies for our front end. Traffic is redirected and configured by our ingress, meaning `/api` requests route to our back end and everything else will go to the front end. The back end of our application is responsible for the connection to the database, where we’re using MongoDB Atlas Operator to link to an Atlas instance. 


## Deploying the application on Kubernetes

To simplify the installation process of the application, we can use a single `kubectl` command to deploy our demo application on Kubernetes. The single file we’ll use includes all of the deployments and services for the back end and front end of our application, and uses containers created with the Dockerfiles in the folder. 

First, start by cloning the repository that contains the starting source code.

```
git clone https://github.com/mongodb-developer/mern-k8s.git

cd mern-k8s
```

Secondly, as part of this tutorial, you’ll need to run `minikube tunnel` to access our services at `localhost`.

```
minikube tunnel
```

Now, let’s go ahead and deploy everything in our Kubernetes cluster by applying the following `application.yaml` file.

```
kubectl apply -f k8s/application.yaml
```

You can take a look at what you now have running in your cluster by using the `kubectl get` command.

```
kubectl get all
```

You should see multiple pods, services, and deployments for the back end and front end, as well as replicasets. At the moment, they are more likely in a ContainerCreating status. This is because Kubernetes needs to pull the images to its local registry. As soon as the images are ready, the pods will start.

To see the application in action, simply head to `localhost` in your web browser, and the application should be live!

![Browser view of the demo application](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/initial_demo_b606f20aad.jpg)

However, you’ll notice there’s no way to add entries to our application, and this is because we haven’t provided a connection string yet for the back end to connect to a MongoDB instance. For example, if we happen to check the logs for one of the recently created backend pods, we can see that there’s a placeholder for a connection string.

```
kubectl logs pod/mern-k8s-back-d566cc88f-hhghl

Connecting to database using $ATLAS_CONNECTION_STRING
Server started on port 3000
MongoParseError: Invalid scheme, expected connection string to start with "mongodb://" or "mongodb+srv://"
```

We’ve ran into a slight issue, as this demo application is using a placeholder (`$ATLAS_CONNECTION_STRING`) for the MongoDB connection string, which needs to be replaced by a valid connection string from our Atlas cluster. This issue can be taken care of with the MongoDB Atlas Operator, which allows you to manage everything from within Kubernetes and gives you the full advantages of using MongoDB Atlas, including generating a connection string as a Kubernetes secret.

## Using the MongoDB Atlas Operator for Kubernetes

As there’s currently a gap between your Kubernetes cluster and MongoDB Atlas, let’s use the [Atlas Operator](https://www.mongodb.com/kubernetes/atlas-operator) to remedy this issue. Through the operator, we’ll be able to manage our Atlas projects and clusters from Kubernetes. Specifically, getting your connection string to fix the error we received previously can be done now through Kubernetes secrets, meaning we won’t need to retrieve it from the Atlas UI or CLI.


### Why use the Operator?

The Atlas Operator bridges the gap between Atlas, the MongoDB data platform, and your Kubernetes cluster. By using the operator, you can use `kubectl` and your familiar tooling to manage and set up your Atlas deployments. Particularly, it allows for most of the Atlas functionality and tooling to be performed without having to leave your Kubernetes cluster. Installing the Atlas operator creates the Custom Resource Definitions that will connect to the MongoDB Atlas servers.


### Installing the Atlas Operator

The installation process for the Atlas Operator is as simple as running a `kubectl` command. All of the source code for the operator can be found on the [Github repository](https://github.com/mongodb/mongodb-atlas-kubernetes).

```
kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-atlas-kubernetes/main/deploy/all-in-one.yaml
```

This will create new custom resources in your cluster that you can use to create or manage your existing Atlas projects and clusters.

### Creating a MongoDB Atlas cluster 

If you haven't already, head to the [Atlas Registration](https://www.mongodb.com/cloud/atlas/register) page to create your free account. This account will let you create a database on a shared server, and you won't even need a credit card to use it.

### Set up access

In order for the operator to be able to manage your cluster, you will need to provide it with an API key with the appropriate permissions. Firstly, let’s retrieve the organization ID.

In the upper left part of the Atlas UI, you will see your organization name in a dropdown. Right next to the dropdown is a gear icon. Clicking on this icon will open up a page called _Organization Settings_. From this page, look for a box labeled _Organization ID_. 

![Navigating to the organization settings page](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/organization_settings_home_7c3f0daa0e.jpg)

Save that organization ID somewhere for future use. You can also save it in an environment variable.

```
export ORG_ID=60c102....bd
```

>Note: If using Windows, use:

```
set ORG_ID=60c102....bd
```

Next, let’s create an API key. From the same screen, look for the _Access Manager_ option in the left navigation menu. This will bring you to the _Organization Access_ screen. In this screen, follow the [instructions](https://www.mongodb.com/docs/atlas/configure-api-access/#std-label-create-org-api-key) to create a new API key.

![Creating a new API key from the access manager page](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/access_manager_tab_d4bf976fb7.jpg)

The key will need the **Organization Project Creator** role in order to create new projects and clusters. If you want to manage existing clusters, you will need to provide it with the **Organization Owner** role. Save the API private and public keys. You can also add them to the environment.

![Storing the public and private API key information](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/api_key_credentials_bcc611f1f0.png)

```
export ATLAS_PUBLIC_KEY=iwpd...i
export ATLAS_PRIVATE_KEY=e13debfb-4f35-4...cb
```

>Note: If using Windows, use:

```
set ATLAS_PUBLIC_KEY=iwpd...i
set ATLAS_PRIVATE_KEY=e13debfb-4f35-4...cb
```

### Create the Kubernetes secrets

Now that you have created the API key, you can specify those values to the MongoDB Atlas Operator. By creating this secret in our Kubernetes cluster, this will give the operator the necessary permissions to create and manage projects and clusters for our specific Atlas account. 

You can create the secret with `kubectl`, and to keep it simple, let’s name our secret `mongodb-atlas-operator-api-key`. For the operator to be able to find this secret, it needs to be within the namespace `mongodb-atlas-system`.

```
kubectl create secret generic mongodb-atlas-operator-api-key \
    --from-literal="orgId=$ORG_ID" \
    --from-literal="publicApiKey=$ATLAS_PUBLIC_KEY" \
    --from-literal="privateApiKey=$ATLAS_PRIVATE_KEY" \
    -n mongodb-atlas-system
```

Next, we’ll need to label this secret, which helps the Atlas operator in finding the credentials.

```
kubectl label secret mongodb-atlas-operator-api-key atlas.mongodb.com/type=credentials -n mongodb-atlas-system
```

### Create a user password

We’ll need a password for our database user in order to access our databases, create new databases, etc. However, you won't want to hard code this password into your yaml files. It’s safer to save it as a Kubernetes secret. Just like the API key, this secret will need to be labeled too.

```
kubectl create secret generic atlaspassword --from-literal="password=mernk8s"
kubectl label secret atlaspassword atlas.mongodb.com/type=credentials
```

## Create and manage an Atlas deployment

Congrats! You are now ready to manage your Atlas projects and deployments from Kubernetes. This can be done with the three new CRDs that were added to your cluster. Those CRDs are `AtlasProject` to manage projects, `AtlasDeployment` to manage deployments, and `AtlasDatabaseUser` to manage database users within MongoDB Atlas.

* Projects: Allows you to isolate different database environments (for instance, development/qa/prod environments) from each other, as well as users/teams.
* Deployments: Instance of MongoDB running on a cloud provider.
* Users: Database users that have access to MongoDB database deployments.

The process of creating a project, user, and deployment is demonstrated below, but feel free to skip down to simply apply these files by using the `/atlas` folder.
### Create a project

Start by creating a new project in which the new cluster will be deployed. In a new file called `/operator/project.yaml`, add the following:
```
apiVersion: atlas.mongodb.com/v1
kind: AtlasProject
metadata:
  name: mern-k8s-project
spec:
  name: "MERN K8s"
  projectIpAccessList:
    - ipAddress: "0.0.0.0/0"
      comment: "Allowing access to database from everywhere (only for Demo!)"
```

This will create a new project called "MERN K8s" in Atlas. Now, this project will be open to anyone on the web. It’s best practice to only open it to known IP addresses as mentioned in the comment.

### Create a new database user

Now, in order for your application to connect to this database, you will need a database user. To create this user, open a new file called `/operator/user.yaml`, and add the following:

```
apiVersion: atlas.mongodb.com/v1
kind: AtlasDatabaseUser
metadata:
  name: atlas-user
spec:
  roles:
    - roleName: "readWriteAnyDatabase"
      databaseName: "admin"
  projectRef:
    name: mern-k8s-project
  username: mernk8s
  passwordSecretRef:
    name: atlaspassword
```

You can see how the password uses the secret we created earlier, `atlaspassword`, in the `mern-k8s-project` namespace.

### Create a deployment

Finally, as you have a project setup and user to connect to the database, you can create a new deployment inside this project. In a new file called `/operator/deployment.yaml`, add the following yaml.

```
apiVersion: atlas.mongodb.com/v1
kind: AtlasDeployment
metadata:
  name: mern-k8s-cluster
spec:
  projectRef:
    name: mern-k8s-project
  deploymentSpec:
    name: "Cluster0"
    providerSettings:
      instanceSizeName: M0
      providerName: TENANT
      regionName: US_EAST_1
      backingProviderName: AWS
```

This will create a new M0 (free) deployment on AWS, in the US_EAST_1 region. Here, we’re referencing the `mern-k8s-project` in our Kubernetes namespace, and creating a cluster named `Cluster0`. You can use a similar syntax to deploy in any region on AWS, GCP, or Azure. 


### Apply the new files

You now have everything ready to create this new project and cluster. You can apply those new files to your cluster using:

```
kubectl apply -f ./operator
```

This will take a couple of minutes. You can see the status of the cluster and project creation with `kubectl`.

```
kubectl get atlasprojects
kubectl get atlasdeployments
```

In the meantime, you can go to the Atlas UI. The project should already be created, and you should see that a cluster is in the process of being created.

![Viewing the newly created cluster](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/database_deployments_18fac98898.jpg)

### Get your connection string

Getting your connection string to that newly created database can now be done through Kubernetes. Once your new database has been created, you can use the following command that uses `jq` to view the connection strings, without using the Atlas UI, by converting to JSON from Base64. 

```
kubectl get secret mern-k8s-cluster0-mernk8s -o json | jq -r '.data | with_entries(.value |= @base64d)'

{
…
  "connectionStringStandard": "![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/api_key_credentials_bcc611f1f0.png)",
  "connectionStringStandardSrv": "mongodb+srv://mernk8s:mernk8s@cluster0.fb4qw.mongodb.net",
  "password": "mernk8s",
  "username": "mernk8s"
}
```

## Configure the application back end using the Atlas operator

Now that your project and cluster are created, you can access the various properties from your Atlas instance. You can now access the connection string, and even configure your backend service to use that connection string. We’ll go ahead and connect our back end to our database without actually specifying the connection string, instead using the Kubernetes secret we just created.

### Update the backend deployment

Now that you can find your connection string from within Kubernetes, you can use that as part of your deployment to specify the connection string to your back end.

In your `/k8s/application.yaml` file, change the `env` section of the containers template to the following:

```
          env: 
            - name: PORT
              value: "3000"
            - name: "CONN_STR"
              valueFrom:
                secretKeyRef:
                  name: mern-k8s-cluster0-mernk8s
                  key: connectionStringStandardSrv
```

This will use the same connection string you've just seen in your terminal.

Since we’ve changed our deployment, you can apply those changes to your cluster using `kubectl`:

```
​​kubectl apply -f k8s/application.yaml
```

Now, if you take a look at your current pods:

```
kubectl get pods
```

You should see that your backend pods have been restarted. You should now be able to test the application with the back end connected to our newly created Atlas cluster. Now, just head to `localhost` to view the updated application once the deployment has restarted. You’ll see the application fully running, using this newly created cluster.  

![Fully functioning demo application, connected to MongoDB Atlas](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Travel_Demo_3d8885d4a0.gif)

In addition, as you add items or perhaps clear the entries of the travel planner, you’ll notice the entries added and removed from the “Collections” tab of the `Cluster0` database within the Atlas UI. Let’s take a look at our database using [MongoDB Compass](https://www.mongodb.com/products/compass), with username `mernk8s` and password `mernk8s` as we set previously.

![MongoDB Compass connected to our mern-k8s database](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/compass_entries_c4a8cb98ef.jpg)

### Delete project

Let’s finish off by using `kubectl` to delete the Atlas cluster and project and clean up our workspace. We can delete everything from the current namespace by using `kubectl delete`

 
```
kubectl delete atlasdeployment mern-k8s-cluster
kubectl delete atlasproject mern-k8s-project
```


## Summary

You now know how to leverage the MongoDB Atlas Operator to create and manage clusters from Kubernetes. We’ve only demonstrated a small bit of the functionality the operator provides, but feel free to head to the [documentation](https://docs.atlas.mongodb.com/atlas-operator/) to learn more.

If you are using MongoDB Enterprise instead of Atlas, there is also an [Operator](https://www.mongodb.com/try/download/enterprise-kubernetes-operator) available, which works in very similar fashion.

To go through the full lab by Joel Lord, which includes this guide and much more, check out the self-guided [Atlas Operator Workshop](https://joellord.github.io/mern-k8s/).

Get started with application deployment into a Kubernetes cluster using the MongoDB Atlas Operator.

How to Deploy an Application in Kubernetes With the MongoDB Atlas Operator

Johannes Brännström

Senior Solutions Architect

Pierre Petersson

Advisory Solutions Architect

## Why CI/CD?

To increase the speed and quality of development, you may use continuous delivery strategies to manage and deploy your application code changes. However, continuous delivery for databases is often a manual process.

Adopting continuous integration and continuous delivery (CI/CD) for managing the lifecycle of a database has the following benefits:

* An automated multi-environment setup enables you to move faster and focus on what really matters.
* The confidence level of the changes applied increases.
* The process is easier to reproduce.
* All changes to database configuration will be traceable.

### Why CI/CD for MongoDB Atlas?

[MongoDB Atlas](https://www.mongodb.com/cloud/atlas) is a multi-cloud modern database, providing an integrated suite of cloud database and data services to accelerate and simplify how you build with data. MongoDB Atlas also provides a comprehensive [API](https://docs.atlas.mongodb.com/api/atlas-admin-api/), making CI/CD for the actual data platform itself possible.

In this blog, we’ll demonstrate how to set up CI/CD for MongoDB Atlas, in a typical production setting. The intended audience is developers, solutions architects, and database administrators with knowledge of MongoDB Atlas, AWS, and Terraform.

## Our CI/CD Solution Requirements

* Ensure that each environment (dev, test, prod) is isolated to minimize blast radius in case of a human error or from a security perspective. MongoDB Atlas Projects and API Keys will be utilized to enable environment isolation.
* All services used in this solution will use managed services. This to minimize the time needed to spend on managing infrastructure.
* Minimize commercial agreements required. Use as much as possible from AWS and the Atlas ecosystem so that there is no need to purchase external tooling, such as HashiCorp Vault. 
* Minimize time spent on installing local dev tooling, such as git and Terraform. The solution will provide a docker image, with all tooling required to run provisioning of Terraform templates. The same image will be used to also run the pipeline in AWS CodeBuild. 

## Implementation

Enough talk—let’s get to the action. As developers, we love working examples as a way to understand how things work. So, here’s how we did it.

### Prerequisites

First off, we need to have at least an Atlas account to provision Atlas and then somewhere to run our automation. You can [get an Atlas account for free](https://www.mongodb.com/cloud/atlas/register) at mongodb.com. If you want to take this demo for a spin, take the time and create your Atlas account now. Next, you’ll need to [create an organization-level API key](https://docs.atlas.mongodb.com/tutorial/manage-programmatic-access/). If you or your org already have an Atlas account you’d like to use, you’ll need the organization owner to create the organization-level API key.

Second, you’ll need an AWS account. For more information on how to create an AWS account, see [How do I create an AWS account?](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) For this demo, we’ll be using some for-pay services like S3, but you get 12 months free. 

You will also need to have Docker installed as we are using a docker container to run all provisioning. For more information on how to install Docker, see [Get Started with Docker](https://www.docker.com/get-started). We are using Docker as it will make it easier for you to get started, as all the tooling is packaged in the container—such as AWS cli, mongosh, and Terraform.

### What You Will Build

* [MongoDB Atlas Projects](https://docs.atlas.mongodb.com/tutorial/manage-projects/) for dev, test, prod environments, to minimize blast radius in case of a human error and from a security perspective.
* [MongoDB Atlas Cluster](https://docs.atlas.mongodb.com/tutorial/create-new-cluster/) in each Atlas project (dev, test, prod). MongoDB Atlas is a fully managed data platform for modern applications. Storing data the way it is accessed as documents makes developers more productive. It provides a document-based database that is cost-efficient and resizable while automating time-consuming administration tasks such as hardware provisioning, database setup, patching, and backups. It allows you to focus on your applications by providing the foundation of high performance, high availability, security, and compatibility they need.
✅ Already have an AWS account? Atlas supports paying for usage via the AWS Marketplace (AWS MP) without any upfront commitment — simply <a href="https://aws.amazon.com/marketplace/pp/prodview-pp445qepfdy34?trk=38088f03-4928-4b00-9a39-86781df6ab9b&sc_channel=el" target="_blank" rel="noreferrer">sign up for MongoDB Atlas via AWS Marketplace.<a>
* [CodePipeline](https://aws.amazon.com/codepipeline/) orchestrates the CI/CD database migration stages.
* [IAM](https://aws.amazon.com/iam/) roles and policies allow cross-account access to applicable AWS resources.
* [CodeCommit](https://aws.amazon.com/codecommit/) creates a repo to store the SQL statements used when running the database migration.
* [Amazon S3](https://aws.amazon.com/s3/) creates a bucket to store pipeline artifacts.
* [CodeBuild](https://aws.amazon.com/codebuild) creates a project in target accounts using Flyway to apply database changes.
* VPC [security groups](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) ensure the secure flow of traffic between a CodeBuild project deployed within a VPC and MongoDB Atlas. AWS Private Link will also be provisioned.
* [AWS Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html) stores secrets securely and centrally, such as the Atlas API keys and database username and password.
* [Amazon SNS](https://aws.amazon.com/sns/) notifies you by email when a developer pushes changes to the CodeCommit repo.

### Step 1: Bootstrap AWS Resources

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/aws_codepipeline_161a68794c.png)

Next, we’ll fire off the script to bootstrap our AWS environment and Atlas account as shown in Diagram 1 using Terraform.

You will need to use [programmatic access keys](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html) for your AWS account and the Atlas organisation-level API key that you have created as described in the prerequisites.This is also the only time you’ll need to handle the keys manually. 

```
# Set your environment variables

# You'll find this in your Atlas console as described in prerequisites
export ATLAS_ORG_ID=60388113131271beaed5

# The public part of the Atlas Org key you created previously 
export ATLAS_ORG_PUBLIC_KEY=l3drHtms

# The private part of the Atlas Org key you created previously 
export ATLAS_ORG_PRIVATE_KEY=ab02313b-e4f1-23ad-89c9-4b6cbfa1ed4d


# Pick a username, the script will create this database user in Atlas
export DB_USER_NAME=demouser

# Pick a project base name, the script will appended -dev, -test, -prod depending on environment
export ATLAS_PROJECT_NAME=blogcicd6


# The AWS region you want to deploy into
export AWS_DEFAULT_REGION=eu-west-1


# The AWS public programmatic access key
export AWS_ACCESS_KEY_ID=AKIAZDDBLALOZWA3WWQ


# The AWS private programmatic access key
export AWS_SECRET_ACCESS_KEY=nmarrRZAIsAAsCwx5DtNrzIgThBA1t5fEfw4uJA


```

Once all the parameters are defined, you are ready to run the script that will create your CI/CD pipeline.

```
# Clone solution code repository
$ git clone https://github.com/mongodb-developer/atlas-cicd-aws
$ cd atlas-cicd

# Start docker container, which contains all the tooling e.g terraform, mongosh, and other, 
$ docker container run -it --rm -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_DEFAULT_REGION -e ATLAS_ORG_ID -e ATLAS_ORG_PUBLIC_KEY -e ATLAS_ORG_PRIVATE_KEY -e DB_USER_NAME -e ATLAS_PROJECT_NAME -v ${PWD}/terraform:/terraform piepet/cicd-mongodb:46


$ cd terraform   

# Bootstrap AWS account and Atlas Account
$ ./deploy_baseline.sh $AWS_DEFAULT_REGION $ATLAS_ORG_ID $ATLAS_ORG_PUBLIC_KEY $ATLAS_ORG_PRIVATE_KEY $DB_USER_NAME $ATLAS_PROJECT_NAME base apply




```

When deploy:baseline.sh is invoked, provisioning of AWS resources starts, using Terraform templates. The resources created are shown in Diagram 1.


From here on, you'll be able to operate your Atlas infrastructure without using your local docker instance. If you want to blaze through this guide, including cleaning it all up, you might as well keep the container running, though. The final step of tearing down the AWS infrastructure requires an external point like your local docker instance.

Until you’ve committed anything, the pipeline will have a failed Source stage. This is because it tries to check out a branch that does not exist in the code repository. After you’ve committed the Terraform code you want to execute, you’ll see that the Source stage will restart and proceed as expected. You can find the pipeline in the AWS console at this url: [https://eu-west-1.console.aws.amazon.com/codesuite/codepipeline/pipelines?region=eu-west-1](https://eu-west-1.console.aws.amazon.com/codesuite/codepipeline/pipelines?region=eu-west-1)

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/failing_pipeline_in_codepipeline_bc5629211b.png)

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/codepipeline_5ec9666ae0.png)

### Step 2: Deploy Atlas Cluster

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/deploying_an_atlas_cluster_f951a572c1.png)

Next is to deploy the Atlas cluster (projects, users, API keys, etc). This is done by pushing a configuration into the new AWS CodeCommit repo. 

If you’re like me and want to see how provisioning of the Atlas cluster works before setting up IAM properly, you can push the original github repo to AWS CodeCommit directly inside the docker container (inside the Terraform folder) using a bit of a hack. By pushing to the CodeCommit repo, AWS CodePipeline will be triggered and provisioning of the Atlas cluster will start. 

```
cd /terraform
# Push default settings to AWS Codecommit
./git_push_terraform.sh


```

To set up access to the CodeCommit repo properly, for use that survives stopping the docker container, you’ll need a proper git CodeCommit user. Follow the steps in the [AWS documentation](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html) to create and configure your CodeCommit git user in AWS IAM. Then clone the AWS CodeCommit repository that was created in the bootstrapping, outside your docker container, perhaps in another tab in your shell, using your IAM credentials. If you did not use the “hack” to initialize it, it’ll be empty, so copy the Terraform folder that is provided in this solution, to the root of the cloned CodeCommit repository, then commit and push to kick off the pipeline. Now you can use this repo to control your setup! You should now see in the AWS CodePipeline console that the pipeline has been triggered. The pipeline will create Atlas clusters in each of the Atlas Projects and configure AWS PrivateLink. 

Let’s dive into the stages [defined in this Terraform pipeline file](https://github.com/mongodb-developer/atlas-cicd-aws/blob/main/terraform/modules/aws/code-pipeline/main.tf).

**Deploy-Base**
This is basically re-applying what we did in the bootstrapping. This stage ensures we can improve on the AWS pipeline infrastructure itself over time.

This stage creates the projects in Atlas, including Atlas project API keys, Atlas project users, and database users. 

**Deploy-Dev**

This stage creates the corresponding Private Link and MongoDB cluster.

**Deploy-Test**

This stage creates the corresponding Private Link and MongoDB cluster.

**Deploy-Prod**

This stage creates the corresponding Private Link and MongoDB cluster.

**Gate**

Approving means we think it all looks good. Perhaps counter intuitively but great for demos, it proceeds to teardown. This might be one of the first behaviours you’ll change. :)

**Teardown**

This decommissions the dev, test, and prod resources we created above. To decommission the base resources, including the pipeline itself, we recommend you run that externally—for example, from the Docker container on your laptop. We’ll cover that later.

As you advance towards the Gate stage, you’ll see the Atlas clusters build out. Below is an example where the Test stage is creating a cluster. Approving the Gate will undeploy the resources created in the dev, test, and prod stages, but keep projects and users.

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/project_names_and_clusters_041271acd0.png)

### Step 3: Make a Change!

Assuming you took the time to set up IAM properly, you can now work with the infrastructure as code directly from your laptop outside the container. If you just deployed using the hack inside the container, you can continue interacting using the repo created inside the Docker container, but at some point, the container will stop and that repo will be gone. So, beware.

Navigate to the root of the clone of the CodeCommit repo. For example, if you used the script in the container, you’d run, also in the container:

```
cd /${ATLAS_PROJECT_NAME}-base-repo/

```

Then you can edit, for example, the MongoDB version by changing 4.4 to 5.0 in `terraform/environment/dev/variables.tf`.

```
variable "cluster_mongodbversion" {
 description = "The Major MongoDB Version"
 default = "5.0"
}
```

Then push (git add, commit, push) and you’ll see a new run initiated in CodePipeline.

### Step 4: Clean Up Base Infrastructure

Now, that was interesting. Time for cleaning up! To decommission the full environment, you should first approve the Gate stage to execute the teardown job. When that’s been done, only the base infrastructure remains. Start the container again as in Step 1 if it’s not running, and then execute deploy_baseline.sh, replacing the word ***apply*** with ***destroy***: 

```
# inside the /terraform folder of the container

# Clean up AWS and Atlas Account
./deploy_baseline.sh $AWS_DEFAULT_REGION $ATLAS_ORG_ID $ATLAS_ORG_PUBLIC_KEY $ATLAS_ORG_PRIVATE_KEY $DB_USER_NAME $ATLAS_PROJECT_NAME base destroy

```

## Lessons Learned

In this solution, we have separated the creation of AWS resources and the Atlas cluster, as the changes to the Atlas cluster will be more frequent than the changes to the AWS resources. 

When implementing infrastructure as code for a MongoDB Atlas Cluster, you have to consider not just the cluster creation but also a strategy for how to separate dev, qa, and prod environments and how to store secrets. This to minimize blast radius. 

We also noticed how useful resource tagging is to make Terraform scripts portable. By setting tags on AWS resources, the script does not need to know the names of the resources but can look them up by tag instead.

## Conclusion

By using CI/CD automation for Atlas clusters, you can speed up deployments and increase the agility of your software teams. 

MongoDB Atlas offers a powerful API that, in combination with AWS CI/CD services and Terraform, can support continuous delivery of MongoDB Atlas clusters, and version-control the database lifecycle. You can apply the same pattern with other CI/CD tools that aren’t specific to AWS. 

In this blog, we’ve offered an exhaustive, reproducible, and reusable deployment process for MongoDB Atlas, including traceability. A devops team can use our demonstration as inspiration for how to quickly deploy MongoDB Atlas, automatically embedding organisation best practices. 

In this blog, we’ll demonstrate how to set up CI/CD for MongoDB Atlas, in a typical production setting.

Building a Multi-Environment Continuous Delivery Pipeline for MongoDB Atlas

Rachelle Palmer

Product Lead, Developer Experience

Looking for a code example that has microservices in Go with Docker, Kubernetes and MongoDB? Look no further!

Cinema is an example project which demonstrates the use of microservices for a fictional movie theater. The Cinema backend is powered by 4 microservices, all of which happen to be written in Go, using MongoDB for manage the database and Docker to isolate and deploy the ecosystem.

Movie Service: Provides information like movie ratings, title, etc.
Show Times Service: Provides show times information.
Booking Service: Provides booking information.
Users Service: Provides movie suggestions for users by communicating with other services.

This project is available to clone or fork on [github](https://github.com/mmorejon/microservices-docker-go-mongodb) from [Manuel Morejón](https://twitter.com/morejon85). 


Looking for a code example that has microservices in Go with Docker, Kubernetes, and MongoDB? Look no further!

Cinema: Example Go Microservices Application

MongoDB

Maxime Beugnet

Senior Developer Advocate

## What is Quarkus?

When we write a traditional Java application, our Java source code is compiled and transformed into Java bytecode.
This bytecode can then be executed by a Java virtual machine (JVM) specific to the operating system you are
running. This is why we can say that Java is a portable language. You compile once, and you can run it everywhere,
as long as you have the right JVM on the right machine.

This is a great mechanism, but it comes at a cost. Starting a program is slow because the JVM and the entire context
need to be loaded first before running anything. It's not memory-efficient because we need to load hundreds of classes that might not be used at all in the end as the [classpath scanning only occurs after](https://quarkus.io/container-first/).

This was perfectly fine in the old monolithic realm, but this is totally unacceptable in the new world made of lambda
functions, cloud, containers, and Kubernetes. In this context, a low memory footprint and a lightning-fast startup time
are absolutely mandatory.

This is where [Quarkus](https://quarkus.io/about/) comes in. Quarkus is a Kubernetes-native Java framework tailored
for [GraalVM](https://www.graalvm.org/) and [HotSpot](https://en.wikipedia.org/wiki/HotSpot_(virtual_machine)).

With Quarkus, you can build native binaries that can boot and send their first response in 0.042 seconds versus 9.5
seconds for a traditional Java application.

![Quarkus startup time](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltc34b97d1a8137f01/661dddb79cbc2a964ea9cc2a/quarkus.png)

In this tutorial, we are going to build a Quarkus application that can manage a `persons` collection in MongoDB. The
goal is to perform four simple CRUD operations with a REST API using a native application.

## Prerequisites

For this tutorial, you'll need:

- [cURL](https://curl.se/).
- [Docker](https://docs.docker.com/engine/install/).
- [GraalVM](https://www.graalvm.org/latest/docs/getting-started/).
- A [MongoDB Atlas](https://www.mongodb.com/atlas/database) cluster or a local instance. I'll use a Docker container in
  this tutorial.

If you don't want to code along and prefer to check out directly the final code:

```bash
git clone git@github.com:mongodb-developer/quarkus-mongodb-crud.git
```

## How to set up Quarkus with MongoDB

**TL;DR**:
Use [this link](https://code.quarkus.io/?g=com.mongodb&a=quarkus-mongodb-crud&e=mongodb-client&e=smallrye-openapi&e=rest&e=rest-jackson)
and click on `generate your application` or clone
the [GitHub repository](https://github.com/mongodb-developer/quarkus-mongodb-crud).

The easiest way to get your project up and running with Quarkus and all the dependencies you need is to
use [https://code.quarkus.io/](https://code.quarkus.io/).

Similar to [Spring initializr](https://start.spring.io/), the Quarkus project starter website will help you
select your dependencies and build your Maven or Gradle configuration file. Some dependencies will also include a
starter code to assist you in your first steps.

For our project, we are going to need:

- MongoDB client [quarkus-mongodb-client].
- SmallRye OpenAPI [quarkus-smallrye-openapi].
- REST [quarkus-rest].
- REST Jackson [quarkus-rest-jackson].

Feel free to use the `group` and `artifact` of your choice. Make sure the Java version matches the version of your
GraalVM version, and we are ready to go.

Download the zip file and unzip it in your favorite project folder. Once it's done, take some time to read the README.md
file provided.

Finally, we need a MongoDB cluster. Two solutions:

- Create a new cluster on [MongoDB Atlas](https://www.mongodb.com/atlas/database) and retrieve the connection string, or
- Create an ephemeral single-node replica set with Docker.

```bash
docker run --rm -d -p 27017:27017 -h $(hostname) --name mongo mongo:latest --replSet=RS && sleep 5 && docker exec mongo mongosh --quiet --eval "rs.initiate();"
```

Either way, the next step is to set up your connection string in the `application.properties` file.

```properties
quarkus.mongodb.connection-string=mongodb://localhost:27017
```

## CRUD operations in Quarkus with MongoDB

Now that our Quarkus project is ready, we can start developing.

First, we can start the developer mode which includes live coding (automatic refresh) without the need to restart the
program.

```bash
./mvnw compile quarkus:dev
```

The developer mode comes with two handy features:

- [Swagger UI](http://localhost:8080/q/swagger-ui/#/)
- [Quarkus Dev UI](http://localhost:8080/q/dev/)

Feel free to take some time to explore both these UIs and see the capabilities they offer.

Also, as your service is now running, you should be able to receive your first HTTP communication. Open a new terminal and execute the following query:

```bash
curl http://localhost:8080/hello
```

> Note: If you cloned the repo, then it’s `/api/hello`. We are changing this below in a minute.

Result:

```
Hello from Quarkus REST
```

This works because your project currently contains a single class `GreetingResource.java` with the following code.

```java
package com.mongodb;

import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;

@Path("/hello")
public class GreetingResource {

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        return "Hello from Quarkus REST";
    }
}
```

### PersonEntity

"Hello from Quarkus REST" is nice, but it's not our goal! We want to manipulate data from a `persons` collection in
MongoDB.

Let's create a classic `PersonEntity.java` [POJO](https://en.wikipedia.org/wiki/Plain_old_Java_object) class. I created
it in the default `com.mongodb` package which is my `group` from earlier. Feel free to change it.

```java
package com.mongodb;

import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.fasterxml.jackson.databind.ser.std.ToStringSerializer;
import org.bson.types.ObjectId;

import java.util.Objects;

public class PersonEntity {

    @JsonSerialize(using = ToStringSerializer.class)
    public ObjectId id;
    public String name;
    public Integer age;

    public PersonEntity() {
    }

    public PersonEntity(ObjectId id, String name, Integer age) {
        this.id = id;
        this.name = name;
        this.age = age;
    }

    @Override
    public int hashCode() {
        int result = id != null ? id.hashCode() : 0;
        result = 31 * result + (name != null ? name.hashCode() : 0);
        result = 31 * result + (age != null ? age.hashCode() : 0);
        return result;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        PersonEntity that = (PersonEntity) o;

        if (!Objects.equals(id, that.id)) return false;
        if (!Objects.equals(name, that.name)) return false;
        return Objects.equals(age, that.age);
    }

    public ObjectId getId() {
        return id;
    }

    public void setId(ObjectId id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public Integer getAge() {
        return age;
    }

    public void setAge(Integer age) {
        this.age = age;
    }
}
```

We now have a class to map our MongoDB documents to using [Jackson](https://github.com/FasterXML/jackson).

### PersonRepository

Now that we have a `PersonEntity`, we can create a `PersonRepository` template, ready to welcome our CRUD queries.

Create a `PersonRepository.java` class next to the `PersonEntity.java` one.

```java
package com.mongodb;

import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoCollection;
import jakarta.enterprise.context.ApplicationScoped;

@ApplicationScoped
public class PersonRepository {

    private final MongoClient mongoClient;
    private final MongoCollection<PersonEntity> coll;

    public PersonRepository(MongoClient mongoClient) {
        this.mongoClient = mongoClient;
        this.coll = mongoClient.getDatabase("test").getCollection("persons", PersonEntity.class);
    }

    // CRUD methods will go here

}
```

### PersonResource

We are now almost ready to create our first CRUD method. Let's update the default `GreetingResource.java` class to match
our goal.

1. Rename the file `GreetingResource.java` to `PersonResource.java`.
2. In the `test` folder, also rename the default test files to `PersonResourceIT.java` and `PersonResourceTest.java`.
3. Update `PersonResource.java` like this:

```java
package com.mongodb;

import jakarta.inject.Inject;
import jakarta.ws.rs.*;
import jakarta.ws.rs.core.MediaType;

@Path("/api")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public class PersonResource {

    @Inject
    PersonRepository personRepository;

    @GET
    @Path("/hello")
    public String hello() {
        return "Hello from Quarkus REST";
    }

    // CRUD routes will go here

}
```

> Note that with the `@Path("/api")` annotation, the URL of our `/hello` service is now `/api/hello`.

As a consequence, update `PersonResourceTest.java` so our test keeps working.

```java
package com.mongodb;

import io.quarkus.test.junit.QuarkusTest;
import org.junit.jupiter.api.Test;

import static io.restassured.RestAssured.given;
import static org.hamcrest.CoreMatchers.is;

@QuarkusTest
class PersonResourceTest {
    @Test
    void testHelloEndpoint() {
        given().when().get("/api/hello").then().statusCode(200).body(is("Hello from Quarkus REST"));
    }
}
```

### Create a person

All the code blocks are now in place. We can create our first route to be able to create a new person.

In
the [repository](https://github.com/mongodb-developer/quarkus-mongodb-crud/blob/main/src/main/java/com/mongodb/PersonRepository.java),
add the following method that inserts a `PersonEntity` and returns the inserted document's `ObjectId` in `String`
format.

```java
public String add(PersonEntity person) {
    return coll.insertOne(person).getInsertedId().asObjectId().getValue().toHexString();
}
```

In
the [resource](https://github.com/mongodb-developer/quarkus-mongodb-crud/blob/main/src/main/java/com/mongodb/PersonResource.java)
file, we can create the corresponding route:

```java
@POST
@Path("/person")
public String createPerson(PersonEntity person) {
    return personRepository.add(person);
}
```

Without restarting the project (remember the dev mode?), you should be able to test this route.

```bash
curl -X POST http://localhost:8080/api/person \
     -H 'Content-Type: application/json' \
     -d '{"name": "John Doe", "age": 30}'
```

This should return the `ObjectId` of the new `person` document.

```
661dccf785cd323349ca42f7
```

If you connect to the MongoDB instance with [mongosh](https://www.mongodb.com/docs/mongodb-shell/), you can confirm that
the document made it:

```
RS [direct: primary] test> db.persons.find()
[
  {
    _id: ObjectId('661dccf785cd323349ca42f7'),
    age: 30,
    name: 'John Doe'
  }
]
```

### Read persons

Now, we can read all the persons in the database, for example.

In
the [repository](https://github.com/mongodb-developer/quarkus-mongodb-crud/blob/main/src/main/java/com/mongodb/PersonRepository.java),
add:

```java
public List<PersonEntity> getPersons() {
    return coll.find().into(new ArrayList<>());
}
```

In
the [resource](https://github.com/mongodb-developer/quarkus-mongodb-crud/blob/main/src/main/java/com/mongodb/PersonResource.java),
add:

```java
@GET
@Path("/persons")
public List<PersonEntity> getPersons() {
    return personRepository.getPersons();
}
```

Now, we can retrieve all the persons in our database:

```bash
curl http://localhost:8080/api/persons
```

This returns a list of persons:

```json
[
  {
    "id": "661dccf785cd323349ca42f7",
    "name": "John Doe",
    "age": 30
  }
]
```

### Update person

It's John Doe's anniversary! Let's increment his age by one.

In
the [repository](https://github.com/mongodb-developer/quarkus-mongodb-crud/blob/main/src/main/java/com/mongodb/PersonRepository.java),
add:

```java
public long anniversaryPerson(String id) {
    Bson filter = eq("_id", new ObjectId(id));
    Bson update = inc("age", 1);
    return coll.updateOne(filter, update).getModifiedCount();
}
```

In
the [resource](https://github.com/mongodb-developer/quarkus-mongodb-crud/blob/main/src/main/java/com/mongodb/PersonResource.java),
add:

```java
@PUT
@Path("/person/{id}")
public long anniversaryPerson(@PathParam("id") String id) {
    return personRepository.anniversaryPerson(id);
}
```

Time to test this party:

```bash
curl -X PUT http://localhost:8080/api/person/661dccf785cd323349ca42f7
```

This returns `1` which is the number of modified document(s). If the provided `ObjectId` doesn't match a person's id,
then it returns `0` and MongoDB doesn't perform any update.

### Delete person

Finally, it's time to delete John Doe...

In
the [repository](https://github.com/mongodb-developer/quarkus-mongodb-crud/blob/main/src/main/java/com/mongodb/PersonRepository.java),
add:

```java
public long deletePerson(String id) {
    Bson filter = eq("_id", new ObjectId(id));
    return coll.deleteOne(filter).getDeletedCount();
}
```

In
the [resource](https://github.com/mongodb-developer/quarkus-mongodb-crud/blob/main/src/main/java/com/mongodb/PersonResource.java),
add:

```java
@DELETE
@Path("/person/{id}")
public long deletePerson(@PathParam("id") String id) {
    return personRepository.deletePerson(id);
}
```

Let's test:

```bash
curl -X DELETE http://localhost:8080/api/person/661dccf785cd323349ca42f7
```

Again, it returns `1` which is the number of deleted document(s).

Now that we have a working Quarkus application with a MongoDB CRUD service, it's time to experience the full
power of Quarkus.

## Quarkus native build

Quit the developer mode by simply hitting the `q` key in the relevant terminal.

It's time to build
the [native executable](https://www.graalvm.org/latest/reference-manual/native-image/guides/build-static-executables/)
that we can use in production with GraalVM and experience the *insanely* fast start-up time.

Use this command line to build directly with your local GraalVM and other dependencies.

```bash
./mvnw package -Dnative
```

Or use the Docker image that contains everything you need:

```bash
./mvnw package -Dnative -Dquarkus.native.container-build=true
```

The final result is a native application, ready to be launched, in your `target` folder.

```bash
./target/quarkus-mongodb-crud-1.0.0-SNAPSHOT-runner
```

On my laptop, it starts in **just 0.019s**! Remember how much time Spring Boot needs to start an application and respond
to queries for the first time?!

You can read more about how Quarkus makes this miracle a reality in
the [container first documentation](https://quarkus.io/container-first/).

## Conclusion

In this tutorial, we've explored how Quarkus and MongoDB can team up to create a lightning-fast RESTful API with CRUD
capabilities.

Now equipped with these insights, you're ready to build blazing-fast APIs with Quarkus, GraalVM, and MongoDB. Dive into
the
provided [GitHub repository](https://github.com/mongodb-developer/quarkus-mongodb-crud) for more details.

> If you have questions, please head to our [Developer Community website](https://community.mongodb.com/) where the
> MongoDB engineers and the MongoDB community will help you build your next big idea with MongoDB.


Explore the seamless integration of Quarkus, GraalVM, and MongoDB for lightning-fast CRUD RESTful APIs. Harness Quarkus' rapid startup time and Kubernetes compatibility for streamlined deployment.

How to Build a CRUD Application With MongoDB, Quarkus, and GraalVM

## Introduction

"Microservices are awesome and monolithic applications are evil."

If you are reading this article, you have already read that a million times, and I'm not the one who's going to tell
you otherwise!

In this post, we are going to create a microservices architecture using MongoDB.

## TL;DR

The source code is available in these two repositories.

The [README.md](https://github.com/mongodb-developer/microservices-architecture-mongodb/blob/main/README.md) files will
help you start everything.

```bash
git clone git@github.com:mongodb-developer/microservices-architecture-mongodb.git
git clone git@github.com:mongodb-developer/microservices-architecture-mongodb-config-repo.git
```

## Microservices architecture

We are going to use Spring Boot and Spring Cloud dependencies to build our architecture.

Here is what a microservices architecture looks like, [according to Spring](https://spring.io/microservices):

![Spring Microservices Architecture][1]

We need several services to run this project. Let's talk about them one by one.

As you read this post, I suggest that you follow the instructions in the [README.md](https://github.com/mongodb-developer/microservices-architecture-mongodb/blob/main/README.md) file and start the service related to each section.

### Config server

The first service that we need is a configuration server.

This service allows us to store all the configuration files of our microservices in a single repository so our
configurations are easy to version and store.

The configuration of our config server is simple and straight to the point:

```properties
spring.application.name=config-server
server.port=8888
spring.cloud.config.server.git.uri=${HOME}/Work/microservices-architecture-mongodb-config-repo
spring.cloud.config.label=main
```

It allows us to locate the git repository that stores our microservices configuration and the branch that should be
used.

> Note that the only "trick" you need in your Spring Boot project to start a config server is the `@EnableConfigServer`
> annotation.

```java
package com.mongodb.configserver;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.config.server.EnableConfigServer;

@EnableConfigServer
@SpringBootApplication
public class ConfigServerApplication {
    public static void main(String[] args) {
        SpringApplication.run(ConfigServerApplication.class, args);
    }
}
```

### Service registry

A service registry is like a phone book for microservices. It keeps track of which microservices are running and where
they are located (IP address and port). Other services can look up this information to find and communicate with the
microservices they need.

A service registry is useful because it enables client-side load balancing and decouples service providers from
consumers without the need for DNS.

Again, you don't need much to be able to start a Spring Boot service registry. The `@EnableEurekaServer` annotation
makes all the magic happen.

```java
package com.mongodb.serviceregistry;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer;

@SpringBootApplication
@EnableEurekaServer
public class ServiceRegistryApplication {
    public static void main(String[] args) {
        SpringApplication.run(ServiceRegistryApplication.class, args);
    }
}
```

The configuration is also to the point:

```properties
spring.application.name=service-registry
server.port=8761
eureka.client.register-with-eureka=false
eureka.client.fetch-registry=false
```

> The last two lines prevent the service registry from registering to itself and retrieving the registry from itself.

### API gateway

The API gateway service allows us to have a single point of entry to access all our microservices. Of course, you should
have more than one in production, but all of them will be able to communicate with all the microservices and distribute
the workload evenly by load-balancing the queries across your pool of microservices.

Also, an API gateway is useful to address cross-cutting concerns like security, monitoring, metrics gathering, and
resiliency.

When our microservices start, they register themselves to the service registry. The API gateway can use this registry to
locate the microservices and distribute the queries according to its routing configuration.

```Shell
server:
  port: 8080

spring:
  application:
    name: api-gateway
  cloud:
    gateway:
      routes:
        - id: company-service
          uri: lb://company-service
          predicates:
            - Path=/api/company/**,/api/companies
        - id: employee-service
          uri: lb://employee-service
          predicates:
            - Path=/api/employee/**,/api/employees

eureka:
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://localhost:8761/eureka/
  instance:
    hostname: localhost
```

> Note that our API gateway runs on port 8080.

### MongoDB microservices

Finally, we have our MongoDB microservices.

Microservices are supposed to be independent of each other. For this reason, we need two MongoDB instances: one for
each microservice.

Check out the [README.md](https://github.com/mongodb-developer/microservices-architecture-mongodb/blob/main/README.md)
file to run everything.

> Note that in
> the [configuration files](https://github.com/mongodb-developer/microservices-architecture-mongodb-config-repo) for the
> company and employee services, they are respectively running on ports 8081 and 8082.

[company-service.properties](https://github.com/mongodb-developer/microservices-architecture-mongodb-config-repo/blob/main/company-service.properties)

```properties
spring.data.mongodb.uri=${MONGODB_URI_1:mongodb://localhost:27017}
spring.threads.virtual.enabled=true
management.endpoints.web.exposure.include=*
management.info.env.enabled=true
info.app.name=Company Microservice
info.app.java.version=21
info.app.type=Spring Boot
server.port=8081
eureka.client.register-with-eureka=true
eureka.client.fetch-registry=true
eureka.client.service-url.defaultZone=http://localhost:8761/eureka/
eureka.instance.hostname=localhost
```

[employee-service.properties](https://github.com/mongodb-developer/microservices-architecture-mongodb-config-repo/blob/main/employee-service.properties)

```properties
spring.data.mongodb.uri=${MONGODB_URI_2:mongodb://localhost:27018}
spring.threads.virtual.enabled=true
management.endpoints.web.exposure.include=*
management.info.env.enabled=true
info.app.name=Employee Microservice
info.app.java.version=21
info.app.type=Spring Boot
server.port=8082
eureka.client.register-with-eureka=true
eureka.client.fetch-registry=true
eureka.client.service-url.defaultZone=http://localhost:8761/eureka/
eureka.instance.hostname=localhost
```

> Note that the two microservices are connected to two different MongoDB clusters to keep their independence. The
> company service is using the MongoDB node on port 27017 and the employee service is on port 27018.

Of course, this is only if you are running everything locally. In production, I would recommend to use two clusters on
MongoDB Atlas. You can overwrite the MongoDB URI with the environment variables (see [README.md](https://github.com/mongodb-developer/microservices-architecture-mongodb/blob/main/README.md)).

## Test the REST APIs

At this point, you should have five services running:

- A config-server on port 8888
- A service-registry on port 8761
- An api-gateway on port 8080
- Two microservices:
  - company-service on port 8081
  - employee-service on port 8082

And two MongoDB nodes on ports 27017 and 27018 or two MongoDB clusters on MongoDB Atlas.

If you start the
script [2_api-tests.sh](https://github.com/mongodb-developer/microservices-architecture-mongodb/blob/main/2_api-tests.sh),
you should get an output like this.

```
DELETE Companies
2
DELETE Employees
2

POST Company 'MongoDB'
POST Company 'Google'

GET Company 'MongoDB' by 'id'
{
    "id": "661aac7904e1bf066ee8e214",
    "name": "MongoDB",
    "headquarters": "New York",
    "created": "2009-02-11T00:00:00.000+00:00"
}

GET Company 'Google' by 'name'
{
    "id": "661aac7904e1bf066ee8e216",
    "name": "Google",
    "headquarters": "Mountain View",
    "created": "1998-09-04T00:00:00.000+00:00"
}

GET Companies
[
    {
        "id": "661aac7904e1bf066ee8e214",
        "name": "MongoDB",
        "headquarters": "New York",
        "created": "2009-02-11T00:00:00.000+00:00"
    },
    {
        "id": "661aac7904e1bf066ee8e216",
        "name": "Google",
        "headquarters": "Mountain View",
        "created": "1998-09-04T00:00:00.000+00:00"
    }
]

POST Employee Maxime
POST Employee Tim

GET Employee 'Maxime' by 'id'
{
    "id": "661aac79cf04401110c03516",
    "firstName": "Maxime",
    "lastName": "Beugnet",
    "company": "Google",
    "headquarters": "Mountain View",
    "created": "1998-09-04T00:00:00.000+00:00",
    "joined": "2018-02-12T00:00:00.000+00:00",
    "salary": 2468
}

GET Employee 'Tim' by 'id'
{
    "id": "661aac79cf04401110c03518",
    "firstName": "Tim",
    "lastName": "Kelly",
    "company": "MongoDB",
    "headquarters": "New York",
    "created": "2009-02-11T00:00:00.000+00:00",
    "joined": "2023-08-23T00:00:00.000+00:00",
    "salary": 13579
}

GET Employees
[
    {
        "id": "661aac79cf04401110c03516",
        "firstName": "Maxime",
        "lastName": "Beugnet",
        "company": "Google",
        "headquarters": "Mountain View",
        "created": "1998-09-04T00:00:00.000+00:00",
        "joined": "2018-02-12T00:00:00.000+00:00",
        "salary": 2468
    },
    {
        "id": "661aac79cf04401110c03518",
        "firstName": "Tim",
        "lastName": "Kelly",
        "company": "MongoDB",
        "headquarters": "New York",
        "created": "2009-02-11T00:00:00.000+00:00",
        "joined": "2023-08-23T00:00:00.000+00:00",
        "salary": 13579
    }
]
```

> Note that the employee service sends queries to the company service to retrieve the details of the employees' company.

This confirms that the service registry is doing its job correctly because the URL only contains a reference to the company microservice, not its direct IP and port.

```java
private CompanyDTO getCompany(String company) {
    String url = "http://company-service/api/company/name/";
    CompanyDTO companyDTO = restTemplate.getForObject(url + company, CompanyDTO.class);
    if (companyDTO == null) {
        throw new EntityNotFoundException("Company not found: ", company);
    }
    return companyDTO;
}
```

## Conclusion

And voilà! You now have a basic microservice architecture running that is easy to use to kickstart your project.

In this architecture, we could seamlessly integrate additional features to enhance performance and maintainability in
production. Caching would be essential, particularly with a potentially large number of employees within the same
company, significantly alleviating the load on the company service.

The addition of a [Spring Cloud Circuit Breaker](https://spring.io/projects/spring-cloud-circuitbreaker) could also
improve the resiliency in production and a [Spring Cloud Sleuth](https://spring.io/projects/spring-cloud-sleuth) would
help with distributed tracing and auto-configuration.

If you have questions, please head to our [Developer Community website](https://www.mongodb.com/community/forums/) where the MongoDB engineers and the MongoDB community will help you build your next big idea with MongoDB.


[1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt332394d666c28140/661ab5bf188d353a3e2da005/microservices-architecture.svg


Learn about microservices architecture and deploy your first architecture locally using Spring Boot, Spring Cloud, and MongoDB.

Microservices Architecture: Deploy Locally With Java, Spring, and MongoDB

Community

Frank Steimle

Consultant at Digital Frontiers

Atlas Search enables you to perform full-text queries on your MongoDB database. In this post, I want to show how you can use test containers to write integration tests for Atlas Search-based queries, so that you can run them locally and in your CI/CD pipeline without the need to connect to an actual MongoDB Atlas instance.

TL;DR: All the source code explained in this post is available on [GitHub](https://github.com/mongodb-developer/atlas-search-local-testing):

```bash
git clone git@github.com:mongodb-developer/atlas-search-local-testing.git
```

MongoDB Atlas Search is a powerful combination of a document-oriented database and full-text search capabilities. This is not only valuable for use cases where you want to perform full-text queries on your data. With Atlas Search, it is possible to easily enable use cases that would be hard to implement in standard MongoDB due to certain limitations.

Some of these limitations hit us in a recent project in which we developed a webshop. The rather obvious requirement for this shop included that customers should be able to filter products and that the filters should show how many items are available in each category. Over the course of the project, we kept increasing the number of filters in the application. This led to two problems:

- We wanted customers to be able to arbitrarily choose filters. Since every filter needs an index to run efficiently, and since indexes can‘t be combined (intersected), this leads to a proliferation of indexes that are hard to maintain (in addition MongoDB allows only 64 indexes, adding another complexity level).

- With an increasing number of filters, the calculation of the facets for indicating the number of available items in each category also gets more complex and more expensive.

As the developer effort to handle this complexity with standard MongoDB tools grew larger over time, we decided to give Atlas Search a try. We knew that Atlas Search is an embedded full-text search in MongoDB Atlas based on Apache Lucene and that Lucene is a mighty tool for text search, but we were actually surprised at how well it supports our filtering use case.

With Atlas Search, you can create one or more so-called search indexes that contain your documents as a whole or just parts of them. Therefore, you can use just one index for all of your queries without the need to maintain additional indexes, e.g., for the most used filter combinations. Plus, you can also use the search index to calculate the facets needed to show item availability without writing complex queries that are not 100% backed up by an index.

The downside of this approach is that Atlas Search makes it harder to write unit or integration tests. When you're using standard MongoDB, you'll easily find some plug-ins for your testing framework that provide an in-memory MongoDB to run your tests against, or you use some kind of test container to set the stage for your tests. Although Atlas Search queries seamlessly integrate into MongoDB aggregation pipelines on Atlas, standard MongoDB cannot process this type of aggregation stage.

To solve this problem, the recently released Atlas CLI allows you to start a local instance of a MongoDB cluster that can actually handle Atlas Search queries. Internally, it starts two containers, and after deploying your search index via CLI, you can run your tests locally against these containers. While this allows you to run your tests locally, it can be cumbersome to set up this local cluster and start/stop it every time you want to run your tests. This has to be done by each developer on their local machine, adds complexity to the onboarding of new people working on the software, and is rather hard to integrate into a CI/CD pipeline.

Therefore, we asked ourselves if there is a way to provide a solution that does not need a manual setup for these containers and enables automatic start and shutdown. Turns out there is a way to do just that, and the solution we found is, in fact, a rather lean and reusable one that can also help with automated testing in your project.

## Preparing test containers

The key idea of test containers is to provide a disposable environment for testing. As the name suggests, it is based on containers, so in the first step, we need a Docker image or a [Docker Compose](https://docs.docker.com/compose/) script to start with.

Atlas CLI uses two Docker images to create an environment that enables testing Atlas Search queries locally:
mongodb/mongodb-enterprise-server is responsible for providing database capabilities and mongodb/mongodb-atlas-search is providing full-text search capabilities. Both containers are part of a MongoDB cluster, so they need to communicate with each other.

Based on this information, we can create a docker–compose.yml, where we define two containers, create a network, and set some parameters in order to enable the containers to talk to each other. The example below shows the complete docker–compose.yml needed for this article. The naming of the containers is based on the naming convention of the Atlas Search architecture: The `mongod` container provides the database capabilities while the `mongot` container provides the full-text search capabilities. As both containers need to know each other, we use environment variables to let each of them know where to find the other one. Additionally, they need a shared secret in order to connect to each other, so this is also defined using another environment variable.

```bash
version: "2"

services:
    mongod:
        container_name: mongod
        image: mongodb/mongodb-enterprise-server:7.0-ubi8
        entrypoint: "/bin/sh -c \"echo \"$$KEYFILECONTENTS\" > \"$$KEYFILE\"\n\nchmod 400 \"$$KEYFILE\"\n\n\npython3 /usr/local/bin/docker-entrypoint.py mongod --transitionToAuth --keyFile \"$$KEYFILE\" --replSet \"$$REPLSETNAME\" --setParameter \"mongotHost=$$MONGOTHOST\" --setParameter \"searchIndexManagementHostAndPort=$$MONGOTHOST\"\""
        environment:
            MONGOTHOST: 10.6.0.6:27027
            KEYFILE: /data/db/keyfile
            KEYFILECONTENTS: sup3rs3cr3tk3y
            REPLSETNAME: local
        ports:
            - 27017:27017
        networks:
            network:
                ipv4_address: 10.6.0.5
    mongot:
        container_name: mongot
        image: mongodb/mongodb-atlas-search:preview
        entrypoint: "/bin/sh -c \"echo \"$$KEYFILECONTENTS\" > \"$$KEYFILE\"\n\n/etc/mongot-localdev/mongot  --mongodHostAndPort \"$$MONGOD_HOST_AND_PORT\" --keyFile \"$$KEYFILE\"\""
        environment:
            MONGOD_HOST_AND_PORT: 10.6.0.5:27017
            KEYFILE: /var/lib/mongot/keyfile
            KEYFILECONTENTS: sup3rs3cr3tk3y
        ports:
            - 27027:27027
        networks:
            network:
                ipv4_address: 10.6.0.6
networks:
  network:
    driver: bridge
    ipam:
      config:
        - subnet: 10.6.0.0/16
          gateway: 10.6.0.1
```

Before we can use our environment in tests, we still need to create our search index. On top of that, we need to initialize the replica set which is needed as the two containers form a cluster. There are multiple ways to achieve this:

- One way is to use the Testcontainers framework to start the Docker Compose file and a test framework like [jest](https://jestjs.io/) which allows you to define setup and teardown methods for your tests. In the setup method, you can initialize the replica set and create the search index. An advantage of this approach is that you don't need to start your Docker Compose manually before you run your tests.
- Another way is to extend the Docker Compose file by a third container which simply runs a script to accomplish the initialization of the replica set and the creation of the search index.

As the first solution offers a better developer experience by allowing tests to be run using just one command, without the need to start the Docker environment manually, we will focus on that one. Additionally, this enables us to easily run our tests in our CI/CD pipeline.

The following code snippet shows an implementation of a jest setup function. At first, it starts the Docker Compose environment we defined before. After the containers have been started, the script builds a connection string to be able to connect to the cluster using a MongoClient (mind the `directConnection=true` parameter!). The MongoClient connects to the cluster and issues an admin command to initialize the replica set. Since this command takes some milliseconds to complete, the script waits for some time before creating the search index. After that, we load an Atlas Search index definition from the file system and use `createSearchIndex` to create the index on the cluster. The content of the index definition file can be created by simply exporting the definition from the Atlas web UI. The only
information not included in this export is the index name. Therefore, we need to set it explicitly (important: the name needs to match the index name in your production code!). After that, we close the database connection used by MongoClient and save a reference to the Docker environment to tear it down after the tests have run.

```javascript
export default async () => {
    const environment = await new DockerComposeEnvironment(".", "docker-compose.yml").up()
    const port = environment.getContainer("mongod").getFirstMappedPort()
    const host = environment.getContainer("mongod").getHost()
    process.env.MONGO_URL = `mongodb://${host}:${port}/atlas-local-test?directConnection=true`
    const mongoClient = new MongoClient(process.env.MONGO_URL)
    try {
        await mongoClient
            .db()
            .admin()
            .command({
                replSetInitiate: {
                    _id: "local",
                    members: [{_id: 0, host: "10.6.0.5:27017"}]
                }
            })
        await new Promise((r) => setTimeout(r, 500))
        const indexDefinition = path.join(__dirname, "../index.json")
        const definition = JSON.parse(fs.readFileSync(indexDefinition).toString("utf-8"))
        const collection = await mongoClient.db("atlas-local-test").createCollection("items")
        await collection.createSearchIndex({name: "items-index", definition})
    } finally {
        await mongoClient.close()
    }
    global.__MONGO_ENV__ = environment
}
```

## Writing and running tests

When you write integration tests for your queries, you need to insert data into your database before running the tests. Usually, you would insert the needed data at the beginning of your test, run your queries, check the results, and have some clean-up logic that runs after each test. Because the Atlas Search index is located on another container (`mongot`) than the actual data (`mongod`), it takes some time until the Atlas Search node has processed the events from the so-called change stream and $search queries return the expected data. This fact has an impact on the duration of the tests, as the following three scenarios show:

- We insert our test data in each test as before. As inserting or updating documents does not immediately lead to the search index being updated (the `mongot` has to listen to events of the change stream and process them), we would need to wait some time after writing data before we can be sure that the query returns the expected data. That is, we would need to include some kind of sleep() call in every test.
- We create test data for each test suite. Inserting test data once per test suite using a beforeAll() method brings down the time we have to wait for the `mongot` container to process the updates. The disadvantage of this approach is that we have to prepare the test data in such a way that it is suitable for all tests of this test suite.
- We create global test data for all test suites. Using the global setup method from the last section, we could also insert data into the database before creating the index. When the initial index creation has been completed, we will be ready to run our tests without waiting for some events from the change stream to be processed. But also in this scenario, your test data management gets more complex as you have to create test data that fits all your test scenarios.

In our project, we went with the second scenario. We think that it provides a good compromise between runtime requirements and the complexity of test data management. Plus, we think of these tests as integration tests where we do not need to test every corner case. We just need to make sure that the query can be executed and returns the expected data.

The exemplary test suite shown below follows the first approach. In beforeAll, some documents are inserted into the database. After that, the method is forced to “sleep” some time before the actual tests are run.

```javascript
beforeAll(async () => {
    await mongoose.connect(process.env.MONGO_URL!)
    const itemModel1 = new MongoItem({
        name: "Cool Thing",
        price: 1337,
    })
    await MongoItemModel.create(itemModel1)
    const itemModel2 = new MongoItem({
        name: "Nice Thing",
        price: 10000,
    })
    await MongoItemModel.create(itemModel2)
    await new Promise((r) => setTimeout(r, 1000))
})

describe("MongoItemRepository", () => {
    describe("getItemsInPriceRange", () => {
        it("get all items in given price range", async () => {
            const items = await repository.getItemsInPriceRange(1000, 2000)
            expect(items).toHaveLength(1)
        })
    })
})

afterAll(async () => {
    await mongoose.connection.collection("items").deleteMany({})
    await mongoose.connection.close()
})
```

## Conclusion

Before having a more in-depth look at it, we put Atlas Search aside for all the wrong reasons: We had no need for full-text searches and thought it was not really possible to run tests on it. After using it for a while, we can genuinely say that Atlas Search is not only a great tool for applications that use full-text search-based features. It can also be used to realize more traditional query patterns and reduce the load on the database. As for the testing part, there have been some great improvements since the feature was initially rolled out and by now, we have reached a state where testability is not an unsolvable issue anymore, even though it still requires some setup.

With the container images provided by MongoDB and some of the Docker magic introduced in this article, it is now possible to run integration tests for these queries locally and also in your CI/CD pipeline. Give it a try if you haven't yet and let us know how it works for you.

You can find the complete source code for the example described in this post in the
[GitHub repository](https://github.com/mongodb-developer/atlas-search-local-testing). There's still some room for improvement that can be incorporated into the test setup. Future updates of the tools might enable us to write tests without the need to wait some time before we can continue running our tests so that one day, we can all write some MongoDB Atlas Search integration tests without any hassle.

Questions? Comments? Head to the [MongoDB Developer Community](https://www.mongodb.com/community/forums/) to continue the conversation!

In this article, learn how to run unit tests on a local deployment of MongoDB Atlas Search.

How to Enable Local and Automatic Testing of Atlas Search-Based Features

Tobias Quante

## About Ghost

[Ghost](https://ghost.org/) is an open-source blogging platform. Unlike other content management systems like WordPress, its focus lies on professional publishing.

This ensures the core of the system remains lean. To integrate third-party applications, you don't even need to install plugins. Instead, Ghost offers a feature called [Webhooks](https://ghost.org/docs/webhooks/) which runs while you work on your publication.

These webhooks send particular items, such as a [post](https://ghost.org/docs/publishing/) or a [page](https://ghost.org/docs/themes/contexts/page/), to an HTTP endpoint defined by you and thereby provide an excellent base for our real-time service.

## Server-sent events

You are likely familiar with the concept of [an HTTP session](https://developer.mozilla.org/en-US/docs/Web/HTTP/Session). A client sends a request, the server responds and then closes the connection. When using server-sent events (SSEs), said connection remains open. This allows the server to continue writing messages into the response.

![A default HTTP lifecycle consists of a client request and a server response](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt189f6f60c53c4516/64daa9d91abd0f66b695a444/image2.png)

Like [Websockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API) (WS), apps and websites use SSEs for real-time communication. Where WSs use a dedicated protocol and work in both directions, SSEs are unidirectional. They use plain HTTP endpoints to write a message whenever an event occurs on the server side.

![Server Sent events are written into an already existing connection response](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt526d1df07c94e9f4/64daaa10ab46d299692f2409/image3.png)

Clients can subscribe to these endpoints using the [EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) browser API:

```javascript
const subscription = new EventSource("https://example.io/subscribe")
```

## MongoDB Change Streams

Now that we’ve looked at the periphery of our application, it's time to present its core. We'll use MongoDB to store a subset of the received Ghost Webhook data. On top of that, we'll use [MongoDB Change Streams](https://www.mongodb.com/basics/change-streams) to watch our webhook collection.

In a nutshell, Change Streams register data flowing into our database. We can subscribe to this data stream and react to it. Reacting means sending out SSE messages to connected clients whenever a new webhook is received and stored.

The following Javascript code showcases a simple Change Stream subscription.

```javascript
import {MongoClient} from 'mongodb';

const client = new MongoClient("<mongodb-url>");
const ghostDb = client.db('ghost');
const ghostCollection = ghostDb.collection('webhooks');
const ghostChangeStrem = ghostCollection.watch();

ghostChangeStream.on('change', document => {
    /* document is the MongoDB collection entry, e.g. our webhook */
});
```

Its event-based nature matches perfectly with webhooks and SSEs. We can react to newly received webhooks where the data is created, ensuring data integrity over our whole application.

## Build a real-time endpoint

We need an extra application layer to propagate these changes to connected clients. I've decided to use Typescript and Express.js, but you can use any other server-side framework. You will also need a dedicated MongoDB instance*. For a quick start, you can sign up for [MongoDB Atlas](https://www.mongodb.com/atlas). Then, [create a free cluster](https://www.mongodb.com/docs/atlas/tutorial/create-atlas-account/) and [connect to it](https://www.mongodb.com/docs/atlas/tutorial/connect-to-your-cluster/).

Let's get started by cloning the `1-get-started` branch from this Github repository:

```bash
# ssh
$ git clone git@github.com:tq-bit/mongodb-article-mongo-changestreams.git

# HTTP(s)
$ git clone https://github.com/tq-bit/mongodb-article-mongo-changestreams.git

# Change to the starting branch
$ git checkout 1-get-started

# Install NPM dependencies
$ npm install

# Make a copy of .env.example
$ cp .env.example .env
```

> Make sure to fill out the MONGO_HOST environment variable with your connection string!

Express and the database client are already implemented. So in the following, we'll focus on adding MongoDB change streams and server-sent events.

Once everything is set up, you can start the server on `http://localhost:3000` by typing

```bash
npm run dev
```

The application uses two important endpoints which we will extend in the next sections: 

- `/api/notification/subscribe` <- Used by EventSource to receive event messages
- `/api/notification/article/create` <- Used as a webhook target by Ghost


\* If you are not using MongoDB Atlas, make sure to have [Replication Sets](https://www.mongodb.com/docs/manual/tutorial/deploy-replica-set/) enabled.

## Add server-sent events

Open the cloned project in your favorite code editor. We'll add our SSE logic under `src/components/notification/notification.listener.ts`.

In a nutshell, implementing SSE requires three steps:

- Write out an HTTP status 200 header.
- Write out an opening message.
- Add event-based response message handlers.

We’ll start sending a static message and revisit this module after adding ChangeStreams.

> You can also `git checkout 2-add-sse` to see the final result.

### Write the HTTP header

Writing the HTTP header informs clients of a successful connection. It also propagates the response's content type and makes sure events are not cached.

Add the following code to the function `subscribeToArticleNotification` inside:

```javascript
// Replace
// TODO: Add function to write the head
// with
console.log('Step 1: Write the response head and keep the connection open');
res.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    Connection: 'keep-alive'
});
```

### Write an opening message

The first message sent should have an event type of 'open'. It is not mandatory but helps to determine whether the subscription was successful.

Append the following code to the function `subscribeToArticleNotification`:

```javascript
// Replace
// TODO: Add functionality to write the opening message
// with
console.log('Step 2: Write the opening event message');
res.write('event: open\n');
res.write('data: Connection opened!\n'); // Data can be any string
res.write(`id: ${crypto.randomUUID()}\n\n`);
```

### Add response message handlers

We can customize the content and timing of all further messages sent. Let's add a placeholder function that sends messages out every five seconds for now. And while we’re at it, let’s also add a handler to close the client connection:

Append the following code to the function `subscribeToArticleNotification`:

```javascript
setInterval(() => {
    console.log('Step 3: Send a message every five seconds');
    res.write(`event: message\n`);
    res.write(`data: ${JSON.stringify({ message: 'Five seconds have passed' })}\n`);
    res.write(`id: ${crypto.randomUUID()}\n\n`);
}, 5000);


// Step 4: Handle request events such as client disconnect
// Clean up the Change Stream connection and close the connection stream to the client
req.on('close', () => {
    console.log('Step 4: Handle request events such as client disconnect');
    res.end();
});
```

To check if everything works, visit `http://localhost:3000/api/notification/subscribe`.

![server sent event data in a browser screen](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt2724881e861bb1c8/64daaa538cd995c129389110/image1.png)

## Add a POST endpoint for Ghost

Let's visit `src/components/notification/notification.model.ts` next. We'll add a simple `insert` command for our database into the function `createNotificiation`:

> You can also `git checkout 3-webhook-handler` to see the final result.

```javascript
// Replace
// TODO: Add insert one functionality for DB
// with
return notificationCollection.insertOne(notification);
```

And on to `src/components/notification/notification.controller.ts`. To process incoming webhooks, we'll add a handler function into `handleArticleCreationNotification`:

```javascript
// Replace
// TODO: ADD handleArticleCreationNotification
// with
const incomingWebhook: GhostWebhook = req.body;
await NotificationModel.createNotificiation({
    id: crypto.randomUUID(),
    ghostId: incomingWebhook.post?.current?.id,
    ghostOriginalUrl: incomingWebhook.post?.current?.url,
    ghostTitle: incomingWebhook.post?.current?.title,
    ghostVisibility: incomingWebhook.post?.current?.visibility,
    type: NotificationEventType.PostPublished,
});

res.status(200).send('OK');
```

This handler will pick data from the incoming webhook and insert a new notification.

```bash
curl -X POST -d '{
	"post": {
		"current": {
            "id": "sj7dj-lnhd1-kabah9-107gh-6hypo",
            "url": "http://localhost:2368/how-to-create-realtime-notifications",
            "title": "How to create realtime notifications",
            "visibility": "public"
		}
	}
}' http://localhost:3000/api/notification/article/create
```

You can also test the insert functionality by using Postman or [VSCode REST client](https://marketplace.visualstudio.com/items?itemName=humao.rest-client) and then check your MongoDB collection. There is an example request under `/test/notification.rest` in the project's directory, for your convenience.

## Trigger MongoDB Change Streams

So far, we can send SSEs and insert Ghost notifications. Let's put these two features together now.

Earlier, we added a static server message sent every five seconds. Let's revisit `src/components/notification/notification.listener.ts` and make it more dynamic.

First, let's get rid of the whole `setInterval` and its callback. Instead, we'll use our `notificationCollection` and its built-in method `watch`. This method returns a `ChangeStream`.

You can create a change stream by adding the following code above the `export default` code segment:

```javascript
const notificationStream = notificationCollection.watch();
```

The stream fires an event whenever its related collection changes. This includes the `insert` event from the previous section.

We can register callback functions for each of these. The event that fires when a document inside the collection changes is 'change':

```javascript
notificationStream.on('change', (next) => {
    console.log('Step 3.1: Change in Database detected!');
});
```

The variable passed into the callback function is a change stream document. It includes two important information for us:

- The document that's inserted, updated, or deleted.
- The type of operation on the collection.

Let's assign them to one variable each inside the callback:

```javascript
notificationStream.on('change', (next) => {
    // ... previous code
    const {
        // @ts-ignore, fullDocument is not part of the next type (yet)
        fullDocument /* The newly inserted fullDocument */,
        operationType /* The MongoDB operation Type, e.g. insert */,
    } = next;
});
```

Let's write the notification to the client. We can do this by repeating the method we used for the opening message.

```javascript
notificationStream.on('change', (next) => {
    // ... previous code
    console.log('Step 3.2: Writing out response to connected clients');
    res.write(`event: ${operationType}\n`);
    res.write(`data: ${JSON.stringify(fullDocument)}\n`);
    res.write(`id: ${crypto.randomUUID()}\n\n`);
});
```

And that's it! You can test if everything is functional by:

1. Opening your browser under `http://localhost:3000/api/notification/subscribe`.
2. Using the file under `test/notification.rest` with VSCode's HTTP client.
3. Checking if your browser includes an opening and a Ghost Notification.

![server sent event data in a browser screen after fully implementing change streams](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt28fad804140cd0cc/64daaa965572446fc6e22d3c/image6.png)

For an [HTTP webhook implementation](https://ghost.org/docs/webhooks/), you will need a running Ghost instance. I have added a [dockerfile](https://github.com/tq-bit/mongodb-article-mongo-changestreams/blob/master/docker-compose.yml) to this repo for your convenience. [You could also install Ghost yourself locally](https://ghost.org/docs/install/local/).

To start Ghost with the dockerfile, make sure you have [Docker Engine](https://docs.docker.com/engine/install/) or [Docker Desktop](https://docs.docker.com/desktop/) with support for `docker compose` installed.

For a local installation and the first-time setup, you should follow the [official Ghost installation guide](https://ghost.org/docs/install/).

After your Ghost instance is up and running, open your browser at `http://localhost:2368/ghost`. You can set up your site however you like, give it a name, enter details, and so on.

![Configure the Ghost deployment](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltfe21f1c4ad80772e/64daac4c2b55eb944840c8e4/gif1.gif)

In order to create a webhook, you must first create a custom integration. To do so, navigate into your site’s settings and click on the “Integrations” menu point. Click on “Add Webhook,” enter a name, and click on “Create.”

![Ghost third-party custom integrations](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt12432f5aed316b08/64daacb28ef021688773fbf1/gif2.gif)

Inside the newly created integration, you can now configure a webhook to point at your application under `http://<host>:<port>/api/notification/article/create`*.

\* This URL might vary based on your local Ghost setup. For example, if you run Ghost in a container, you can find your machine's local IP using the terminal and `ifconfig` on Linux or `ipconfig` on Windows.

![Local ghost-realtime application](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltca1701eede410cc3/64daad17ab46d294592f241b/gif3.gif)

And that’s it. Now, whenever a post is published, its contents will be sent to our real-time endpoint. After being inserted into MongoDB, an event message will be sent to all connected clients.

## Subscribe to Change Streams from your Ghost theme

There are a few ways to add real-time notifications to your Ghost theme. Going into detail is beyond the scope of this article. I have prepared two files, a `plugin.js` and a `plugin.css` file [you can inject into the default Casper theme](https://ghost.org/tutorials/use-code-injection-in-ghost/).

Try this out by starting a local Ghost instance using the provided dockerfile.

You must then instruct your application to serve the JS and CSS assets. Add the following to your `index.ts` file:

```javascript
// ... other app.use hooks
app.use(express.static('public'));
// ... app.listen()
```

Finally, navigate to [Code Injection](http://localhost:2368/ghost/#/settings/code-injection) and add the following two entries in the 'Site Header':

```html
<script src="http://localhost:3000/js/plugin.js"></script>
<link href="http://localhost:3000/css/plugin.css" rel="stylesheet">
```

> The core piece of the plugin is the [EventSource browser API](https://developer.mozilla.org/en-US/docs/Web/API/EventSource). You will want to use it when integrating this application with other themes.

When going back into your Ghost publication, you should now see a small bell icon on the upper right side.

![subscription notifications in Ghost](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt3fbfeb0c37c5ed78/64daab2bd7ccb0bad5e43892/image4.png)

## Moving ahead

If you’ve followed along, congratulations! You now have a working real-time notification service for your Ghost blog. And if you haven’t, what are you waiting for? Sign up for a free account on MongoDB Atlas and start building. You can use the [final branch of this repository](https://github.com/tq-bit/mongodb-article-mongo-changestreams/tree/5-add-frontend-example) to get started and explore the full power of MongoDB’s toolkit.

Learn how to work with MongoDB change streams and develop a server-sent event application that integrates with Ghost CMS.

Adding Real-Time Notifications to Ghost CMS Using MongoDB and Server-Sent Events

Michael Lynn

Principal Developer Advocate

Join Michael Lynn and Nic Raboy as they chat with Gaspard Petit of Square Enix to learn how one of the largest and best-loved gaming brands in the world is using MongoDB to scale and grow.

Scaling the Gaming Industry with Gaspard Petit of Square Enix

Nic Raboy

Staff Developer Advocate

Anaiya Raisinghani

Associate Developer Advocate

In the developer community, ensuring your projects run accurately regardless of the environment can be a pain. Whether it’s trying to recreate a demo from an online tutorial or working on a code review, hearing the words, “Well, it works on my machine…” can be frustrating. Instead of spending hours debugging, we want to introduce you to a platform that will change your developer experience: [Docker](https://www.docker.com/). 

Docker is a great tool to learn because it provides developers with the ability for their applications to be used easily between environments, and it's resource-efficient in comparison to virtual machines. This tutorial will gently guide you through how to navigate Docker, along with how to integrate [Go](https://golang.org/) on the platform. We will be using this project to connect to our previously built MongoDB Atlas Search Cluster made for using [Synonyms in Atlas Search](https://www.mongodb.com/developer/how-to/get-hyped-synonyms-atlas-search/). Stay tuned for a fun read on how to learn all the above while also expanding your Gen-Z slang knowledge from our synonyms cluster. Get hyped! 

## The Prerequisites

There are a few requirements that must be met to be successful with this tutorial.

- A M0 or better MongoDB Atlas cluster
- Docker Desktop

To use MongoDB with the Golang driver, you only need a free M0 cluster. To create this cluster, follow the instructions listed on the [MongoDB documentation](https://docs.atlas.mongodb.com/tutorial/create-new-cluster/). However, we’ll be making many references to a [previous tutorial](https://www.mongodb.com/developer/how-to/get-hyped-synonyms-atlas-search/) where we used Atlas Search with custom synonyms.

Since this is a Docker tutorial, you’ll need [Docker Desktop](https://www.docker.com/products/docker-desktop). You don’t actually need to have Golang configured on your host machine because Docker can take care of this for us as we progress through the tutorial.

## Building a Go API with the MongoDB Golang Driver 

Like previously mentioned, you don’t need Go installed and configured on your host computer to be successful. However, it wouldn’t hurt to have it in case you wanted to test things prior to creating a Docker image.

On your computer, create a new project directory, and within that project directory, create a **src** directory with the following files:

- go.mod
- main.go

The **go.mod** file is our dependency management file for Go modules. It could easily be created manually or by using the following command:

```bash
go mod init
```

The **main.go** file is where we’ll keep all of our project code.

Starting with the **go.mod** file, add the following lines:

```
module github.com/mongodb-developer/docker-golang-example
go 1.15
require go.mongodb.org/mongo-driver v1.7.0
require github.com/gorilla/mux v1.8.0
```

Essentially, we’re defining what version of Go to use and the modules that we want to use. For this project, we’ll be using the MongoDB Go driver as well as the [Gorilla Web Toolkit](https://www.gorillatoolkit.org/).

This brings us into the building of our simple API.

Within the **main.go** file, add the following code:

```golang
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"net/http"
	"os"
	"time"

	"github.com/gorilla/mux"
	"go.mongodb.org/mongo-driver/bson"
	"go.mongodb.org/mongo-driver/mongo"
	"go.mongodb.org/mongo-driver/mongo/options"
)

var client *mongo.Client
var collection *mongo.Collection

type Tweet struct {
	ID       int64  `json:"_id,omitempty" bson:"_id,omitempty"`
	FullText string `json:"full_text,omitempty" bson:"full_text,omitempty"`
	User     struct {
		ScreenName string `json:"screen_name" bson:"screen_name"`
	} `json:"user,omitempty" bson:"user,omitempty"`
}

func GetTweetsEndpoint(response http.ResponseWriter, request *http.Request) {}
func SearchTweetsEndpoint(response http.ResponseWriter, request *http.Request) {}

func main() {
	fmt.Println("Starting the application...")
	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()
	client, err := mongo.Connect(ctx, options.Client().ApplyURI(os.Getenv("MONGODB_URI")))
	defer func() {
		if err = client.Disconnect(ctx); err != nil {
			panic(err)
		}
	}()
collection = client.Database("synonyms").Collection("tweets")
	router := mux.NewRouter()
	router.HandleFunc("/tweets", GetTweetsEndpoint).Methods("GET")
	router.HandleFunc("/search", SearchTweetsEndpoint).Methods("GET")
	http.ListenAndServe(":12345", router)
}
```

There’s more to the code, but before we see the rest, let’s start breaking down what we have above to make sense of it.

You’ll probably notice our `Tweets` data structure:

```golang
type Tweet struct {
	ID       int64  `json:"_id,omitempty" bson:"_id,omitempty"`
	FullText string `json:"full_text,omitempty" bson:"full_text,omitempty"`
	User     struct {
		ScreenName string `json:"screen_name" bson:"screen_name"`
	} `json:"user,omitempty" bson:"user,omitempty"`
}
```

Earlier in the tutorial, we mentioned that this example is heavily influenced by a [previous tutorial](https://www.mongodb.com/developer/how-to/get-hyped-synonyms-atlas-search/) that used Twitter data. We highly recommend you take a look at it. This data structure has some of the fields that represent a tweet that we scraped from Twitter. We didn’t map all the fields because it just wasn’t necessary for this example.

Next, you’ll notice the following:

```golang
func GetTweetsEndpoint(response http.ResponseWriter, request *http.Request) {}
func SearchTweetsEndpoint(response http.ResponseWriter, request *http.Request) {}
```

These will be the functions that hold our API endpoint logic. We’re going to skip these for now and focus on understanding the connection and configuration logic.

As of now, most of what we’re interested in is happening in the `main` function.

The first thing we’re doing is connecting to MongoDB:

```golang
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(ctx, options.Client().ApplyURI(os.Getenv("MONGODB_URI")))
defer func() {
	if err = client.Disconnect(ctx); err != nil {
		panic(err)
	}
}()
collection = client.Database("synonyms").Collection("tweets")
```

You’ll probably notice the `MONGODB_URI` environment variable in the above code. It’s not a good idea to hard-code the MongoDB connection string in the application. This prevents us from being flexible and it could be a security risk. Instead, we’re using environment variables that we’ll pass in with Docker when we deploy our containers.

You can visit the MongoDB Atlas dashboard for your URI string.

The database we plan to use is `synonyms` and we plan to use the `tweets` collection, both of which we talked about in that previous tutorial.

After connecting to MongoDB, we focus on configuring the Gorilla Web Toolkit:

```golang
router := mux.NewRouter()
router.HandleFunc("/tweets", GetTweetsEndpoint).Methods("GET")
router.HandleFunc("/search", SearchTweetsEndpoint).Methods("GET")
http.ListenAndServe(":12345", router)
```

In this code, we are defining which endpoint path should route to which function. The functions are defined, but we haven’t yet added any logic to them. The application itself will be serving on port 12345.

As of now, the application has the necessary basic connection and configuration information. Let’s circle back to each of those endpoint functions.

We’ll start with the `GetTweetsEndpoint` because it will work fine with an M0 cluster:

```golang
func GetTweetsEndpoint(response http.ResponseWriter, request *http.Request) {
	response.Header().Set("content-type", "application/json")
	var tweets []Tweet
	ctx, _ := context.WithTimeout(context.Background(), 30*time.Second)
	cursor, err := collection.Find(ctx, bson.M{})
	if err != nil {
		response.WriteHeader(http.StatusInternalServerError)
		response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
		return
	}
	if err = cursor.All(ctx, &tweets); err != nil {
		response.WriteHeader(http.StatusInternalServerError)
		response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
		return
	}
	json.NewEncoder(response).Encode(tweets)
}
```

In the above code, we’re saying that we want to use the `Find` operation on our collection for all documents in that collection, hence the empty filter object.

If there were no errors, we can get all the results from our cursor, load them into a `Tweet` slice, and then JSON encode that slice for sending to the client. The client will receive JSON data as a result.

Now we can look at the more interesting endpoint function.

```golang
func SearchTweetsEndpoint(response http.ResponseWriter, request *http.Request) {
	response.Header().Set("content-type", "application/json")
	queryParams := request.URL.Query()
	var tweets []Tweet
	ctx, _ := context.WithTimeout(context.Background(), 30*time.Second)
	searchStage := bson.D{
		{"$search", bson.D{
			{"index", "synsearch"},
			{"text", bson.D{
				{"query", queryParams.Get("q")},
				{"path", "full_text"},
				{"synonyms", "slang"},
			}},
		}},
	}
	cursor, err := collection.Aggregate(ctx, mongo.Pipeline{searchStage})
	if err != nil {
		response.WriteHeader(http.StatusInternalServerError)
		response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
		return
	}
	if err = cursor.All(ctx, &tweets); err != nil {
		response.WriteHeader(http.StatusInternalServerError)
		response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
		return
	}
	json.NewEncoder(response).Encode(tweets)
}
```

The idea behind the above function is that we want to use an aggregation pipeline for Atlas Search. It does use the synonym information that we outlined in the [previous tutorial](https://www.mongodb.com/developer/how-to/get-hyped-synonyms-atlas-search/).

The first important thing in the above code to note is the following:

```golang
​​queryParams := request.URL.Query()
```

We’re obtaining the query parameters passed with the HTTP request. We’re expecting a `q` parameter to exist with the search query to be used.

To keep things simple, we make use of a single stage for the MongoDB aggregation pipeline:

```golang
searchStage := bson.D{
	{"$search", bson.D{
		{"index", "synsearch"},
		{"text", bson.D{
			{"query", queryParams.Get("q")},
			{"path", "full_text"},
			{"synonyms", "slang"},
		}},
	}},
}
```

In this stage, we are doing a text search with a specific index and a specific set of synonyms. The query that we use for our text search comes from the query parameter of our HTTP request.

Assuming that everything went well, we can load all the results from the cursor into a `Tweet` slice, JSON encode it, and return it to the client that requested it.

If you have Go installed and configured on your computer, go ahead and try to run this application. Just don’t forget to add the `MONGODB_URI` to your environment variables prior.

If you want to learn more about API development with the Gorilla Web Toolkit and MongoDB, check out [this tutorial](https://www.thepolyglotdeveloper.com/2019/02/developing-restful-api-golang-mongodb-nosql-database/) on the subject.

## Configuring a Docker Image for Go with MongoDB

Let’s get started with Docker! If it’s a platform you’ve never used before, it might seem a bit daunting at first, but let us guide you through it, step by step. We will be showing you how to download Docker and get started with setting up your first Dockerfile to connect to our Gen-Z Synonyms Atlas Cluster. 

First things first. Let’s download Docker. This can be done through their [website](https://www.docker.com/products/docker-desktop) in just a couple of minutes. 

Once you have that up and running, it’s time to create your very first Dockerfile. 

At the root of your project folder, create a new **Dockerfile** file with the following content:

```
#get a base image
FROM golang:1.16-buster

MAINTAINER anaiya raisinghani <anaiya.raisinghani@mongodb.com>

WORKDIR /go/src/app
COPY ./src .

RUN go get -d -v
RUN go build -v

CMD ["./docker-golang-example"]
```

This format is what many Dockerfiles are composed of, and a lot of it is heavily customizable and can be edited to fit your project's needs. 

The first step is to grab a base image that you’re going to use to build your new image. You can think of using Dockerfiles as layers to a cake. There are a multitude of different base images out there, or you can use `FROM scratch` to start from an entirely blank image. Since this project is using the programming language Go, we chose to start from the `golang` base image and add the tag `1.16` to represent the version of Go that we plan to use. Whenever you include a tag next to your base image, be sure to set it up with a colon in between, just like this: `golang:1.16`. To learn more about which tag will benefit your project the best, check out [Docker’s documentation](https://hub.docker.com/_/golang?tab=tags&page=1&ordering=last_updated) on the subject.

This site holds a lot of different tags that can be used on a Golang base image. Tags are important because they hold very valuable information about the base image you’re using such as software versions, operating system flavor, etc. 

Let’s run through the rest of what will happen in this Dockerfile!

It's optional to include a `MAINTAINER` for your image, but it’s good practice so that people viewing your Dockerfile can know who created it. It's not necessary, but it’s helpful to include your full name and your email address in the file. 

The `WORKDIR /go/src/app` command is crucial to include in your Dockerfile since `WORKDIR` specifies which working directory you’re in. All the commands after will be run through whichever directory you choose, so be sure to be aware of which directory you’re currently in.

The `COPY ./src .` command allows you to copy whichever files you want from the specified location on the host machine into the Docker image. 

Now, we can use the `RUN` command to set up exactly what we want to happen at image build time before deploying as a container. The first command we have is `RUN go get -d -v`, which will download all of the Go dependencies listed in the **go.mod** file that was copied into the image.. 

Our second `RUN` command is `RUN go build -v`, which will build our project into an executable binary file. 

The last step of this Dockerfile is to use a `CMD` command, `CMD [“./docker-golang-example”]`. This command will define what is run when the container is deployed rather than when the image is built. Essentially we’re saying that we want the built Go application to be run when the container is deployed.

Once you have this Dockerfile set up, you can build and execute your project using your entire MongoDB URI link:

To build the Docker image and deploy the container, execute the following from the command line:

```bash
docker build -t docker-syn-image .
docker run -d -p 12345:12345 -e “MONGODB_URI=YOUR_URI_HERE” docker-syn-image
```

Following these instructions will allow you to run the project and access it from http://localhost:12345. **But**! It’s so tedious. What if we told you there was an easier way to run your application without having to write in the entire URI link? There is! All it takes is one extra step: setting up a Docker Compose file. 

## Setting Up a Docker Compose File to Streamline Deployments

A Docker Compose file is a nice little step to run all your container files and dependencies through a simple command: `docker compose up`.

In order to set up this file, you need to establish a YAML configuration file first. Do this by creating a new file in the root of your project folder, naming it **docker-compose**, and adding **.yml** at the end. You can name it something else if you like, but this is the easiest since when running the `docker compose up` command, you won’t need to specify a file name. Once that is in your project folder, follow the steps below.

This is what your Docker Compose file will look like once you have it all set up: 

```yaml
version: "3.9" 
services:
   web:
       build: .
       ports:
           - "12345:12345"
       environment:
           MONGODB_URI: your_URI_here
```

Let’s run through it!

First things first. Determine which schema version you want to be running. You should be using the most recent version, and you can find this out through [Docker’s documentation](https://docs.docker.com/compose/compose-file/).

Next, define which services, otherwise known as containers, you want to be running in your project. We have included `web` since we are attaching to our Atlas Search cluster. The name isn’t important and it acts more as an identifier for that particular service. Next, specify that you are building your application, and put in your `ports` information in the correct spot. For the next step, we can set up our `environment` as our MongoDB URI and we’re done! 

Now, run the command `docker compose up` and watch the magic happen. Your container should build, then run, and you’ll be able to connect to your port and see all the tweets!

## Conclusion

This tutorial has now left you equipped with the knowledge you need to build a Go API with the MongoDB Golang driver, create a Dockerfile, create a Docker Compose file, and connect your newly built container to a MongoDB Atlas Cluster. 

Using these new platforms will allow you to take your projects to a whole new level. 

If you’d like to take a look at the code used in our project, you can access it on [GitHub](https://github.com/mongodb-developer/docker-golang-example).

Using Docker or Go, but have a question? Check out the [MongoDB Community Forums](https://community.mongodb.com)!

Learn how to create and deploy Golang-powered micro-services that interact with MongoDB using Docker.

Get Hyped: Using Docker + Go with MongoDB

Need a consistent development and deployment experience as developers work across teams and use different machines for their daily tasks? That is where Docker has you covered with containers. A common experience might include running a local version of MongoDB Community in a container and an application in another container. This strategy works for some organizations, but what if you want to leverage all the benefits that come with MongoDB Atlas in addition to a container strategy for your application development?

In this tutorial we'll see how to create a MongoDB-compatible web application, bundle it into a container with Docker, and manage creation as well as destruction for MongoDB Atlas with the [Atlas CLI](https://www.mongodb.com/docs/atlas/cli/stable/) during container deployment.

It should be noted that this tutorial was intended for a development or staging setting through your local computer. It is not advised to use all the techniques found in this tutorial in a production setting. Use your best judgment when it comes to the code included.

If you’d like to try the results of this tutorial, check out the repository and instructions on [GitHub](https://github.com/mongodb-developer/atlas-cli-docker-example).

## The prerequisites

There are a lot of moving parts in this tutorial, so you'll need a few things prior to be successful:

- A [MongoDB Atlas](https://www.mongodb.com/atlas/database) account
- [Docker](https://www.docker.com/)
- Some familiarity with Node.js and JavaScript

The [Atlas CLI](https://www.mongodb.com/docs/atlas/cli/stable/) can create an Atlas account for you along with any keys and ids, but for the scope of this tutorial you'll need one created along with quick access to the "Public API Key", "Private API Key", "Organization ID", and "Project ID" within your account. You can see how to do this in the [documentation](https://www.mongodb.com/docs/atlas/configure-api-access/#grant-programmatic-access-to-service).

Docker is going to be the true star of this tutorial. You don't need anything beyond [Docker](https://www.docker.com/) because the Node.js application and the Atlas CLI will be managed by the Docker container, not your host computer.

On your host computer, create a project directory. The name isn't important, but for this tutorial we'll use **mongodbexample** as the project directory.

## Create a simple Node.js application with Express Framework and MongoDB

We're going to start by creating a Node.js application that communicates with MongoDB using the [Node.js driver for MongoDB](https://www.mongodb.com/docs/drivers/node/current/). The application will be simple in terms of functionality. It will connect to MongoDB, create a database and collection, insert a document, and expose an API endpoint to show the document with an HTTP request.

Within the project directory, create a new **app** directory for the Node.js application to live. Within the **app** directory, using a command line, execute the following:

```bash
npm init -y
npm install express mongodb
```

If you don't have Node.js installed, just create a **package.json** file within the **app** directory with the following contents:

```json
{
    "name": "mongodbexample",
    "version": "1.0.0",
    "description": "",
    "main": "main.js",
    "scripts": {
        "test": "echo \"Error: no test specified\" && exit 1",
        "start": "node main.js"
    },
    "keywords": [],
    "author": "",
    "license": "ISC",
    "dependencies": {
        "express": "^4.18.2",
        "mongodb": "^4.12.1"
    }
}
```

Next, we'll need to define our application logic. Within the **app** directory we need to create a **main.js** file. Within the **main.js** file, add the following JavaScript code:

```javascript
const { MongoClient } = require("mongodb");
const Express = require("express");

const app = Express();

const mongoClient = new MongoClient(process.env.MONGODB_ATLAS_URI);
let database, collection;

app.get("/data", async (request, response) => {
    try {
        const results = await collection.find({}).limit(5).toArray();
        response.send(results);
    } catch (error) {
        response.status(500).send({ "message": error.message });
    }
});

const server = app.listen(3000, async () => {
    try {
        await mongoClient.connect();
        database = mongoClient.db(process.env.MONGODB_DATABASE);
        collection = database.collection(`${process.env.MONGODB_COLLECTION}`);
        collection.insertOne({ "firstname": "Nic", "lastname": "Raboy" });
        console.log("Listening at :3000");
    } catch (error) {
        console.error(error);
    }
});

process.on("SIGTERM", async () => {
    if(process.env.CLEANUP_ONDESTROY == "true") {
        await database.dropDatabase();
    }
    mongoClient.close();
    server.close(() => {
        console.log("NODE APPLICATION TERMINATED!");
    });
});
```

There's a lot happening in the few lines of code above. We're going to break it down!

Before we break down the pieces, take note of the environment variables used throughout the JavaScript code. We'll be passing these values through Docker in the end so we have a more dynamic experience with our local development.

The first important snippet of code to focus on is the start of our application service:

```javascript
const server = app.listen(3000, async () => {
    try {
        await mongoClient.connect();
        database = mongoClient.db(process.env.MONGODB_DATABASE);
        collection = database.collection(`${process.env.MONGODB_COLLECTION}`);
        collection.insertOne({ "firstname": "Nic", "lastname": "Raboy" });
        console.log("Listening at :3000");
    } catch (error) {
        console.error(error);
    }
});
```

Using the client that was configured near the top of the file, we can connect to MongoDB. Once connected, we can get a reference to a database and collection. This database and collection doesn't need to exist before that because it will be created automatically when data is inserted. With the reference to a collection, we insert a document and begin listening for API requests through HTTP.

This brings us to our one and only endpoint:

```javascript
app.get("/data", async (request, response) => {
    try {
        const results = await collection.find({}).limit(5).toArray();
        response.send(results);
    } catch (error) {
        response.status(500).send({ "message": error.message });
    }
});
```

When the `/data` endpoint is consumed, the first five documents in our collection are returned to the user. Otherwise if there was some issue, an error message would be returned.

This brings us to something optional, but potentially valuable when it comes to a Docker deployment for local development:

```javascript
process.on("SIGTERM", async () => {
    if(process.env.CLEANUP_ONDESTROY == "true") {
        await database.dropDatabase();
    }
    mongoClient.close();
    server.close(() => {
        console.log("NODE APPLICATION TERMINATED!");
    });
});
```

The above code says that when a termination event is sent to the application, drop the database we had created and close the connection to MongoDB as well as the Express Framework service. This could be useful if we want to undo everything we had created when the container stops. If you want your changes to persist, it might not be necessary. For example, if you want your data to exist between container deployments, persistence would be required. On the other hand, maybe you are using the container as part of a test pipeline and you want to clean up when you’re done, the termination commands could be valuable.

So we have an environment variable heavy Node.js application. What's next?

## Deploying a MongoDB Atlas cluster with network rules, user roles, and sample data

While we have the application, our MongoDB Atlas cluster may not be available to us. For example, maybe this is our first time being exposed to Atlas and nothing has been created yet. We need to be able to quickly and easily create a cluster, configure our IP access rules, specify users and permissions, and then connect with our Node.js application.

This is where the MongoDB Atlas CLI does the heavy lifting!

There are many different ways to create a script. Some like Bash, some like ZSH, some like something else. We're going to be using ZX which is a JavaScript wrapper for Bash.

Within your project directory, not your **app** directory, create a **docker_run_script.mjs** file with the following code:

```javascript
#!/usr/bin/env zx

$.verbose = true;

const runtimeTimestamp = Date.now();

process.env.MONGODB_CLUSTER_NAME = process.env.MONGODB_CLUSTER_NAME || "examples";
process.env.MONGODB_USERNAME = process.env.MONGODB_USERNAME || "demo";
process.env.MONGODB_PASSWORD = process.env.MONGODB_PASSWORD || "password1234";
process.env.MONGODB_DATABASE = process.env.MONGODB_DATABASE || "business_" + runtimeTimestamp;
process.env.MONGODB_COLLECTION = process.env.MONGODB_COLLECTION || "people_" + runtimeTimestamp;
process.env.CLEANUP_ONDESTROY = process.env.CLEANUP_ONDESTROY || false;

var app;

process.on("SIGTERM", () => { 
    app.kill("SIGTERM");
});

try {
    let createClusterResult = await $`atlas clusters create ${process.env.MONGODB_CLUSTER_NAME} --tier M0 --provider AWS --region US_EAST_1 --output json`;
    await $`atlas clusters watch ${process.env.MONGODB_CLUSTER_NAME}`
    let loadSampleDataResult = await $`atlas clusters loadSampleData ${process.env.MONGODB_CLUSTER_NAME} --output json`;
} catch (error) {
    console.log(error.stdout);
}

try {
    let createAccessListResult = await $`atlas accessLists create --currentIp --output json`;
    let createDatabaseUserResult = await $`atlas dbusers create --role readWriteAnyDatabase,dbAdminAnyDatabase --username ${process.env.MONGODB_USERNAME} --password ${process.env.MONGODB_PASSWORD} --output json`;
    await $`sleep 10`
} catch (error) {
    console.log(error.stdout);
}

try {
    let connectionString = await $`atlas clusters connectionStrings describe ${process.env.MONGODB_CLUSTER_NAME} --output json`;
    let parsedConnectionString = new URL(JSON.parse(connectionString.stdout).standardSrv);
    parsedConnectionString.username = encodeURIComponent(process.env.MONGODB_USERNAME);
    parsedConnectionString.password = encodeURIComponent(process.env.MONGODB_PASSWORD);
    parsedConnectionString.search = "retryWrites=true&w=majority";
    process.env.MONGODB_ATLAS_URI = parsedConnectionString.toString();
    app = $`node main.js`;
} catch (error) {
    console.log(error.stdout);
}
```

Once again, we're going to break down what's happening!

Like with the Node.js application, the ZX script will be using a lot of environment variables. In the end, these variables will be passed with Docker, but you can hard-code them at any time if you want to test things outside of Docker.

The first important thing to note is the defaulting of environment variables:

```javascript
process.env.MONGODB_CLUSTER_NAME = process.env.MONGODB_CLUSTER_NAME || "examples";
process.env.MONGODB_USERNAME = process.env.MONGODB_USERNAME || "demo";
process.env.MONGODB_PASSWORD = process.env.MONGODB_PASSWORD || "password1234";
process.env.MONGODB_DATABASE = process.env.MONGODB_DATABASE || "business_" + runtimeTimestamp;
process.env.MONGODB_COLLECTION = process.env.MONGODB_COLLECTION || "people_" + runtimeTimestamp;
process.env.CLEANUP_ONDESTROY = process.env.CLEANUP_ONDESTROY || false;
```

The above snippet isn't a requirement, but if you want to avoid setting or passing around variables, defaulting them could be helpful. In the above example, the use of `runtimeTimestamp` will allow us to create a unique database and collection should we want to. This could be useful if numerous developers plan to use the same Docker images to deploy containers because then each developer would be in a sandboxed area. If the developer chooses to undo the deployment, only their unique database and collection would be dropped.

Next we have the following:

```javascript
process.on("SIGTERM", () => { 
    app.kill("SIGTERM");
});
```

We have something similar in the Node.js application as well. We have it in the script because eventually the script controls the application. So when we (or Docker) stops the script, the same stop event is passed to the application. If we didn't do this, the application would not have a graceful shutdown and the drop logic wouldn't be applied.

Now we have three try / catch blocks, each focusing on something particular.

The first block is responsible for creating a cluster with sample data:

```javascript
try {
    let createClusterResult = await $`atlas clusters create ${process.env.MONGODB_CLUSTER_NAME} --tier M0 --provider AWS --region US_EAST_1 --output json`;
    await $`atlas clusters watch ${process.env.MONGODB_CLUSTER_NAME}`
    let loadSampleDataResult = await $`atlas clusters loadSampleData ${process.env.MONGODB_CLUSTER_NAME} --output json`;
} catch (error) {
    console.log(error.stdout);
}
```

If the cluster already exists, an error will be caught. We have three blocks because in our scenario, it is alright if certain parts already exist.

Next we worry about users and access:

```javascript
try {
    let createAccessListResult = await $`atlas accessLists create --currentIp --output json`;
    let createDatabaseUserResult = await $`atlas dbusers create --role readWriteAnyDatabase,dbAdminAnyDatabase --username ${process.env.MONGODB_USERNAME} --password ${process.env.MONGODB_PASSWORD} --output json`;
    await $`sleep 10`
} catch (error) {
    console.log(error.stdout);
}
```

We want our local IP address to be added to the access list and we want a user to be created. In this example, we are creating a user with extensive access, but you may want to refine the level of permission they have in your own project. For example, maybe the container deployment is meant to be a sandboxed experience. In this scenario, it makes sense that the user created access only the database and collection in the sandbox. We `sleep` after these commands because they are not instant and we want to make sure everything is ready before we try to connect.

Finally we try to connect:

```javascript
try {
    let connectionString = await $`atlas clusters connectionStrings describe ${process.env.MONGODB_CLUSTER_NAME} --output json`;
    let parsedConnectionString = new URL(JSON.parse(connectionString.stdout).standardSrv);
    parsedConnectionString.username = encodeURIComponent(process.env.MONGODB_USERNAME);
    parsedConnectionString.password = encodeURIComponent(process.env.MONGODB_PASSWORD);
    parsedConnectionString.search = "retryWrites=true&w=majority";
    process.env.MONGODB_ATLAS_URI = parsedConnectionString.toString();
    app = $`node main.js`;
} catch (error) {
    console.log(error.stdout);
}
```

After the first try / catch block finishes, we'll have a connection string. We can finalize our connection string with a Node.js URL object by including the username and password, then we can run our Node.js application. Remember, the environment variables and any manipulations we made to them in our script will be passed into the Node.js application.

## Transition the MongoDB Atlas workflow to containers with Docker and Docker Compose

At this point, we have an application and we have a script for preparing MongoDB Atlas and launching the application. It's time to get everything into a Docker image to be deployed as a container.

At the root of your project directory, add a **Dockerfile** file with the following:

```dockerfile
FROM node:18

WORKDIR /usr/src/app

COPY ./app/* ./
COPY ./docker_run_script.mjs ./

RUN curl https://fastdl.mongodb.org/mongocli/mongodb-atlas-cli_1.3.0_linux_x86_64.tar.gz --output mongodb-atlas-cli_1.3.0_linux_x86_64.tar.gz
RUN tar -xvf mongodb-atlas-cli_1.3.0_linux_x86_64.tar.gz && mv mongodb-atlas-cli_1.3.0_linux_x86_64 atlas_cli
RUN chmod +x atlas_cli/bin/atlas
RUN mv atlas_cli/bin/atlas /usr/bin/

RUN npm install -g zx
RUN npm install

EXPOSE 3000

CMD ["./docker_run_script.mjs"]
```

The custom Docker image will be based on a Node.js image which will allow us to run our Node.js application as well as our ZX script.

After our files are copied into the image, we run a few commands to download and extract the MongoDB Atlas CLI.

Finally, we install ZX and our application dependencies and run the ZX script. The `CMD` command for running the script is done when the container is run. Everything else is done when the image is built.

We could build our image from this **Dockerfile** file, but it is a lot easier to manage when there is a Compose configuration. Within the project directory, create a **docker-compose.yml** file with the following YAML:

```yaml
version: "3.9"
services:
    web:
        build:
            context: .
            dockerfile: Dockerfile
        ports:
            - "3000:3000"
        environment:
            MONGODB_ATLAS_PUBLIC_API_KEY: YOUR_PUBLIC_KEY_HERE
            MONGODB_ATLAS_PRIVATE_API_KEY: YOUR_PRIVATE_KEY_HERE
            MONGODB_ATLAS_ORG_ID: YOUR_ORG_ID_HERE
            MONGODB_ATLAS_PROJECT_ID: YOUR_PROJECT_ID_HERE
            MONGODB_CLUSTER_NAME: examples
            MONGODB_USERNAME: demo
            MONGODB_PASSWORD: password1234
            # MONGODB_DATABASE: sample_mflix
            # MONGODB_COLLECTION: movies
            CLEANUP_ONDESTROY: true
```

You'll want to swap the environment variable values with your own. In the above example, the database and collection variables are commented out so the defaults would be used in the ZX script.

To see everything in action, execute the following from the command line on the host computer:

```bash
docker-compose up
```

The above command will use the **docker-compose.yml** file to build the Docker image if it doesn't already exist. The build process will bundle our files, install our dependencies, and obtain the MongoDB Atlas CLI. When Compose deploys a container from the image, the environment variables will be passed to the ZX script responsible for configuring MongoDB Atlas. When ready, the ZX script will run the Node.js application, further passing the environment variables. If the `CLEANUP_ONDESTROY` variable was set to `true`, when the container is stopped the database and collection will be removed.

## Conclusion

The [MongoDB Atlas CLI](https://www.mongodb.com/docs/atlas/cli/stable/) can be a powerful tool for bringing MongoDB Atlas to your local development experience on Docker. Essentially you would be swapping out a local version of MongoDB with Atlas CLI logic to manage a more feature-rich cloud version of MongoDB.

MongoDB Atlas enhances the MongoDB experience by giving you access to more features such as Atlas Search, Charts, and App Services, which allow you to build great applications with minimal effort.

Learn how to use the MongoDB Atlas CLI with Docker in this example that includes JavaScript and Node.js.

Docker

Featured

How to Build a CRUD Application With MongoDB, Quarkus, and GraalVM

Microservices Architecture: Deploy Locally With Java, Spring, and MongoDB

How to Enable Local and Automatic Testing of Atlas Search-Based Features

All Docker Content