Docs Menu
Docs Home
/
MongoDB Enterprise Kubernetes Operator
/
Deploy MongoDB Resources on Multiple Kubernetes Clusters

Multi-Kubernetes-Cluster Quick Start

On this page

  • Prefer to Learn by Watching?
  • Deploy a MongoDBMultiCluster Resource
  • Create the secret for the TLS certificate of your MongoDBMultiCluster resource.
  • Create the ConfigMap to link your CA with your MongoDBMultiCluster resource.
  • Run the kubectl mongodb plugin.
  • Optional: Set the Istio injection webhook in each member cluster.
  • Configure kubectl to use the central cluster's namespace.
  • Deploy the MongoDB Enterprise Kubernetes Operator in the central cluster.
  • Create a secret to use with Ops Manager and create a ConfigMap.
  • Deploy Ops Manager on the central cluster and connect to Ops Manager.
  • Deploy the MongoDBMultiCluster resource resource.
  • Verify that the MongoDBMultiCluster resources are running.
  • Renew TLS Certificates for a MongoDBMultiCluster Resource
  • Run the kubectl mongodb plugin.
  • Optional: Set the Istio injection webhook in each member cluster.
  • Configure kubectl to use the central cluster's namespace.
  • Deploy the MongoDB Enterprise Kubernetes Operator in the central cluster.
  • Create a secret to use with Ops Manager and create a ConfigMap.
  • Deploy Ops Manager on the central cluster and connect to Ops Manager.
  • Deploy the MongoDB resource.
  • Verify that the MongoDBMultiCluster resources are running.
  • Next Steps

Use the quick start to deploy a MongoDB replica set across three Kubernetes member clusters, using GKE (Google Kubernetes Engine) and Istio service mesh.

Before you begin:

  • Learn about multi-Kubernetes-cluster deployments

  • Review the list of multi-Kubernetes-cluster services and tools

  • Complete the prerequisites

Note

The following procedures scope your multi-Kubernetes-cluster deployment to a single namespace named mongodb. You can configure your multi-Kubernetes-cluster deployment to watch resources in multiple namespaces or all namespaces.

Prefer to Learn by Watching?

Follow along with this video tutorial walk-through that demonstrates how to create a multi-Kubernetes-cluster deployment.

Duration: 12 Minutes

Deploying a MongoDB Replica Set across Multiple Kubernetes Clusters

Deploy a MongoDBMultiCluster Resource

Select the appropriate tab based on whether you want to encrypt replica set connections in your multi-Kubernetes-cluster deployments using TLS certificates.

You can use the following procedures in this TLS-Encrypted Connections tab:

  • Deploy a MongoDBMultiCluster resource

  • Renew TLS Certificates for a MongoDBMultiCluster resource

These procedures establish TLS-encrypted connections between MongoDB hosts in a replica set, and between client applications and MongoDB deployments.

Before you begin, you must have valid certificates for TLS encryption.

1

Create the secret for the TLS certificate of your MongoDBMultiCluster resource.

Run the kubectl command to create a new secret that stores the MongoDBMultiCluster resource certificate:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
  --namespace=<metadata.namespace> \
  create secret tls <prefix>-<metadata.name>-cert \
  --cert=<resource-tls-cert> \
  --key=<resource-tls-key>

Note

You must prefix your secrets with <prefix>-<metadata.name>.

Example

If you call your deployment my-deployment and you set the prefix to mdb, you must name the TLS secret for the client TLS communications mdb-my-deployment-cert. Also, you must name the TLS secret for internal cluster authentication (if enabled) mdb-my-deployment-clusterfile.

2

Create the ConfigMap to link your CA with your MongoDBMultiCluster resource.

Run the kubectl command to link your CA to your MongoDBMultiCluster resource. Specify the CA certificate file that you must always name ca-pem for the MongoDBMultiCluster resource:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
  --namespace=<metadata.namespace> \
  create configmap custom-ca -from-file=ca-pem=<your-custom-ca-file>
3

Run the kubectl mongodb plugin.

By default, the Kubernetes Operator is scoped to the mongodb namespace. When you run the following command, the kubectl mongodb plugin:

  • Creates one central cluster, three member clusters, and a namespace labeled mongodb in each of the clusters.

  • Creates a default ConfigMap with the hard-coded name mongodb-enterprise-operator-member-list that contains all the member clusters. You can't change the ConfigMap's name.

  • Creates the service accounts and Roles required for running database workloads in the member clusters.

Run the kubectl mongodb plugin:

kubectl mongodb multicluster setup \
  --central-cluster="${MDB_CENTRAL_CLUSTER_FULL_NAME}" \
  --member-clusters="${MDB_CLUSTER_1_FULL_NAME},${MDB_CLUSTER_2_FULL_NAME},${MDB_CLUSTER_3_FULL_NAME}" \
  --member-cluster-namespace="mongodb" \
  --central-cluster-namespace="mongodb" \
  --create-service-account-secrets \
  --install-database-roles=true
4

Optional: Set the Istio injection webhook in each member cluster.

If you're using Istio, run the following command on the central cluster, specifying the context for each of the member clusters in the deployment. These commands add the istio-injection=enabled label to the mongodb namespace on each member cluster. This label configures Istio's injection webhook which enables adding a sidecar to any Pods that you create in this namespace. To learn more, see Automatic sidecar injection in the Istio documentation.

kubectl label \
  --context=$MDB_CLUSTER_1_FULL_NAME \
  namespace mongodb \
  istio-injection=enabled
kubectl label \
  --context=$MDB_CLUSTER_2_FULL_NAME \
  namespace mongodb \
  istio-injection=enabled
kubectl label \
  --context=$MDB_CLUSTER_3_FULL_NAME \
  namespace mongodb \
  istio-injection=enabled
5

Configure kubectl to use the central cluster's namespace.

If you have not done so already, run the following commands to run all kubectl commands on the central cluster in the default namespace.

kubectl config use-context $MDB_CENTRAL_CLUSTER_FULL_NAME
kubectl config set-context $(kubectl config current-context) \
  --namespace=mongodb
6

Deploy the MongoDB Enterprise Kubernetes Operator in the central cluster.

Deploy the MongoDB Enterprise Kubernetes Operator in the central cluster in the mongodb namespace with Helm or kubectl.

  1. Add the MongoDB Helm Charts for Kubernetes repository to Helm.

    helm repo add mongodb https://mongodb.github.io/helm-charts

  2. Use the MongoDB Helm Charts for Kubernetes to deploy the Kubernetes Operator.

    helm upgrade \
      --install \
        mongodb-enterprise-operator-multi-cluster \
        mongodb/enterprise-operator \
          --namespace mongodb \
          --set namespace=mongodb \
          --version <mongodb-kubernetes-operator-version> \
          --set operator.name=mongodb-enterprise-operator-multi-cluster \
          --set operator.createOperatorServiceAccount=false \
          --set "multiCluster.clusters={$MDB_CLUSTER_1_FULL_NAME,$MDB_CLUSTER_2_FULL_NAME,$MDB_CLUSTER_3_FULL_NAME}" \
          --set multiCluster.performFailover=false

  1. Apply the Kubernetes Operator custom resources.

    kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-enterprise-kubernetes/master/crds.yaml

  2. Download the Kubernetes Operator YAML template.

    curl https://raw.githubusercontent.com/mongodb/mongodb-enterprise-kubernetes/master/mongodb-enterprise-multi-cluster.yaml -o operator.yaml

  3. Optional: Customize the Kubernetes Operator YAML template.

    To learn about optional Kubernetes Operator installation settings, see MongoDB Enterprise Kubernetes Operator kubectl and oc Installation Settings.

  4. Apply the Kubernetes Operator YAML file.

    kubectl apply -f operator.yaml

  5. Verify that the Kubernetes Operator is deployed.

    To verify that the Kubernetes Operator installed correctly, run the following command and verify the output:

    kubectl describe deployments mongodb-enterprise-operator -n <metadata.namespace>
    oc describe deployments mongodb-enterprise-operator -n <metadata.namespace>

    By default, deployments exist in the mongodb namespace. If the following error message appears, ensure you use the correct namespace:

    Error from server (NotFound): deployments.apps "mongodb-enterprise-operator" not found

    To troubleshoot your Kubernetes Operator, see Review Logs from the Kubernetes Operator and other troubleshooting topics.

    Important

    If you need to remove the Kubernetes Operator or the namespace, you first must remove MongoDB resources.

7

Create a secret to use with Ops Manager and create a ConfigMap.

  1. Create a secret so that the Kubernetes Operator can create and update objects in your Ops Manager project. To learn more, see Create Credentials for the Kubernetes Operator.

  2. Create a ConfigMap to link the Kubernetes Operator to your Ops Manager project. To learn more, see Create One Project using a ConfigMap.

8

Deploy Ops Manager on the central cluster and connect to Ops Manager.

Review the multi-Kubernetes-cluster deployment exceptions and follow the linked procedures in Deploy an Ops Manager Resource on the Central Cluster and Connect to Ops Manager.

9

Deploy the MongoDBMultiCluster resource resource.

Set spec.credentials, spec.opsManager.configMapRef.name, and security settings and deploy the MongoDBMultiCluster resource. In the following code sample, duplicateServiceObjects is set to true to enable DNS proxying in Istio.

Note

To enable the cross-cluster DNS resolution by the Istio service mesh, this tutorial creates service objects with a single ClusterIP address per each Kubernetes Pod.

kubectl apply -f - <<EOF
apiVersion: mongodb.com/v1
kind: MongoDBMultiCluster
metadata:
 name: multi-replica-set
spec:
 version: 4.4.0-ent
 type: ReplicaSet
 persistent: false
 duplicateServiceObjects: true
 credentials: my-credentials
 opsManager:
   configMapRef:
     name: my-project
 security:
   certsSecretPrefix: <prefix>
   tls:
     ca: custom-ca
 clusterSpecList:
   - clusterName: ${MDB_CLUSTER_1_FULL_NAME}
     members: 3
   - clusterName: ${MDB_CLUSTER_2_FULL_NAME}
     members: 2
   - clusterName: ${MDB_CLUSTER_3_FULL_NAME}
     members: 3
EOF

The Kubernetes Operator copies the ConfigMap with the CA that you created in previous steps to each member cluster, generates a concatenated PEM secret, and distributes it to the member clusters.

10

Verify that the MongoDBMultiCluster resources are running.

  1. For member clusters, run the following commands to verify that the MongoDB Pods are in the running state:

    kubectl get pods \
     --context=$MDB_CLUSTER_1_FULL_NAME \
     --namespace mongodb
    kubectl get pods \
     --context=$MDB_CLUSTER_2_FULL_NAME \
     --namespace mongodb
    kubectl get pods \
     --context=$MDB_CLUSTER_3_FULL_NAME \
     --namespace mongodb

  2. In the central cluster, run the following command to verify that the MongoDBMultiCluster resource is in the running state:

    kubectl --context=$MDB_CENTRAL_CLUSTER_FULL_NAME \
      --namespace mongodb \
      get mdbmc multi-replica-set -o yaml -w

Renew TLS Certificates for a MongoDBMultiCluster Resource

Renew your TLS certificates periodically using the following procedure.

1

Renew the secret for a MongoDBMultiCluster resource.

Run this kubectl command to renew an existing secret that stores the certificates for the MongoDBMultiCluster resource:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
--namespace=<metadata.namespace> \
create secret tls <prefix>-<metadata.name>-cert \
--cert=<resource-tls-cert> \
--key=<resource-tls-key> \
--dry-run=client \
-o yaml |
kubectl apply -f -

This procedure doesn't encrypt connections between MongoDB hosts in a replica set, and between client applications and MongoDB deployments.

1

Run the kubectl mongodb plugin.

By default, the Kubernetes Operator is scoped to the mongodb namespace. When you run the following command, the kubectl mongodb plugin:

  • Creates one central cluster, three member clusters, and a namespace labeled mongodb in each of the clusters.

  • Creates a default ConfigMap with the hard-coded name mongodb-enterprise-operator-member-list that contains all the member clusters. You can't change the ConfigMap's name.

  • Creates the service accounts and Roles required for running database workloads in the member clusters.

Run the kubectl mongodb plugin:

kubectl mongodb multicluster setup \
  --central-cluster="${MDB_CENTRAL_CLUSTER_FULL_NAME}" \
  --member-clusters="${MDB_CLUSTER_1_FULL_NAME},${MDB_CLUSTER_2_FULL_NAME},${MDB_CLUSTER_3_FULL_NAME}" \
  --member-cluster-namespace="mongodb" \
  --central-cluster-namespace="mongodb" \
  --create-service-account-secrets \
  --install-database-roles=true
2

Optional: Set the Istio injection webhook in each member cluster.

If you're using Istio, run the following command on the central cluster, specifying the context for each of the member clusters in the deployment. These commands add the istio-injection=enabled label to the mongodb namespace on each member cluster. This label configures Istio's injection webhook which enables adding a sidecar to any Pods that you create in this namespace. To learn more, see Automatic sidecar injection in the Istio documentation.

kubectl label \
  --context=$MDB_CLUSTER_1_FULL_NAME \
  namespace mongodb \
  istio-injection=enabled
kubectl label \
  --context=$MDB_CLUSTER_2_FULL_NAME \
  namespace mongodb \
  istio-injection=enabled
kubectl label \
  --context=$MDB_CLUSTER_3_FULL_NAME \
  namespace mongodb \
  istio-injection=enabled
3

Configure kubectl to use the central cluster's namespace.

If you have not done so already, run the following commands to run all kubectl commands on the central cluster in the default namespace.

kubectl config use-context $MDB_CENTRAL_CLUSTER_FULL_NAME
kubectl config set-context $(kubectl config current-context) \
  --namespace=mongodb
4

Deploy the MongoDB Enterprise Kubernetes Operator in the central cluster.

Deploy the MongoDB Enterprise Kubernetes Operator in the central cluster in the mongodb namespace with Helm or kubectl.

  1. Add the MongoDB Helm Charts for Kubernetes repository to Helm.

    helm repo add mongodb https://mongodb.github.io/helm-charts

  2. Use the MongoDB Helm Charts for Kubernetes to deploy the Kubernetes Operator.

    helm upgrade \
      --install \
        mongodb-enterprise-operator-multi-cluster \
        mongodb/enterprise-operator \
          --namespace mongodb \
          --set namespace=mongodb \
          --version <mongodb-kubernetes-operator-version> \
          --set operator.name=mongodb-enterprise-operator-multi-cluster \
          --set operator.createOperatorServiceAccount=false \
          --set "multiCluster.clusters={$MDB_CLUSTER_1_FULL_NAME,$MDB_CLUSTER_2_FULL_NAME,$MDB_CLUSTER_3_FULL_NAME}" \
          --set multiCluster.performFailover=false

  1. Apply the Kubernetes Operator custom resources.

    kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-enterprise-kubernetes/master/crds.yaml

  2. Download the Kubernetes Operator YAML template.

    curl https://raw.githubusercontent.com/mongodb/mongodb-enterprise-kubernetes/master/mongodb-enterprise-multi-cluster.yaml -o operator.yaml

  3. Optional: Customize the Kubernetes Operator YAML template.

    To learn about optional Kubernetes Operator installation settings, see MongoDB Enterprise Kubernetes Operator kubectl and oc Installation Settings.

  4. Apply the Kubernetes Operator YAML file.

    kubectl apply -f operator.yaml

  5. Verify that the Kubernetes Operator is deployed.

    To verify that the Kubernetes Operator installed correctly, run the following command and verify the output:

    kubectl describe deployments mongodb-enterprise-operator -n <metadata.namespace>
    oc describe deployments mongodb-enterprise-operator -n <metadata.namespace>

    By default, deployments exist in the mongodb namespace. If the following error message appears, ensure you use the correct namespace:

    Error from server (NotFound): deployments.apps "mongodb-enterprise-operator" not found

    To troubleshoot your Kubernetes Operator, see Review Logs from the Kubernetes Operator and other troubleshooting topics.

    Important

    If you need to remove the Kubernetes Operator or the namespace, you first must remove MongoDB resources.

5

Create a secret to use with Ops Manager and create a ConfigMap.

  1. Create a secret so that the Kubernetes Operator can create and update objects in your Ops Manager project. To learn more, see Create Credentials for the Kubernetes Operator.

  2. Create a ConfigMap to link the Kubernetes Operator to your Ops Manager project. To learn more, see Create One Project using a ConfigMap.

6

Deploy Ops Manager on the central cluster and connect to Ops Manager.

Review the multi-Kubernetes-cluster deployment exceptions and follow the linked procedures in Deploy an Ops Manager Resource on the Central Cluster and Connect to Ops Manager.

7

Deploy the MongoDB resource.

Set spec.credentials, spec.opsManager.configMapRef.name, and security settings and deploy the MongoDBMultiCluster resource. In the following code sample, duplicateServiceObjects is set to true to enable DNS proxying in Istio.

Note

To enable the cross-cluster DNS resolution by the Istio service mesh, this tutorial creates service objects with a single ClusterIP address per each Kubernetes Pod.

kubectl apply -f - <<EOF
apiVersion: mongodb.com/v1
kind: MongoDBMultiCluster
metadata:
 name: multi-replica-set
spec:
 version: 4.4.0-ent
 type: ReplicaSet
 persistent: false
 duplicateServiceObjects: true
 credentials: my-credentials
 opsManager:
   configMapRef:
     name: my-project
 security:
   certsSecretPrefix: <prefix>
   tls:
     ca: custom-ca
 clusterSpecList:
   - clusterName: ${MDB_CLUSTER_1_FULL_NAME}
     members: 3
   - clusterName: ${MDB_CLUSTER_2_FULL_NAME}
     members: 2
   - clusterName: ${MDB_CLUSTER_3_FULL_NAME}
     members: 3
EOF

The Kubernetes Operator copies the ConfigMap with the CA that you created in previous steps to each member cluster, generates a concatenated PEM secret, and distributes it to the member clusters.

8

Verify that the MongoDBMultiCluster resources are running.

  1. For member clusters, run the following commands to verify that the MongoDB Pods are in the running state:

    kubectl get pods \
     --context=$MDB_CLUSTER_1_FULL_NAME \
     --namespace mongodb
    kubectl get pods \
     --context=$MDB_CLUSTER_2_FULL_NAME \
     --namespace mongodb
    kubectl get pods \
     --context=$MDB_CLUSTER_3_FULL_NAME \
     --namespace mongodb

  2. In the central cluster, run the following command to verify that the MongoDBMultiCluster resource is in the running state:

    kubectl --context=$MDB_CENTRAL_CLUSTER_FULL_NAME \
      --namespace mongodb \
      get mdbmc multi-replica-set -o yaml -w

Next Steps

After deploying your MongoDB replica set across three Kubernetes member clusters, you can add a database user so you can connect to your MongoDB database. See Manage Database Users.

Back

Prerequisites

Next

Deploy a MongoDBMultiCluster Resource

On this page