Deploy Replica Sets in a Multi-Kubernetes Cluster without a Service Mesh

Create the secret for the TLS certificate of your MongoDBMultiCluster resource.

Run the kubectl command to create a new secret that stores the MongoDBMultiCluster resource certificate:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
  --namespace=<metadata.namespace> \
  create secret tls <prefix>-<metadata.name>-cert \
  --cert=<resource-tls-cert> \
  --key=<resource-tls-key>

For example, if you call your deployment my-deployment and you set the prefix to mdb, you must name the TLS secret for the client TLS communications mdb-my-deployment-cert. Also, you must name the TLS secret for internal cluster authentication (if enabled) mdb-my-deployment-clusterfile.

Create the ConfigMap to link your CA with your MongoDBMultiCluster resource.

Run the kubectl command to link your CA to your MongoDBMultiCluster resource. Specify the CA certificate file that you must always name ca-pem for the MongoDBMultiCluster resource:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
  --namespace=<metadata.namespace> \
  create configmap custom-ca -from-file=ca-pem=<your-custom-ca-file>

Configure kubectl to use the central cluster's namespace.

If you have not done so already, run the following commands to run all kubectl commands on the central cluster in the default namespace.

kubectl config use-context $MDB_CENTRAL_CLUSTER_FULL_NAME
kubectl config set-context $(kubectl config current-context) \
  --namespace=mongodb

Copy and paste the sample resource.

1. Copy the sample replica set YAML file and paste it into a new text file.

2. Change the file's settings to match your desired replica set configuration.

1
# Provides statefulSet override per cluster
2
3
apiVersion: mongodb.com/v1
4
kind: MongoDBMultiCluster
5
metadata:
6
  name: multi-replica-set
7
spec:
8
  version: 6.0.0-ent
9
  type: ReplicaSet
10
  credentials: my-credentials
11
  opsManager:
12
    configMapRef:
13
      name: my-project
14
  externalAccess:
15
    externalService: 
16
      annotations:
17
        # Global cloud-specific annotations added to external services in all clusters
18
       spec:
19
        # ServiceSpec attributes to override in external services in all clusters
20
  clusterSpecList:
21
    - clusterName: cluster1.example.com
22
      members: 2
23
      externalAccess:
24
        # Domain suffix that mongod processes will use in cluster1
25
        externalDomain: cluster1.example.com
26
        externalService:
27
          annotations:
28
            # Cloud-specific annotations for external services
29
          spec:
30
            # ServiceSpec attributes to override if necessary
31
    - clusterName: cluster2.example.com
32
      members: 1
33
      externalAccess:
34
        # Domain suffix that mongod processes will use in cluster2
35
        externalDomain: cluster2.example.com
36
        externalService:
37
          annotations:
38
            # Cloud-specific annotations for external services
39
          spec:
40
            # ServiceSpec attributes to override if necessary
41
    - clusterName: cluster3.example.com
42
      members: 1
43
      externalAccess:
44
        # Domain suffix that mongod processes will use in cluster3
45
        externalDomain: cluster3.example.com
46
        externalService:
47
          annotations:
48
            # Cloud-specific annotations for external services
49
          spec:
50
            # ServiceSpec attributes to override if necessary
51
52
...

Define external connectivity settings.

Specify global values that affect all clusters in a multi-Kubernetes cluster MongoDB deployment using the spec.externalAccess settings and cluster-specific overrides using the spec.clusterSpecList.externalAccess.externalService settings.

When you provide these settings in the MongoDBMultiCluster resource specification, the Kubernetes Operator creates external services for each Pod in all Kubernetes clusters. You then use these services to establish external connectivity to all mongod processes in your deployment.

Define an external domain for each Kubernetes member cluster.

Define an external domain for each member cluster using the spec.clusterSpecList.externalAccess.externalDomain setting.

As a result, the Kubernetes Operator registers all mongod processes in the Kubernetes member cluster under a hostname according to the following convention:

<pod-name>.<externalDomain>

For example, a mongod process may have the following hostname: my-replica-set-0-0.cluster-1.example.com.

Change the settings to your preferred values.

Add any additional accepted settings for a MongoDBMultiCluster resource deployment.

You can also add any optional settings to the object specification. See Multi-Kubernetes-Cluster Resource Specification.

Save this replica set config file with a .yaml extension.

Start your replica set deployment.

In any directory, invoke the following Kubernetes command to create your replica set:

kubectl apply -f <replica-set-conf>.yaml

Verify external connectivity for each member cluster.

1. Check the status of external services in all member clusters:

   kubectl get services

   Kubernetes should return one external service created for each Pod of the replica set in all member clusters.

2. Verify that each external service is exposed externally and is reachable. Run the command similar to the following example:

   mongosh mongodb://my-replica-set-0-0.cluster-0.example.com:27017 \
   -tls -tlsCAFile "issuer-ca.pem"

   Connecting to my-replica-set-0-0.cluster-0.example.com:27017 should direct client traffic to an external service named my-replica-set-0-0-svc-external, which, in turn, directs traffic to the mongod process.

3. Configure your DNS zone for the specified external domain to point to the corresponding external services. This configuration depends on your environment or the cloud provider you are using.

Track the status of your multi-Kubernetes cluster MongoDB deployment.

To check the status of your MongoDBMultiCluster resource, use the following command on the central cluster:

kubectl get mdbmc <resource-name> -o yaml -w

With the -w (watch) flag set, when the configuration changes, the output refreshes immediately until the status phase achieves the Running state. To learn more about resource deployment statuses, see Troubleshoot the Kubernetes Operator.