Ops Manager Architecture in Kubernetes
On this page
Important
This section is for single Kubernetes cluster deployments only. For multi-Kubernetes-cluster deployments, see Architecture, Capabilities, and Limitations.
You can use the Kubernetes Operator to deploy Ops Manager and MongoDB resources to a Kubernetes cluster. The Kubernetes Operator manages the lifecycle of each of these deployments differently.
All Kubernetes secrets that the Kubernetes Operator creates can later be migrated to a different secret storage tool to avoid storing secrets in Kubernetes.
The MongoDBOpsManager Custom Resource Definition
The Kubernetes Operator manages Ops Manager deployments using the
MongoDBOpsManager custom resource. The Kubernetes Operator watches
the custom resource's specification for changes. When the
specification changes, the Kubernetes Operator validates the changes and
makes the appropriate updates to the resources in the Kubernetes cluster.
MongoDBOpsManager custom resources specification defines the
following Ops Manager components:
the Application Database,
the Ops Manager application, and
the Backup Daemon.
Application Database
For the Application Database, the Kubernetes Operator deploys a MongoDB
replica set as a StatefulSet to the Kubernetes cluster. Kubernetes creates
one Pod in the StatefulSet for each member
that comprises your Application Database replica set. Each Pod in
the StatefulSet runs a mongod and the MongoDB Agent.
To enable each MongoDB Agent to start mongod on its
Pod in the StatefulSet, you must specify a specific MongoDB Server
version for the Application Database using the
spec.applicationDatabase.version setting. The version
that you specify in this setting must correspond to the tag in the
container registry. Each MongoDB Agent
starts the Application Database with the specified version on its Pod
in the StatefulSet.
After each MongoDB Agent starts mongods on its Application Database
Pod, the MongoDB Agents add all mongod processes to the Application
Database replica set.
You configure the number of replicas in and other
configuration options for the Application Database replica set in the
spec.applicationDatabase collection in the
MongoDBOpsManager custom resource. The Kubernetes Operator passes
this configuration to the MongoDB Agents using a secret that the
Kubernetes Operator mounts to each Pod in the Application Database StatefulSet.
Each time that you update
the spec.applicationDatabase collection, the
Kubernetes Operator applies the changes to the MongoDB Agent configuration and
the StatefulSet specification, if applicable. If the StatefulSet
specification changes, Kubernetes upgrades the Pods in a rolling
fashion and restarts each Pod.
The Kubernetes Operator creates a service with clusterIp=none to
provide connectivity to each Application Database Pod from within the
Kubernetes cluster.
Note
Depending on the StorageClass or the environment to which you deploy the Kubernetes Operator, Kubernetes might create the Persistent Volumes using dynamic volume provisioning.
You can customize the Persistent Volume Claims for the Application Database Pods using
the spec.applicationDatabase.podSpec.persistence.single or
spec.applicationDatabase.podSpec.persistence.multiple options.
Ops Manager Application
After the Application Database reaches a Running state, the Kubernetes Operator starts the Ops Manager Application. For Ops Manager, the Kubernetes Operator deploys a StatefulSet to the Kubernetes cluster. Kubernetes creates one Pod in the StatefulSet for each Ops Manager replica that you want to deploy. Each Pod contains one Ops Manager Application process.
Note
Deploy multiple Ops Manager replicas to make your deployment highly available in the event of an Ops Manager Pod failure.
The Kubernetes Operator creates a service with clusterIp=none to
allow clients deployed to the Kubernetes cluster to connect to Ops Manager. To
allow clients external to the Kubernetes cluster to connect to Ops Manager,
configure the spec.externalConnectivity collection in the
specification for your Ops Manager deployment.
Backup Daemon
If spec.backup.enabled is true, the Kubernetes Operator
starts the Backup Daemon after the Ops Manager Application reaches a Running
stage. For the Backup Daemon, Kubernetes Operator deploys a StatefulSet
to the Kubernetes cluster. Kubernetes creates one pod in the
StatefulSet for the Backup Daemon.
If you enable backup, you must provide additional fields in the
spec.backup collection to configure:
the oplog store and a blockstore or an S3 snapshot store. You can also
encrypt backup jobs, but
limitations apply to deployments where the same
Kubernetes Operator instance is not managing both the
MongoDBOpsManager and MongoDB
custom resources.
If you enable backup, the Kubernetes Operator creates a Persistent Volume Claim for the
Backup Daemon's head database. You can
configure the head database using the spec.backup.headDB
setting.
The Kubernetes Operator invokes Ops Manager APIs to ensure that the Ops Manager Application's backup configuration matches the one that you define in the custom resource definition.
Reconciling the MongoDBOpsManager Custom Resource
The following diagram describes how the Kubernetes Operator reconciles
changes to the MongoDBOpsManager CustomResourceDefinition.
The Kubernetes Operator creates or updates the
<om_resource_name>-db-configsecret. This secret contains the configurations that the MongoDB Agent uses to start the Application Database replica set.The Kubernetes Operator creates or updates the
<om_resource_name>-dbApplication Database StatefulSet. This StatefulSet contains at least three Pods.Each Pod runs one MongoDB Agent instance. Each MongoDB Agent starts a
mongodinstance on its Pod.The Kubernetes Operator mounts the
<om_resource_name>-db-configsecret to each Pod. The MongoDB Agent uses this secret to configure the Application Database replica set.
The Kubernetes Operator creates or updates the
<om_resource_name>StatefulSet. This StatefulSet contains one Pod for each Ops Manager replica. Each Ops Manager replica connects to the Application Database.Note
Most changes to the
MongoDBOpsManagercustom resource trigger a rolling upgrade of the Pods in the<om_resource_name>StatefulSet. Enabling TLS for the Application Database also triggers a rolling restart because the connection string to the Application Database changes.Changes to the
spec.backupMongoDBOpsManagercustom resource collection don't trigger a rolling upgrade.The Kubernetes Operator invokes Ops Manager APIs to create an admin user. The Kubernetes Operator saves this admin user's credentials in the
<om_resource_name>-admin-keysecret. The Kubernetes Operator uses these credentials for all other Ops Manager API invocations.Note
This reconciliation step happens only once: when you use the Kubernetes Operator to create an Ops Manager resource. The Kubernetes Operator skips this step when it updates the resource.
The Kubernetes Operator performs a rolling upgrade of the Pods in the
<om_resource_name>-dbApplication Database StatefulSet to enable Ops Manager to monitor it.Note
This reconciliation step happens only when you enable Monitoring for an application database for the first time. This happens most often when you deploy a new Ops Manager resource.
If
spec.backup.enabledis true, the Kubernetes Operator creates the<om_resource_name>-backup-daemonStatefulSet or verifies that it is running. The Kubernetes Operator mounts a Persistent Volume for the head database.Note
The Backup Daemon connects to the same Application Database as the Ops Manager deployment.
If
spec.backup.enabledis true, the Kubernetes Operator invokes Ops Manager APIs to ensure that the Ops Manager Application's backup configuration matches the one that you define in the custom resource definition.