Guidance for Atlas Scalability

Features for Atlas Scalability

Auto-scaling enables clusters to automatically adjust their tier, storage capacity, or both, in response to real-time use. Atlas uses complementary auto-scaling mechanisms:

• Reactive auto-scaling analyzes current CPU and memory utilization to trigger scaling events based on sustained high or low resource usage. Reactive auto-scaling occurs after workload changes and handles both upscaling and downscaling.

• Predictive auto-scaling uses machine learning to anticipate future scaling needs based on historical usage patterns and attempts to scale up before cyclical workload spikes that it forecasts will arrive. Predictive auto-scaling is available for eligible clusters and supports only upscaling. Downscaling remains reactive.

See Cluster tier scaling to learn more about the conditions under which Atlas scales your cluster nodes up or down. You can also specify a range of maximum and minimum cluster sizes that your cluster can automatically scale to in order to guarantee minimum performance or control costs. Atlas generally won't scale a cluster if scaling would result in a tier that falls outside of your specified size range, though cluster tier and storage might scale in parallel if disk needs require it. Atlas also won't scale if your memory or CPU usage would exceed the capacity of the scaled tier.

Auto-scaling is best suited for steadily growing or declining application loads and cyclical workload patterns. For unexpected, one-time traffic spikes such as product launches or unprecedented events, pre-scale programmatically. Reactive auto-scaling introduces a throttled delay to prevent application impact, and predictive auto-scaling handles only recurring patterns.

Atlas deployment templates, as referenced in the Recommended Deployment Topologies, provide you with horizontal and vertical scaling options. Specifically, sharding distributes data across numerous machines, which is useful when no single server can handle your workloads. Sharding follows a shared-nothing architecture, a distributed computing architecture where none of the nodes share any resources with each other. See Choose a Shard Key to learn more about the ideal choice of shard key that allows MongoDB to distribute documents evenly throughout your cluster while facilitating common query patterns. Furthermore, see Performance Best Practices: Sharding to learn about the key sharding strategies, such as ranged sharding, hashed sharding, and zoned sharding.

Upgrading an Atlas cluster to the next available Atlas tier is available through the Atlas control plane GUI, the Atlas Administration API, or through IaC tools, such as the Atlas Kubernetes Operator, the MongoDB & HashiCorp Terraform, or the Atlas CLI. See Guidance for Atlas Automated Infrastructure Provisioning to learn more. Changing an Atlas tier, either upscaling or downscaling, allows zero downtime. The tier changes in a rolling fashion, which involves electing a secondary member as a replacement, promoting this secondary member to become the new primary, then restoring or replacing the failing member to ensure that the cluster is returned to its target configuration as soon as possible. Horizontal scaling occurs post-deployment based on Administrator action, which can be triggered from a programmatic script. Some cluster templates require sharded clusters. Starting with MongoDB version 8.0, you may make use of embedded config servers to reduce costs associated with config servers on small sharded clusters.

The low CPU option in Atlas helps applications that require higher memory but not as much processing power. This option provides instances with half the vCPUs compared to the General tier of the same cluster size, reducing costs for workloads that are memory-intensive but not CPU-bound.

Data tiering and archival allows you to archive data in low-cost storage while still enabling queries alongside live cluster data, which is particularly useful for long-term record retention. To optimize this process, MongoDB recommends that you automate data archiving with simple, configurable rules. See Archive Data to learn more about the criteria that you can specify in an archiving rule. For scenarios where data retention is not a priority, Atlas offers the option to automatically delete unused data based on date criteria. For infrequently accessed data, TTL indexes are special single-field indexes that automatically delete documents from a collection after a specified period or at a set clock time. This is particularly useful for data like logs, session information, or event data that only needs to be retained for a limited time. To create a TTL index, you can define an index on a field that holds date values and specify a time-to-live duration in seconds.

Atlas also provides you with automated tools, such as the Performance Advisor, to identify and optimize inefficient queries by adding or removing an index or changing your client's query structure. You can reduce unnecessary compute time and resource consumption by following Performance Advisor's actionable recommendations to enhance your query performance. Additionally, you can leverage intelligent index recommendations provided by Atlas to further improve data retrieval efficiency and minimize the resources needed for database operations.

Recommendations for Atlas Scalability

Automation Examples: Atlas Scalability

The following examples enable auto-scaling compute and storage using Atlas tools for automation.

These examples also apply other recommended configurations, including:

{
    "clusterType": "REPLICASET",
    "links": [],
    "name": "CustomerPortalProd",
    "mongoDBMajorVersion": "8.0",
    "replicationSpecs": [
      {
        "numShards": 1,
        "regionConfigs": [
          {
            "electableSpecs": {
              "instanceSize": "M30",
              "nodeCount": 3
            },
            "priority": 7,
            "providerName": "GCP",
            "regionName": "EASTERN_US",
            "analyticsSpecs": {
              "nodeCount": 0,
              "instanceSize": "M30"
            },
            "autoScaling": {
              "compute": {
                "enabled": true,
                "scaleDownEnabled": true
              },
              "diskGB": {
                "enabled": true
              }
            },
            "readOnlySpecs": {
              "nodeCount": 0,
              "instanceSize": "M30"
            }
          }
        ],
        "zoneName": "Zone 1"
      }
    ]
  }

atlas cluster create --projectId 5e2211c17a3e5a48f5497de3 --file cluster.json

# Create a Group to Assign to Project
resource "mongodbatlas_team" "project_group" {
  org_id = var.atlas_org_id
  name   = var.atlas_group_name
  usernames = [
    "user1@example.com",
    "user2@example.com"
  ]
}
# Create a Project
resource "mongodbatlas_project" "atlas-project" {
  org_id = var.atlas_org_id
  name = var.atlas_project_name
}
# Assign the team to project with specific roles
resource "mongodbatlas_team_project_assignment" "project_team" {
  project_id = mongodbatlas_project.atlas-project.id
  team_id    = mongodbatlas_team.project_group.team_id
  role_names = ["GROUP_READ_ONLY", "GROUP_CLUSTER_MANAGER"]
}
# Create an Atlas Advanced Cluster
resource "mongodbatlas_advanced_cluster" "atlas-cluster" {
  project_id = mongodbatlas_project.atlas-project.id
  name = "ClusterPortalProd"
  cluster_type = "REPLICASET"
  mongo_db_major_version = var.mongodb_version
  use_effective_fields = true
  replication_specs = [
    {
      region_configs = [
        {
          electable_specs = {
            instance_size = var.cluster_instance_size_name
            node_count    = 3
            disk_size_gb  = var.disk_size_gb
          }
          auto_scaling = {
            disk_gb_enabled = var.auto_scaling_disk_gb_enabled
            compute_enabled = var.auto_scaling_compute_enabled
            compute_max_instance_size = var.compute_max_instance_size
          }
          priority      = 7
          provider_name = var.cloud_provider
          region_name   = var.atlas_region
        }
      ]
    }
  ]
  tags = {
    BU       = "ConsumerProducts"
    TeamName = "TeamA"
    AppName  = "ProductManagementApp"
    Env      = "Production"
    Version  = "8.0"
   Email    = "marissa@example.com"
  }
}
# Outputs to Display
output "atlas_cluster_connection_string" { value = mongodbatlas_advanced_cluster.atlas-cluster.connection_strings.standard_srv }
output "project_name"      { value = mongodbatlas_project.atlas-project.name }

replication_specs = [
  {
    region_configs = [
      {
        electable_specs = {
          instance_size = "M10"
          node_count    = 2
        }
        provider_name = "GCP"
        priority      = 7
        region_name   = "NORTH_AMERICA_NORTHEAST_1"
      },
      {
        electable_specs = {
          instance_size = "M10"
          node_count    = 3
        }
        provider_name = "GCP"
        priority      = 6
        region_name   = "WESTERN_US"
      }
    ]
  }
]

# Atlas Organization ID
variable "atlas_org_id" {
  type        = string
  description = "Atlas Organization ID"
}
# Atlas Project Name
variable "atlas_project_name" {
  type        = string
  description = "Atlas Project Name"
}
# Atlas Group Name
variable "atlas_group_name" {
  type        = string
  description = "Atlas Group Name"
}
# Atlas Project Environment
variable "environment" {
  type        = string
  description = "The environment to be built"
}
# Cluster Instance Size Name
variable "cluster_instance_size_name" {
  type        = string
  description = "Cluster instance size name"
}
# Cloud Provider to Host Atlas Cluster
variable "cloud_provider" {
  type        = string
  description = "AWS or GCP or Azure"
}
# Atlas Region
variable "atlas_region" {
  type        = string
  description = "Atlas region where resources will be created"
}
# MongoDB Version
variable "mongodb_version" {
  type        = string
  description = "MongoDB Version"
}
# Storage Auto-scaling Enablement Flag
variable "auto_scaling_disk_gb_enabled" {
  type        = bool
  description = "Flag that specifies whether disk auto-scaling is enabled"
}
# Compute Auto-scaling Enablement Flag
variable "auto_scaling_compute_enabled" {
  type        = bool
  description = "Flag that specifies whether cluster tier auto-scaling is enabled"
}
# Disk Size in GB
variable "disk_size_gb" {
  type        = int
  description = "Disk Size in GB"
}

atlas_org_id = "32b6e34b3d91647abb20e7b8"
atlas_project_name = "Customer Portal - Prod"
atlas_group_name = "Atlas Group"
environment = "prod"
cluster_instance_size_name = "M30"
cloud_provider = "AWS"
atlas_region = "US_WEST_2"
mongodb_version = "8.0"
auto_scaling_disk_gb_enabled = true
auto_scaling_compute_enabled = true
disk_size_gb = 40000

# Define the MongoDB Atlas Provider
terraform {
  required_providers {
    mongodbatlas = {
      source = "mongodb/mongodbatlas"
      version = "~> 2.2"
    }
  }
  required_version = ">= 1.0"
}
# Configure the MongoDB Atlas Provider
provider "mongodbatlas" {
  # Legacy API key authentication (backward compatibility)
  public_key  = var.mongodbatlas_public_key
  private_key = var.mongodbatlas_private_key
  # Recommended: Service account authentication
  # Uncomment and configure the following for service account auth:
  # service_account_id = var.mongodb_service_account_id
  # private_key_file   = var.mongodb_service_account_key_file
}

terraform init

terraform plan

terraform apply