Vm.swappiness configuration on EC2 for MongoDB

Gareth_Furnell · 2024-05-31T07:59:28.829Z

Hi

I work on a production replica set of 10 nodes which has been previously increased to this size due to read ticket drops when high load occurs - it is a read heavy replica set, these are the custom settings on the EC2 servers which was recommended by Ops Manager after an upgrade from 5.0 to 6.0.14-ent:
cat /etc/sysctl.conf
vm.max_map_count=102400

I’ve noticed that this setting is default on the server:
cat /proc/sys/vm/swappiness
60

is this highly recommended to change to a value of 1 or do EC2 instances handle this differently?

Peter_Hubbard · 2024-05-31T08:27:57.721Z

Hi Gareth, I’m not aware of any specific reason why EC2 instances should handle swappiness differently. MongoDB relies on system memory for its own cache and the file system cache, so as to keep slower disk reads to a minimum. Anything which is being swapped from memory to disk is going to negatively affect that.

mongodb.com

Production Notes - MongoDB Manual v7.0

Gareth_Furnell · 2024-05-31T09:05:17.953Z

Hi @Peter_Hubbard

thanks for the speedy response,

The production notes is what I was double checking for this yes, thanks for the reference.
cat /proc/swaps
returns no filenames

we run on m5.12xlarge instance types
free -h

              total        used        free      shared  buff/cache   available
Mem:           184G         85G        1.1G        632K         97G         97G
Swap:            0B          0B          0B

with most likely ssd/nvme storage

with these settings in mind is it still best practice to set the vm.swappiness from 60 to 1?

Peter_Hubbard · 2024-05-31T12:18:51.262Z

If there is no swap file or partition then the OS will crash if it runs out of memory. So it is preferable to have a swap partition. In that case, make sure that the OS only uses the swap area in an emergency (ie. to prevent an OOM) by setting vm.swappiness to 1

Unless you explicitly change the Wiredtiger cache settings, MongoDB will use around half your RAM for its own cache. The rest of RAM will be available to the OS. The OS will use as much RAM as it can as a file-system cache, again to reduce the number of reads to disk it must make, but that RAM can always be made available other processes (this is the difference between ‘free’ and ‘available’ RAM).

My recommendations would be:

Create a swap file - 16Gb would be more than sufficient
Set swappiness to 1
Optimally make sure your machine is dedicated to MongoDB (ie, has no other major applications running).

Gareth_Furnell · 2024-06-03T10:33:46.957Z

Thanks for the detailed explanation @Peter_Hubbard

these nodes are dedicated to MongoDB only -
The WiredTigerCacheSizeGB is dedicated 100GB
the Total memory is 184GB as mentioned in the free -h command run and its usage.
index size has grown to 84GB

image1677×499 48.3 KB

only around 50% memory usage showing on the server side, just to reiterate, would this mean that all of the working set fits in RAM and there is no worries of swapping to disk since there is no swap filenames even with swappiness set to 60 in this current configuration, only in the case of a swap filenames existence would 60 do any action?

I understand the implementation you suggested for worst case scenarios to avoid OOM and will add it to my documentation… in Atlas, since resources are automatically scaled - would it increase the size of memory if it detects an alert of running out of memory or do configurations also have such a fail-safe by default?

system · 2024-06-08T10:34:29.243Z

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.