MongoDB Server Version: v6.0.16
Mongo Client Version: v1.16.0 (golang driver)
VM Details
CPU - 2 cores
Memory - 4 GB
Running on GCP
It is a 3 node replicaset
In our production environment, we have noticed spikes in CPU load occurring every 20 minutes within one of our applications that handles a workload characterized by heavy write and read operations. The production setup consists of a 40-core machine with 200 GB RAM configured as a three-node cluster. Notably, reads are directed to the primary node due to the application’s requirement for strong consistency. Following this observation in the production environment, we attempted to replicate the scenario in our staging environment using reduced hardware resources. Surprisingly, we were able to reproduce similar CPU load spikes with fewer resources and operations compared to what was initially highlighted.
The screenshot below shows the distinct pattern of the CPU load average spiking precisely every 20 minutes, rising from nearly 0 to 2-3. At the same time, the CPU usage remains consistently stable without any spikes, contrasting the fluctuations in the CPU load graph and signifying a steady workload.
What can be the reason for such spikes? Is WiredTiger doing some regular cleanup activity? Or is it something to do with GCP? Or Mongo itself?
We are currently unable to identify the root cause of this issue. Any assistance you could provide on this issue would be greatly appreciated!