Hello,
For a consequent amount of time, i have been struggling to run a simple timeseries database.
I have a 3 replica set by 3 replica cluster with a sharded timeseries collection with 2 mongos routers. This collection receives around 1000 messages / second. The timeseries use the default sharding index (metadata + timestamp).
The issue I am facing is an unstable and low ingestion rate.
When the collection is empty, the rate is “normal” (can go up to 2000 messages / sec), but after a few minutes, the rate goes down finally to stay around 100 messages / sec.
It is fed by a Kafka sink connector for mongoDB
Here are the specs and metrics:
- The cluster is running in docker (on a single server)
- 48 CPU, 128Gb RAM, 3T SSD
The ingestion rate profile I have look like :
The data production starts at the middle, then the production is stable and finally, i get totally unstable.
The mongoDB settings are set to defaults.
My question is : Is there any mongoDB mechanism that may be the root cause of this behavior?