Hello,
We have a MongoDB database with a few 100 collections, all of which have a TTL index. A total of around 100 documents per second are inserted into these collections, whereby the data rate per collection is not equally distributed across all collections.
We can now see in the logs that the TTL monitor removes expired data from around 15 to 20 collections per hour.
I know that the TTL process should actually run every 60 seconds. However, the TTLMonitor info logs do not reflect this. In addition, the time of 60 seconds would probably not be enough to process all collections. Deleting data in a collection usually takes a few 100ms according to the log, but in extreme cases it can take several seconds.
We now observe that TTL activity increases sharply at a very specific time of day (18:00 UTC). Almost 90 collections are processed within 6 minutes (I currently have no log that goes beyond these 6 minutes). This obviously leads to free (RAM) memory in the end. After that, the memory consumption increases again until the next day at 18:00.
My questions now are
- Is the TTLMonitor event logged for every collection, or only for the collections where deletions occurred (or possibly only for the collections where the deletion took a long time)?
- Why does the system behave completely differently once a day? The input data rate is actually the same throughout the day, so there should be no different priorities between input/delete?
Is there an explanation on the MongoDB side why the TTL monitor strikes once a day and clearly gains presence?
Or do I have to look for special evens in the environment - if so, which ones?
If you know anything, I would be grateful for any tips.
Best regards
Franz