Does Mongo cache the whole index or just the parts that it needs to service a Mongo Query?

John_Cantu · 2024-06-25T21:33:20.923Z

If I have an index that is sized at 1 GB and have the WiredTiger cache set to 100 GB. Then, I perform that very first query that uses the 1GB index, but the query only returns back one row. Will Mongo store the whole 1 GB index in cache or just the parts that it needed to service that query?

Thank you,

John

chris_wright · 2024-06-26T12:05:40.739Z

Hello, @John_Cantu

MongoDB’s WiredTiger cache will store only the parts of the 1 GB index that are needed to service the query, not the entire index.

I hope this will be help you
chris1058

Gregory_Chavez · 2024-06-26T12:05:53.910Z

John_Cantu:

If I have an index that is sized at 1 GB and have the WiredTiger cache set to 100 GB. Then, I perform that very first query that uses the 1GB index, but the query only returns back one row. Will Mongo store the whole 1 GB index in cache or just the parts that it needed to service that query?

Thank you,

John

Hello, @John_Cantu

When using the WiredTiger storage engine in MongoDB, the cache behavior is interesting. Here’s how it works:

Cache Allocation:
WiredTiger doesn’t reserve separate portions of the cache for reads and writes.
Instead, it allocates its cache to the entire mongod instance.
There’s no per-database or per-collection cache allocation.
Index Caching:
When you perform a query, WiredTiger prioritizes index caching.
So, even if your query only returns one row, it will still load the relevant index pages into the cache.
However, it won’t necessarily load the entire 1 GB index; it’ll focus on the parts needed for the query.
Heavy Write Workload:
If there’s a heavy write workload, it can affect performance.
But WiredTiger still tries to keep index pages in cache for efficient querying.

I hope this info is helpful to you.

Best Regard,
Gregory Chavez

steevej · 2024-06-26T12:34:45.081Z

@chris_wright and @Gregory_Chavez, Could you please provide documentation that confirms your assertion?

Gregory_Chavez · 2024-06-27T10:33:36.494Z

Hello, @steevej

Here’s the requested documentation that confirms the behavior of the WiredTiger storage engine in MongoDB:

WiredTiger Storage Engine - MongoDB Manual:
The MongoDB Manual provides detailed information about the WiredTiger storage engine.
It covers topics such as cache allocation, index caching, and heavy write workloads.
You can find more details in the WiredTiger Storage Engine section of the MongoDB Manual here: https://www.mongodb.com/docs/manual/core/wiredtiger/.

I hope this is helpful to you.

Best Regard,
mysedgwick walmart

steevej · 2024-06-27T12:57:24.971Z

Thanks for the documentation link.

Shame for the spam link.

John_Cantu · 2024-06-27T15:40:33.638Z

Thanks for your responses Chris, Gregory and Steeve.

I didn’t think it would be the best thing for Mongo to cache the entire index, but I needed to ask. The reason I asked this question was based on the following recommendation:

Mongo recommends to “Ensure Indexes Fit in RAM”
https://www.mongodb.com/docs/manual/tutorial/ensure-indexes-fit-ram/#std-label-indexes-ensure-indexes-fit-ram

If only parts of the index are used, then I don’t think it is necessary to have enough RAM to hold the entire index just the index pages frequently used.

Also, I am glad to hear that Mongo will will prioritize keeping index pages cached.

John