I merged all my collections into one and the CPU got super high

fberrez · 2024-08-13T16:03:10.372Z

I had my data model like this, where each - is a collection name:

${datatype-1}-${client-a}
${datatype-2}-${client-a}
${datatype-1}-${client-b}
${datatype-2}-${client-b}
…

Each collection has a maximum storage size of 40mb.

I wanted to merge these collections in one because I realised that the “duplicate” indexes has a real weight on my cluster memory.
So the new collection has a data model like:

{
  "category":${datatype},
  "contract":${client},
  ...
}

And since I merge all my documents (the new collection has a storage size of 960mb, 4.1m docs) the CPU is at 100%. I checked in the query profiler on mongo atlas and all queries I execute in this new collection have super high response time while they all use indexes (IXSCAN).

Do you have any idea why?

Gurjit_Butalia · 2024-09-09T09:57:55.526Z

Hello,

When you merged the collections into one collection, I am assuming you added the category and contract to all your existing indexes and you would have modified the code to include the category and contract in the where clause…

Thanks