MongoDB Atlas Search - slow count of matched documents

Pedro_Cristina · 2020-09-15T12:32:12.661Z

Hello, I have a collection with 11M records, which contains among other properties, a “Name” property with a client name:

{
    "Name" : "Teressa Bella"
}

I have created an Atlas Search index on the “Name” field with the following definition:

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "Name": {
        "analyzer": "lucene.standard",
        "type": "string"
      }
    }
  }
}

My use case counting all the documents that match a certain term which I’m doing like so:

db.getCollection('recipients').aggregate([
  {
    $search: {
      "text": {
        "query": "Teressa",
        "path": "Name"
      }
    }
  },
  {
    $count: "total"
  }
])

The problem is that this query takes about 40s to complete on an M30 Atlas cluster (sometimes 10s or 20s if entirely fetched from memory, I presume).
Is this within the expected execution time for matching a term on 11M documents?
I would appreciate any suggestions on how i can optimize this kind of query.

steevej · 2020-09-15T12:43:59.115Z

I would try to project just the Name field. Since it is part of the index it is likely that the documents do no need be read from permanent storage.

This being said, if your use-case for using the Atlas Search is as simple as this, a normal index on the Name field might be a better fit.

Pedro_Cristina · 2020-09-15T18:26:58.744Z

Thanks for the tip, unfortunately it didn’t make a difference, probably because I’m only performing a count on the records.
My real use case will actually include more complex operations, like partial text searches. However, as I performed this simple test, i found the performance to be worse than I was expecting, so I was looking to know if there’s something I’m missing.

Jonathan_Gautier · 2020-09-26T17:22:19.847Z

I got same problem !

But i have 500k documents and M40.

For same search as you, searching time was between 40s-50s…

Any help from Mongo Employee ?

Marcus · 2020-09-29T06:51:35.287Z

@Jonathan_Gautier we should set up some time to chat. I want to help you and feel your pain.

Same with you @Pedro_Cristina. I’m the new Product Manager on Atlas Search and I want to make this better for you both.

Jonathan_Gautier · 2020-09-29T09:06:05.310Z

@Marcus

Hi how can we help you to fix this issue ?

I have try to change my cluster tier from m30 to m50 but no change for search delay.
Guys from developper support said i need to change my cluster tier because m30 was overload but with m50 i got same result and it was not overload when my query was running.

image1631×396 28.7 KB

Jonathan_Gautier · 2020-10-04T21:26:24.500Z

@Marcus

New test today:

Search index size was : 4,84MB
Pipeline:

[{"$search":{"range":{"gte":0,"path":"status"}}},{"$count":"total"}]

496 688 documents match from query.
And take 27sec.

Same query with $project ( Fields “status” not indexed )

[{$project: {"status": { "$gte": ["$status", 0] }}},{"$count":"total"}]

Take 700ms

I have made many tests and every time “large match from query” with search was made, this take several time same for simplest query.

Is not normal i think but i dont know how it’s work behind

If i can made some tests for you dont hesitate

Tejasvi_Penumarthi · 2021-04-08T08:14:38.191Z

Is it possible to reduce response time by using $search pipeline?
or I am using wildcard in $search, is it possible to use similar to wildcard in $project?

Marcus · 2021-04-08T16:07:05.631Z

eventually. It’s being built now.

piotr · 2021-06-21T17:19:42.948Z

Hi Marcus, any updates?

Marcus · 2021-06-25T01:05:25.142Z

This work has been started and it is coming in the next month or two. It will be lightning fast and it will also be usable in MongoDB Charts. If you’d like to be updated when this feature is released you can vote for it here

Augusto_Ochoa · 2025-04-15T12:02:38.235Z

Hello @Pedro_Cristina,

The problem you had at that time can be solved by using $searchMeta, which retrieves the count you needed back then.

You should definitely give Atlas Search another try.

Resultados da Atlas Search Search - Atlas - Documentos MongoDB