allowDiskUse: true not working for serverless instance

Sohel_Kabirq · 2024-07-16T17:22:08.256Z

I’m trying to this aggregation for a collection but getting memory limit error, then i used allowDiskUse: true . But it’s not working.

My aggreagation:

[
  {
    "$match": {
      "createdAt": {
        "$gte": ISODate("2024-05-15T14:42:55Z"),
        "$lte": ISODate("2024-07-15T14:42:55Z")
      }
    }
  },
  {
    "$group": {
      "_id": "$keyword",
      "count": { "$sum": 1 },
      "unique_ips": { "$addToSet": "$remoteHost" }
    }
  },
  {
    "$project": {
      "count": 1,
      "unique_ip_count": { "$size": "$unique_ips" }
    }
  },
  {
    "$sort": { "count": -1 }
  },
  {
    "$limit": 20
  }
]

Errror: 
 PlanExecutor error during aggregation :: caused by :: Exceeded memory limit for $group, but didn't allow external spilling; pass allowDiskUse:true to opt in, full

Anurag_Kadasne · 2024-07-16T17:25:04.608Z

Hi there

Please see the limitations page of MongoDB Atlas Serverless. Serverless instances don’t support the allowDiskUse option.

Sohel_Kabirq · 2024-07-17T15:40:44.671Z

Hey thanks for the reply, but how can i run same aggregation without allowDiskUse option? Any workarounds? Thank you.

Vivekanandan_Sakthivelu · 2024-07-18T06:50:39.506Z

Sohel_Kabirq:

allowDiskUse

@Sohel_Kabirq , Since your instance is serverless and you dont have option to specify the allowDiskFlag . Try to avoid the querying large set of document that exceeds your client memory .

In the mentioned example , $match stage spans to 2 days . Try to reduce it or see if you can think other way around . A set of NoSQL patterns can help you in place .Looking at your query , it seems bucket pattern is a close one that suits your need

See Building with Patterns: The Bucket Pattern | MongoDB Blog

Other set of pattern can be found here , Building with Patterns: A Summary | MongoDB Blog

Its not always the case the you have to query the large set of collection for your need , you can avoid running into this kind of situation by planning your data model ahead .

Justin_LeDuc · 2025-01-28T08:28:59.482Z

hi, i’m assuming even with allowDiskUseByDefault that serverless instances still won’t allow stages that exceed the 100mb limit?

Anurag_Kadasne · 2025-01-28T16:17:28.518Z

Hi Justin,

Please see my message above

" Please see the limitations page of MongoDB Atlas Serverless . Serverless instances don’t support the allowDiskUse option."