6 / 6
Jan 28

I’m trying to this aggregation for a collection but getting memory limit error, then i used allowDiskUse: true . But it’s not working.

My aggreagation:

[ { "$match": { "createdAt": { "$gte": ISODate("2024-05-15T14:42:55Z"), "$lte": ISODate("2024-07-15T14:42:55Z") } } }, { "$group": { "_id": "$keyword", "count": { "$sum": 1 }, "unique_ips": { "$addToSet": "$remoteHost" } } }, { "$project": { "count": 1, "unique_ip_count": { "$size": "$unique_ips" } } }, { "$sort": { "count": -1 } }, { "$limit": 20 } ]
Errror: PlanExecutor error during aggregation :: caused by :: Exceeded memory limit for $group, but didn't allow external spilling; pass allowDiskUse:true to opt in, full

Hey thanks for the reply, but how can i run same aggregation without allowDiskUse option? Any workarounds? Thank you.

@Sohel_Kabirq , Since your instance is serverless and you dont have option to specify the allowDiskFlag . Try to avoid the querying large set of document that exceeds your client memory .

In the mentioned example , $match stage spans to 2 days . Try to reduce it or see if you can think other way around . A set of NoSQL patterns can help you in place .Looking at your query , it seems bucket pattern is a close one that suits your need

See Building with Patterns: The Bucket Pattern | MongoDB Blog

Other set of pattern can be found here , Building with Patterns: A Summary | MongoDB Blog

Its not always the case the you have to query the large set of collection for your need , you can avoid running into this kind of situation by planning your data model ahead .

6 months later

hi, i’m assuming even with allowDiskUseByDefault that serverless instances still won’t allow stages that exceed the 100mb limit?