Experiencing issues with parallel mongosh executions

As part of ETL process our shell scripts are using mongosh to execute queries and retrieve data from MongoDB. When multiple processes are attempting to use mongosh in parallel, we randomly observe Error: ENOENT: no such file or directory, lstat '~/.mongodb/mongosh/xxxxxxxxxxxxx_log' and process gets terminated.

It appears mongosh can only store 100 log files in ~/.mongodb/mongosh directory and the rest gets purged. When mongosh attempts to reuse an old / existing session_id (which I assume is controlled by Mongo server) and if log file with that session_id is already purged, it throws an error. We couldn’t find a way to either disable logs generated by mongosh or increase the number of log files mongosh can hold.

Has anyone experienced a similar situation? If there is any workaround or an alternate solution to try out, please do advise.

Thanks.

Hi Gaurab,

Thanks for your response.

We had parallelized certain processes so we can complete our entire pipeline execution in a small window. We haven’t experienced this issue when using mongo cli. It appears we have to wait until new version of mongosh is released.

Even if the log file count / storage limit is made configurable, would we still have challenges in determining the exact configuration value… as mongosh attempts to reuse existing session_id for a period of time and the respective log file getting purged in background?

Is session_id created from client side or created & managed in server? If there is a documentation explaining the process of session_id management, please do share for reference.

Thanks.