Constantly high CPU usage after OS restart

Thomas_Marksteienr · 2025-01-23T20:28:45.940Z

We have used a mongo v8.0.1 replicaSet (2 nodes + arbiter) on our production servers (Rocky 9.4 with 12 cores) since some weeks. I restartet the whole system of primary after RAM upgrade some days ago.

We have seen 100% cpu usage caused by mongo process on all cores since that.
We have tested folllowing steps:

mongo service restart
look for slow queries (We also tried to stop all our services calling mongo, but cpu usage won’t change.)
node reinitilized
switch primary to another node
disable transparent huge page on os level

According to mongo, the connection count of node slowly increases overtime. The result of
following statement:

db.currentOp({“secs_running”: {$gte: 3}})

contains a lot of hello commands.

Does anyone has an idea?
Any help would be greatly appreciated

regards

chris · 2025-01-24T01:12:08.969Z

Hi @Thomas_Marksteienr

Check out:
https://jira.mongodb.org/browse/SERVER-92749

Downgrading openssl to 3.0 rectifies the issue, per the latest comment it is addressed in an upcoming release:

About the fix - we’ve actually got it working in our master branch and have backported it to 8.0 (it’ll be in the upcoming 8.0.5 release). We’re still working on bringing it back to 7.0 - that should take a few more weeks.

Thomas_Marksteienr · 2025-01-27T09:50:27.482Z

hi,
thank you for your answer.

It seems to be our problem. If we disabled tls, there was no problem.

Unfortunately, downgrading openssl is no option for us. Is there any estimation of release date of 8.0.5?

Thomas_Marksteienr · 2025-01-28T13:55:36.224Z

after further investigations, we found a way to only change openssl version for mongodb:

cd /tmp
wget http://d.rockylinux.org/vault/rocky/9.4/BaseOS/x86_64/os/Packages/o/openssl-3.0.7-28.el9_4.x86_64.rpm
wget http://d.rockylinux.org/vault/rocky/9.4/BaseOS/x86_64/os/Packages/o/openssl-libs-3.0.7-28.el9_4.x86_64.rpm
mkdir /opt/openssl-3.0.7-28
cd /opt/openssl-3.0.7-28

rpm2cpio /tmp/openssl-3.0.7-28.el9_4.x86_64.rpm | cpio -di
rpm2cpio /tmp/openssl-libs-3.0.7-28.el9_4.x86_64.rpm | cpio -di

set new path in systemd file:

Environment="LD_LIBRARY_PATH=/opt/openssl-3.0.7-28/usr/lib64"

Although the origin problem is gone, we would appreciate a bug fix.

regards

chris · 2025-02-25T15:13:16.596Z

@Thomas_Marksteienr 8.0.5 is available for download now.