Failed to start MongoDB Database Server [CentOS7]

Sufian_Ahmed · 2024-05-05T14:01:25.701Z

Hi,

Suddenly my VPS running with CentOS7 & CWP7 Panel giving “Failed to start MongoDB Database Server” error

Server Details/Specs:

Distro Name: CentOS Linux release 7.9.2009 (Core)
Kernel Version: 3.10.0-1160.88.1.el7.x86_64
Platform: x86_64 kvm
mongod --version
db version v4.2.25
git version: 41b59c2bfb5121e66f18cc3ef40055a1b5fb6c2e
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
allocator: tcmalloc
modules: none
build environment:
    distmod: rhel70
    distarch: x86_64
    target_arch: x86_64
	
CWP version: 0.9.8.1177
PHP version: 7.0.33 
MySQL version: 10.4.33-MariaDB
CPU Model: DO-Premium-Intel
CPU Details: 1 Core (2494 MHz)

Logs: available at PrivateBin

# systemctl status mongod.service
● mongod.service - MongoDB Database Server
   Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sun 2024-05-05 07:30:30 UTC; 2min 9s ago
     Docs: https://docs.mongodb.org/manual
  Process: 4247 ExecStart=/usr/bin/mongod $OPTIONS (code=exited, status=100)
  Process: 4245 ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb (code=exited, status=0/SUCCESS)
  Process: 4243 ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS)
  Process: 4242 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS)

May 05 07:30:30 centos-7-examamine systemd[1]: Starting MongoDB Database Server...
May 05 07:30:30 centos-7-examamine mongod[4247]: about to fork child process, waiting until server is ready for connections.
May 05 07:30:30 centos-7-examamine mongod[4247]: forked process: 4250
May 05 07:30:30 centos-7-examamine systemd[1]: mongod.service: control process exited, code=exited status=100
May 05 07:30:30 centos-7-examamine systemd[1]: Failed to start MongoDB Database Server.
May 05 07:30:30 centos-7-examamine systemd[1]: Unit mongod.service entered failed state.
May 05 07:30:30 centos-7-examamine systemd[1]: mongod.service failed.
[root@centos-7-examamine ~]# cd /var/log/mongodb
[root@centos-7-examamine mongodb]# ls
mongod.log
[root@centos-7-examamine mongodb]# cat mongod.log
2024-05-03T13:00:06.746+0000 I  CONTROL  [main] ***** SERVER RESTARTED *****
2024-05-03T13:00:06.756+0000 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2024-05-03T13:00:06.839+0000 W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
2024-05-03T13:00:06.839+0000 I  CONTROL  [initandlisten] MongoDB starting : pid=2914 port=27017 dbpath=/var/lib/mongo 64-bit host=centos-7-examamine
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten] db version v4.2.25
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten] git version: 41b59c2bfb5121e66f18cc3ef40055a1b5fb6c2e
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten] allocator: tcmalloc
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten] modules: none
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten] build environment:
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten]     distmod: rhel70
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten]     distarch: x86_64
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten]     target_arch: x86_64
2024-05-03T13:00:06.840+0000 I  CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid", timeZoneInfo: "/usr/share/zoneinfo" }, storage: { dbPath: "/var/lib/mongo", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2024-05-03T13:00:06.840+0000 E  NETWORK  [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted
2024-05-03T13:00:06.840+0000 F  -        [initandlisten] Fatal Assertion 40486 at src/mongo/transport/transport_layer_asio.cpp 684
2024-05-03T13:00:06.840+0000 F  -        [initandlisten] \n\n***aborting after fassert() failure\n\n

I will really appreciate if you please guide me i=on how the persisted issue could be resolved?

Regards,

Fabio_Ramohitaj · 2024-05-05T20:24:03.704Z

Hi @Sufian_Ahmed and welcome to the community!
here is your error:

Sufian_Ahmed:

Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted

So you simply need to fix the permissions to that file.
Regards

Yilmaz_Durmaz · 2024-05-06T22:54:19.079Z

this file is the issue but this is a bit more than a permission issue.

the server would normally delete this file after mongodb shuts down, but socket files tend to stuck sometimes. delete that socket file and let mongodb create a new one on its next start up
there might still be another mongodb instance using the port 27017. it might be a lingering instance that kept running after some program crash. if this is the case, you cannot remove socket file. check processes and kill that instance first.
I hope this is not the case: the disk might be getting corrupted and this socket file might have been written in one of those areas. you may need a disk check.