My mongodb server failing to start/restart

karan_sharma3 · June 28, 2023, 12:39am

I’m using a ubuntu VPS only for MongoDB.
I recently installed mongodb from official site.
I changed few configs like adding security etc… and it was working fine.

Few days later, today when I made a minor change in config and restarted the service. It suddenly stopped working and even after reverting the change, its still the same.
DB contains some data that is very previous to me, so i can’t just start over

Here’s mongoDB status:

# service mongod status
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2023-06-28 00:27:41 UTC; 2s ago
       Docs: https://docs.mongodb.org/manual
    Process: 302981 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=1/FAILURE)
   Main PID: 302981 (code=exited, status=1/FAILURE)

Jun 28 00:27:41 creazilla-mongo systemd[1]: Started MongoDB Database Server.
Jun 28 00:27:41 creazilla-mongo mongod[302981]: about to fork child process, waiting until server is ready for connections.
Jun 28 00:27:41 creazilla-mongo mongod[302996]: forked process: 302996
Jun 28 00:27:41 creazilla-mongo mongod[302981]: ERROR: child process failed, exited with 1
Jun 28 00:27:41 creazilla-mongo mongod[302981]: To see additional information in this output, start without the "--fork" option.
Jun 28 00:27:41 creazilla-mongo systemd[1]: mongod.service: Main process exited, code=exited, status=1/FAILURE
Jun 28 00:27:41 creazilla-mongo systemd[1]: mongod.service: Failed with result 'exit-code'.

My config file:

#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
#  engine:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/lib/mongodb/mongod.log

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1
  bindIpAll: true

# how the process runs
processManagement:
  fork: true
  timeZoneInfo: /usr/share/zoneinfo

security:
  authorization: enabled

#operationProfiling:

#replication:

#sharding:

## Enterprise-Only Options:

#auditLog:

#snmp:

Let me know if anything else is needed.

Jack_Woehr · June 28, 2023, 1:29am

Did you trying doing this, starting mongod manually? As shown earlier in the output, the command is:

/usr/bin/mongod --config /etc/mongod.conf

and set

fork: false

in the config

chris · June 28, 2023, 1:33am

Do this as mongodb though sudo -u mongodb /usr/bin/mongod --config /etc/mongod.conf

Also see https://jira.mongodb.org/browse/SERVER-74345 for which the easy resolution is to upgrade to the latest patch release. Or remove fork: true.

karan_sharma3 · June 28, 2023, 4:08am

Hello, thank you for responding. Its very kind of you and I appreciate it.
So here’s what i did based on your response:

ran command:

/usr/bin/mongod --config /etc/mongod.conf

output:

root@creazilla-mongo:~# /usr/bin/mongod --config /etc/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 304905
ERROR: child process failed, exited with 14
To see additional information in this output, start without the "--fork" option.

Clearly, as you stated I need to turn fork off, so after doing that. I re-ran the command.
This time i got no output actually.

So I tried:

systemctl status mongod

I think I restarted the mongodb service as well.

output:

● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2023-06-28 04:02:11 UTC; 2s ago
       Docs: https://docs.mongodb.org/manual
    Process: 304930 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=1/FAILURE)
   Main PID: 304930 (code=exited, status=1/FAILURE)

Jun 28 04:02:11 creazilla-mongo systemd[1]: Started MongoDB Database Server.
Jun 28 04:02:11 creazilla-mongo mongod[304930]: {"t":{"$date":"2023-06-28T04:02:11.453Z"},"s":"F",  "c":"CONTROL",  "id":20574,   "ctx":"-","msg":"Error during global initialization","attr":{"error":{"code":38,"codeName":"FileNotOpen","errmsg":"Can't initialize rotatable log file :: caused by :: Failed to open /var/lib/mongodb/mongod.log"}}}
Jun 28 04:02:11 creazilla-mongo systemd[1]: mongod.service: Main process exited, code=exited, status=1/FAILURE
Jun 28 04:02:11 creazilla-mongo systemd[1]: mongod.service: Failed with result 'exit-code'.

karan_sharma3 · June 28, 2023, 4:25am

Hi, thank you for responding.
Here’s output to command you suggested:

# sudo -u mongodb /usr/bin/mongod --config /etc/mongod.conf
{"t":{"$date":"2023-06-28T04:23:46.785Z"},"s":"F",  "c":"CONTROL",  "id":20574,   "ctx":"-","msg":"Error during global initialization","attr":{"error":{"code":38,"codeName":"FileNotOpen","errmsg":"Can't initialize rotatable log file :: caused by :: Failed to open /var/lib/mongodb/mongod.log"}}}

It seems that the issue is with log file permission, even in my previous research on pursuit of solving this issue. I found a thread where people were discussing exactly this.

Jack_Woehr · June 28, 2023, 4:50am

karan_sharma3:

Jun 28 04:02:11 creazilla-mongo mongod[304930]: {"t":{"$date":"2023-06-28T04:02:11.453Z"},"s":"F",  "c":"CONTROL",  "id":20574,   "ctx":"-","msg":"Error during global initialization","attr":{"error":{"code":38,"codeName":"FileNotOpen","errmsg":"Can't initialize rotatable log file :: caused by :: Failed to open /var/lib/mongodb/mongod.log"}}}

Why can’t mongod open /var/lib/mongodb/mongod.log

Is that where the logfile should go? Or maybe /var/log/mongodb/mongod.log ? Is there a typo in the config?

If so, then if you figure out why it’s not opening the logfile, which seems to be the error which is making mongod quit, then you’ve solved your problem.

karan_sharma3 · June 28, 2023, 4:57am

Seems like it, I might have made that changed while playing around with values.

Now I’ve updated the value with one you provided and restarted the server. service status looks like this:

# systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2023-06-28 04:54:12 UTC; 8s ago
       Docs: https://docs.mongodb.org/manual
    Process: 305122 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=14)
   Main PID: 305122 (code=exited, status=14)

Jun 28 04:54:11 systemd[1]: Started MongoDB Database Server.
Jun 28 04:54:12  systemd[1]: mongod.service: Main process exited, code=exited, status=14/n/a
Jun 28 04:54:12  systemd[1]: mongod.service: Failed with result 'exit-code'.

Ramachandra_Tummala · June 28, 2023, 8:37am

Does mongod have write permissions on the new logpath directory you have given?

chris · June 28, 2023, 12:23pm

Check the log now.

If you ran mongod as root earlier then some files may have been created with root permissions (why I said to run it with sudo -u mongodb)

You may have permissions that need to be reset on the data/log dirs and mongo socket.

chown -R mongodb:mongodb /var/lib/mongodb
chown -R mongodb:mongodb /var/log/mongodb
chown mongodb:mongodb /tmp/mongo*.sock

karan_sharma3 · June 28, 2023, 3:17pm

Here’s updated config:

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
#  engine:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1
  bindIpAll: true

# how the process runs
processManagement:
  timeZoneInfo: /usr/share/zoneinfo

security:
  authorization: enabled

Status:

# systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2023-06-28 15:15:41 UTC; 1min 16s ago
       Docs: https://docs.mongodb.org/manual
    Process: 310905 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=14)
   Main PID: 310905 (code=exited, status=14)

Jun 28 15:15:40  systemd[1]: Started MongoDB Database Server.
Jun 28 15:15:41 systemd[1]: mongod.service: Main process exited, code=exited, status=14/n/a
Jun 28 15:15:41 systemd[1]: mongod.service: Failed with result 'exit-code'.

Fabio_Ramohitaj · June 28, 2023, 7:09pm

Hi @karan_sharma3,
Which is the user created from installation of MongoDB?
Is mongod or mongodb? Please, cat & paste the output of this file:

cat /etc/passwd

There is a file named mongodb-27017.lock?
It should be place in /tmp or you can find It with the follow command:

find / -iname mongodb-27017.lock

When you have find It, check the permission of it.
If the permission of user and group are different from mongod change the permission in:

chown mongod?:mongod? /tmp/mongo-27017.lock

After all restart the service.

Regards

chris · June 28, 2023, 8:22pm

The next thing you need to do is look in the log file. /var/log/mongodb/mongod.log

karan_sharma3 · June 28, 2023, 9:07pm

Last 200 lines of log file:

{"t":{"$date":"2023-06-27T23:58:44.770+00:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"-","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2023-06-27T23:58:44.776+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"-","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2023-06-27T23:58:44.778+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}}
{"t":{"$date":"2023-06-27T23:58:44.779+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2023-06-27T23:58:44.793+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}}
{"t":{"$date":"2023-06-27T23:58:44.793+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}
{"t":{"$date":"2023-06-27T23:58:44.793+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}}
{"t":{"$date":"2023-06-27T23:58:44.793+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2023-06-27T23:58:44.794+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":302287,"port":27018,"dbPath":"/var/lib/mongodb","architecture":"64-bit","host":"creazilla-mongo"}}
{"t":{"$date":"2023-06-27T23:58:44.794+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.6","gitVersion":"26b4851a412cc8b9b4a18cdb6cd0f9f642e06aa7","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2023-06-27T23:58:44.794+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2023-06-27T23:58:44.794+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/etc/mongod.conf","net":{"bindIp":"*","port":27018},"processManagement":{"fork":true,"timeZoneInfo":"/usr/share/zoneinfo"},"security":{"authorization":"enabled"},"storage":{"dbPath":"/var/lib/mongodb"},"systemLog":{"destination":"file","logAppend":true,"path":"/var/log/mongodb/mongod.log"}}}}
{"t":{"$date":"2023-06-27T23:58:44.795+00:00"},"s":"E",  "c":"NETWORK",  "id":23024,   "ctx":"initandlisten","msg":"Failed to unlink socket file","attr":{"path":"/tmp/mongodb-27018.sock","error":"Operation not permitted"}}
{"t":{"$date":"2023-06-27T23:58:44.795+00:00"},"s":"F",  "c":"ASSERT",   "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":40486,"file":"src/mongo/transport/transport_layer_asio.cpp","line":1126}}
{"t":{"$date":"2023-06-27T23:58:44.795+00:00"},"s":"F",  "c":"ASSERT",   "id":23092,   "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}

{"t":{"$date":"2023-06-28T04:54:12.045+00:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"-","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2023-06-28T04:54:12.048+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2023-06-28T04:54:12.048+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}}
{"t":{"$date":"2023-06-28T04:54:12.048+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2023-06-28T04:54:12.060+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}}
{"t":{"$date":"2023-06-28T04:54:12.060+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}
{"t":{"$date":"2023-06-28T04:54:12.060+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}}
{"t":{"$date":"2023-06-28T04:54:12.060+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2023-06-28T04:54:12.060+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":305122,"port":27017,"dbPath":"/var/lib/mongodb","architecture":"64-bit","host":"creazilla-mongo"}}
{"t":{"$date":"2023-06-28T04:54:12.061+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.6","gitVersion":"26b4851a412cc8b9b4a18cdb6cd0f9f642e06aa7","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2023-06-28T04:54:12.061+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2023-06-28T04:54:12.061+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/etc/mongod.conf","net":{"bindIp":"*","port":27017},"processManagement":{"timeZoneInfo":"/usr/share/zoneinfo"},"security":{"authorization":"enabled"},"storage":{"dbPath":"/var/lib/mongodb"},"systemLog":{"destination":"file","logAppend":true,"path":"/var/log/mongodb/mongod.log"}}}}
{"t":{"$date":"2023-06-28T04:54:12.062+00:00"},"s":"E",  "c":"NETWORK",  "id":23024,   "ctx":"initandlisten","msg":"Failed to unlink socket file","attr":{"path":"/tmp/mongodb-27017.sock","error":"Operation not permitted"}}
{"t":{"$date":"2023-06-28T04:54:12.062+00:00"},"s":"F",  "c":"ASSERT",   "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":40486,"file":"src/mongo/transport/transport_layer_asio.cpp","line":1126}}
{"t":{"$date":"2023-06-28T04:54:12.062+00:00"},"s":"F",  "c":"ASSERT",   "id":23092,   "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}

{"t":{"$date":"2023-06-28T04:56:30.783+00:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"-","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2023-06-28T04:56:30.784+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}}
{"t":{"$date":"2023-06-28T04:56:30.786+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2023-06-28T04:56:30.787+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2023-06-28T04:56:30.800+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}}
{"t":{"$date":"2023-06-28T04:56:30.800+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}
{"t":{"$date":"2023-06-28T04:56:30.800+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}}
{"t":{"$date":"2023-06-28T04:56:30.800+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2023-06-28T04:56:30.800+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":305155,"port":27017,"dbPath":"/var/lib/mongodb","architecture":"64-bit","host":"creazilla-mongo"}}
{"t":{"$date":"2023-06-28T04:56:30.800+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.6","gitVersion":"26b4851a412cc8b9b4a18cdb6cd0f9f642e06aa7","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2023-06-28T04:56:30.800+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2023-06-28T04:56:30.800+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/etc/mongod.conf","net":{"bindIp":"*","port":27017},"processManagement":{"timeZoneInfo":"/usr/share/zoneinfo"},"security":{"authorization":"enabled"},"storage":{"dbPath":"/var/lib/mongodb"},"systemLog":{"destination":"file","logAppend":true,"path":"/var/log/mongodb/mongod.log"}}}}
{"t":{"$date":"2023-06-28T04:56:30.801+00:00"},"s":"I",  "c":"STORAGE",  "id":22270,   "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/var/lib/mongodb","storageEngine":"wiredTiger"}}
{"t":{"$date":"2023-06-28T04:56:30.801+00:00"},"s":"I",  "c":"STORAGE",  "id":22297,   "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
{"t":{"$date":"2023-06-28T04:56:30.802+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=7303M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],"}}
{"t":{"$date":"2023-06-28T04:56:31.385+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":11,"message":"[1687928191:385730][305155:0x7efe804abcc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __posix_file_lock, 393: /var/lib/mongodb/WiredTiger.lock: handle-lock: fcntl: Resource temporarily unavailable"}}
{"t":{"$date":"2023-06-28T04:56:31.386+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":16,"message":"[1687928191:386758][305155:0x7efe804abcc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __conn_single, 1806: WiredTiger database is already being managed by another process: Device or resource busy"}}
{"t":{"$date":"2023-06-28T04:56:31.387+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":11,"message":"[1687928191:387072][305155:0x7efe804abcc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __posix_file_lock, 393: /var/lib/mongodb/WiredTiger.lock: handle-lock: fcntl: Resource temporarily unavailable"}}
{"t":{"$date":"2023-06-28T04:56:31.387+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":16,"message":"[1687928191:387141][305155:0x7efe804abcc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __conn_single, 1806: WiredTiger database is already being managed by another process: Device or resource busy"}}
{"t":{"$date":"2023-06-28T04:56:31.387+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":11,"message":"[1687928191:387352][305155:0x7efe804abcc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __posix_file_lock, 393: /var/lib/mongodb/WiredTiger.lock: handle-lock: fcntl: Resource temporarily unavailable"}}
{"t":{"$date":"2023-06-28T04:56:31.387+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":16,"message":"[1687928191:387412][305155:0x7efe804abcc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __conn_single, 1806: WiredTiger database is already being managed by another process: Device or resource busy"}}
{"t":{"$date":"2023-06-28T04:56:31.387+00:00"},"s":"W",  "c":"STORAGE",  "id":22347,   "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
{"t":{"$date":"2023-06-28T04:56:31.387+00:00"},"s":"F",  "c":"STORAGE",  "id":28595,   "ctx":"initandlisten","msg":"Terminating.","attr":{"reason":"16: Device or resource busy"}}
{"t":{"$date":"2023-06-28T04:56:31.387+00:00"},"s":"F",  "c":"ASSERT",   "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":28595,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":708}}
{"t":{"$date":"2023-06-28T04:56:31.387+00:00"},"s":"F",  "c":"ASSERT",   "id":23092,   "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}

{"t":{"$date":"2023-06-28T04:58:42.411+00:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"-","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2023-06-28T04:58:42.411+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"-","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2023-06-28T04:58:42.412+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}}
{"t":{"$date":"2023-06-28T04:58:42.413+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2023-06-28T04:58:42.424+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}}
{"t":{"$date":"2023-06-28T04:58:42.424+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}
{"t":{"$date":"2023-06-28T04:58:42.424+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}}
{"t":{"$date":"2023-06-28T04:58:42.424+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2023-06-28T04:58:42.426+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":305177,"port":27017,"dbPath":"/var/lib/mongodb","architecture":"64-bit","host":"creazilla-mongo"}}
{"t":{"$date":"2023-06-28T04:58:42.426+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.6","gitVersion":"26b4851a412cc8b9b4a18cdb6cd0f9f642e06aa7","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2023-06-28T04:58:42.426+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2023-06-28T04:58:42.426+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/etc/mongod.conf","net":{"bindIp":"*","port":27017},"processManagement":{"timeZoneInfo":"/usr/share/zoneinfo"},"security":{"authorization":"enabled"},"storage":{"dbPath":"/var/lib/mongodb"},"systemLog":{"destination":"file","logAppend":true,"path":"/var/log/mongodb/mongod.log"}}}}
{"t":{"$date":"2023-06-28T04:58:42.426+00:00"},"s":"E",  "c":"NETWORK",  "id":23024,   "ctx":"initandlisten","msg":"Failed to unlink socket file","attr":{"path":"/tmp/mongodb-27017.sock","error":"Operation not permitted"}}
{"t":{"$date":"2023-06-28T04:58:42.426+00:00"},"s":"F",  "c":"ASSERT",   "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":40486,"file":"src/mongo/transport/transport_layer_asio.cpp","line":1126}}
{"t":{"$date":"2023-06-28T04:58:42.426+00:00"},"s":"F",  "c":"ASSERT",   "id":23092,   "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}

{"t":{"$date":"2023-06-28T15:15:40.472+00:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"-","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2023-06-28T15:15:40.474+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2023-06-28T15:15:40.474+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}}
{"t":{"$date":"2023-06-28T15:15:40.475+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2023-06-28T15:15:40.487+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}}
{"t":{"$date":"2023-06-28T15:15:40.487+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}
{"t":{"$date":"2023-06-28T15:15:40.487+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}}
{"t":{"$date":"2023-06-28T15:15:40.487+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2023-06-28T15:15:40.487+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":310905,"port":27017,"dbPath":"/var/lib/mongodb","architecture":"64-bit","host":"creazilla-mongo"}}
{"t":{"$date":"2023-06-28T15:15:40.487+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.6","gitVersion":"26b4851a412cc8b9b4a18cdb6cd0f9f642e06aa7","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2023-06-28T15:15:40.487+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2023-06-28T15:15:40.487+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/etc/mongod.conf","net":{"bindIp":"*","port":27017},"processManagement":{"timeZoneInfo":"/usr/share/zoneinfo"},"security":{"authorization":"enabled"},"storage":{"dbPath":"/var/lib/mongodb"},"systemLog":{"destination":"file","logAppend":true,"path":"/var/log/mongodb/mongod.log"}}}}
{"t":{"$date":"2023-06-28T15:15:40.489+00:00"},"s":"I",  "c":"STORAGE",  "id":22270,   "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/var/lib/mongodb","storageEngine":"wiredTiger"}}
{"t":{"$date":"2023-06-28T15:15:40.489+00:00"},"s":"I",  "c":"STORAGE",  "id":22297,   "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
{"t":{"$date":"2023-06-28T15:15:40.489+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=7303M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],"}}
{"t":{"$date":"2023-06-28T15:15:41.065+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":11,"message":"[1687965341:64720][310905:0x7f8f8e3f6cc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __posix_file_lock, 393: /var/lib/mongodb/WiredTiger.lock: handle-lock: fcntl: Resource temporarily unavailable"}}
{"t":{"$date":"2023-06-28T15:15:41.065+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":16,"message":"[1687965341:65062][310905:0x7f8f8e3f6cc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __conn_single, 1806: WiredTiger database is already being managed by another process: Device or resource busy"}}
{"t":{"$date":"2023-06-28T15:15:41.065+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":11,"message":"[1687965341:65767][310905:0x7f8f8e3f6cc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __posix_file_lock, 393: /var/lib/mongodb/WiredTiger.lock: handle-lock: fcntl: Resource temporarily unavailable"}}
{"t":{"$date":"2023-06-28T15:15:41.066+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":16,"message":"[1687965341:65967][310905:0x7f8f8e3f6cc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __conn_single, 1806: WiredTiger database is already being managed by another process: Device or resource busy"}}
{"t":{"$date":"2023-06-28T15:15:41.067+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":11,"message":"[1687965341:66842][310905:0x7f8f8e3f6cc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __posix_file_lock, 393: /var/lib/mongodb/WiredTiger.lock: handle-lock: fcntl: Resource temporarily unavailable"}}
{"t":{"$date":"2023-06-28T15:15:41.067+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":16,"message":"[1687965341:67034][310905:0x7f8f8e3f6cc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __conn_single, 1806: WiredTiger database is already being managed by another process: Device or resource busy"}}
{"t":{"$date":"2023-06-28T15:15:41.067+00:00"},"s":"W",  "c":"STORAGE",  "id":22347,   "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
{"t":{"$date":"2023-06-28T15:15:41.067+00:00"},"s":"F",  "c":"STORAGE",  "id":28595,   "ctx":"initandlisten","msg":"Terminating.","attr":{"reason":"16: Device or resource busy"}}
{"t":{"$date":"2023-06-28T15:15:41.067+00:00"},"s":"F",  "c":"ASSERT",   "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":28595,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":708}}
{"t":{"$date":"2023-06-28T15:15:41.067+00:00"},"s":"F",  "c":"ASSERT",   "id":23092,   "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}

karan_sharma3 · June 28, 2023, 9:27pm

Output of /etc/passwd:

# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
systemd-network:x:100:102:systemd Network Management,,,:/run/systemd:/usr/sbin/nologin
systemd-resolve:x:101:103:systemd Resolver,,,:/run/systemd:/usr/sbin/nologin
systemd-timesync:x:102:104:systemd Time Synchronization,,,:/run/systemd:/usr/sbin/nologin
messagebus:x:103:106::/nonexistent:/usr/sbin/nologin
syslog:x:104:110::/home/syslog:/usr/sbin/nologin
_apt:x:105:65534::/nonexistent:/usr/sbin/nologin
tss:x:106:111:TPM software stack,,,:/var/lib/tpm:/bin/false
uuidd:x:107:112::/run/uuidd:/usr/sbin/nologin
tcpdump:x:108:113::/nonexistent:/usr/sbin/nologin
landscape:x:109:115::/var/lib/landscape:/usr/sbin/nologin
pollinate:x:110:1::/var/cache/pollinate:/bin/false
fwupd-refresh:x:111:116:fwupd-refresh user,,,:/run/systemd:/usr/sbin/nologin
usbmux:x:112:46:usbmux daemon,,,:/var/lib/usbmux:/usr/sbin/nologin
sshd:x:113:65534::/run/sshd:/usr/sbin/nologin
systemd-coredump:x:999:999:systemd Core Dumper:/:/usr/sbin/nologin
lxd:x:998:100::/var/snap/lxd/common/lxd:/bin/false
mongodb:x:114:65534::/home/mongodb:/usr/sbin/nologin

And then,

# find / -iname mongodb-27017.sock
/tmp/mongodb-27017.sock

Lastly, this command got executed:

# chown mongodb:mongodb /tmp/mongodb-27017.sock

and then, restarted the server, but still same error!

# systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2023-06-28 21:25:57 UTC; 11s ago
       Docs: https://docs.mongodb.org/manual
    Process: 313061 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=14)
   Main PID: 313061 (code=exited, status=14)

Jun 28 21:25:56 creazilla-mongo systemd[1]: Started MongoDB Database Server.
Jun 28 21:25:57 creazilla-mongo systemd[1]: mongod.service: Main process exited, code=exited, status=14/n/a
Jun 28 21:25:57 creazilla-mongo systemd[1]: mongod.service: Failed with result 'exit-code'.

Fabio_Ramohitaj · June 28, 2023, 10:02pm

Hi @karan_sharma3,
No file.sock, but file.lock

chris · June 29, 2023, 12:12pm

{"t":{"$date":"2023-06-28T15:15:41.065+00:00"},"s":"E", "c":"WT", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":11,"message":"[1687965341:64720][310905:0x7f8f8e3f6cc0], wiredtiger_open: [WT_VERB_DEFAULT][ERROR]: __posix_file_lock, 393: /var/lib/mongodb/WiredTiger.lock: handle-lock: fcntl: Resource temporarily unavailable"}}

The starting process can’t get the lock file. This could be permissions or another mongod is running.

Check for a running mongod and kill it if there is one pkill -x mongod.

I posted earlier about setting the permissions on all the files, you possibly started mongod as root so this should be done.

Finally keep checking the log file, now that the log can be written to the error(s) preventing startup will be logged there.

karan_sharma3 · June 29, 2023, 4:26pm

I’m unable to find lock file in system tho. It’s not present.

find / -iname mongodb-27017.lock

Output of above command is empty/nothing.

Also I tried steps mentioned in solution from askubuntu.com link, didn’t work.

karan_sharma3 · June 29, 2023, 4:36pm

Hey, I think I may have deleted the .lock file myself after seeing some answer in stackoverflow, before starting out here.

I searched for lock in output of history command and found this:

I’m not sure how bad is it, but I definitely want to get over this

Is this still solvable and/or data is recoverable.

karan_sharma3 · June 29, 2023, 5:02pm

Heyyyy guys,
IDK what fixed this but its working fine now.

I would like to thank this community for helping me out and responding to this thread
@chris @Fabio_Ramohitaj @Ramachandra_Tummala @Jack_Woehr Thank you so much

Can’t mention more than 2 people as new member.

Here’s the last 50 commands that I ran:

~# history 50
  379  systemctl status mongod
  380  /usr/bin/mongod --config /etc/mongod.conf
  381  systemctl status mongod
  382  nano /etc/mongod.conf 
  383  systemctl status mongod
  384  systemctl restart mongod
  385  systemctl status mongod
  386  exit
  387  sudo -u mongodb
  388  chown -R mongodb:mongodb /var/lib/mongodb
  389  chown -R mongodb:mongodb /var/log/mongodb
  390  chown mongodb:mongodb /tmp/mongo*.sock
  391  systemctl restart mongod
  392  systemctl status mongod
  393  nano /etc/mongod.conf 
  394  systemctl status mongod
  395  ls
  396  tail -n 200 /var/log/mongodb/mongod.log
  397  /var/log/mongodb/mongod.log
  398  tail -n 200 /var/log/mongodb/mongod.log
  399  tail -n 100 /var/log/mongodb/mongod.log
  400  find / -iname mongodb-27017.lock
  401  cat /etc/passwd
  402  find / -iname mongodb-27017.lock
  403  ls /tmp/mongodb-27017.sock 
  404  cat /tmp/mongodb-27017.sock 
  405  ls /tmp/mongodb-27017.sock 
  406  find / -iname mongodb-27017.sock
  407  find / -iname mongodb-27017.lock
  408  chown mongod?:mongod? /tmp/mongo-27017.lock
  409  chown mongod:mongod /tmp/mongo-27017.lock
  410  chown mongodb:mongodb /tmp/mongo-27017.lock
  411  chown mongodb:mongodb /tmp/mongodb-27017.sock
  412  systemctl restart mongod
  413  systemctl status mongod
  414  ls migration/
  415  sudo chown -R mongodb:mongodb /var/lib/mongodb
  416  sudo chown mongodb:mongodb /tmp/mongodb-27017.sock
  417  sudo service mongod restart
  418  sudo service mongod status
  419  sudo chown -R mongodb:mongodb /var/lib/mongodb
  420  sudo chown mongodb:mongodb /tmp/mongodb-27017.sock
  421  sudo service mongod restart
  422  systemctl restart mongod
  423  systemctl status mongod
  424  pwd
  425  whoami
  426  service mongod status
  427  history -n 50
  428  history 50

chris · June 29, 2023, 9:37pm

I’ll @mention them for you: @Jack_Woehr @Ramachandra_Tummala @Fabio_Ramohitaj

Thank you for a good initial post that was well formatted, it matters!