Mongodb on Ubuntu -' Main process exited, code=exited, status=1/FAILURE'

chris · 2022-03-23T13:02:27.683Z

As the logging is not providing error information. I would suggest running mongod manually at this point.

sudo -u mongodb /usr/bin/mongod --auth --config /etc/mongod.conf

This should error out. Given that nothing is appearing

Otherwise you’re basically randomly changing/removing files/configuration without cause.

Konstantina_Arnaoutaki · 2022-03-23T15:40:57.217Z

I removed this

tmp/mongodb-27017.sock

There is only 23654. sock available now, still it didn’t work.

Konstantina_Arnaoutaki · 2022-03-23T15:51:55.931Z

Nothing seems to be logged on the file

/var/lib/mongodb/mongodb.log

it is empty.
I tried also adding quotes as per the official instructions

mongodb.com

Configuration File Options — MongoDB Manual

so the path to log files looks like
path: "/var/log/mongodb/mongod.log"

still the same error…

steevej · 2022-03-23T16:11:04.967Z

We need output of the following commands

ss -tlnp
ps -aef | grep [m]ongod
ls -l /tmp/mongodb-*

Is the log file

Konstantina_Arnaoutaki:

/var/lib/mongodb/mongodb.log

or

Konstantina_Arnaoutaki:

/var/log/mongodb/mongod.log

Konstantina_Arnaoutaki · 2022-03-23T16:41:35.663Z

apologies for the misunderstanding, the correct file is

/var/log/mongodb/mongod.log

Result of

ss -tlnp
is

State       Recv-Q Send-Q                                                                         Local Address:Port                                                                                        Peer Address:Port
LISTEN      0      128                                                                                        *:26603                                                                                                  *:*                   users:(("node",pid=2711,fd=23))
LISTEN      0      128                                                                                        *:80                                                                                                     *:*
LISTEN      0      5                                                                                  127.0.1.1:53                                                                                                     *:*
LISTEN      0      128                                                                                        *:22                                                                                                     *:*
LISTEN      0      5                                                                                  127.0.0.1:631                                                                                                    *:*
LISTEN      0      128                                                                                        *:3000                                                                                                   *:*                   users:(("node",pid=2420,fd=23))
LISTEN      0      128                                                                                127.0.0.1:33980                                                                                                  *:*                   users:(("node",pid=2711,fd=22))
LISTEN      0      128                                                                                       :::80                                                                                                    :::*
LISTEN      0      128                                                                                       :::22                                                                                                    :::*
LISTEN      0      5                                                                                        ::1:631                                                                                                   :::*

and result of ls -l /tmp/mongodb-*
is
srwx------ 1 mongodb mongodb 0 Μάρ 23 17:24 /tmp/mongodb-23654.sock

The ps -aef | grep [m]ongod gave nothing.

steevej · 2022-03-23T18:08:20.143Z

Okay.

Now remove /tmp/mongodb-… with

sudo /bin/rm -f /tmp/mongodb-23654.sock

Now manually start with

chris:

sudo -u mongodb /usr/bin/mongod --auth --config /etc/mongod.conf

and post a screenshot of doing the above. Also post the content of

Konstantina_Arnaoutaki:

/var/log/mongodb/mongod.log

Konstantina_Arnaoutaki · 2022-03-23T19:16:16.382Z

It still does not work and the /var/log/mongodb/mongod.log seems empty.

sudo -u mongodb /usr/bin/mongod --auth --config /etc/mongod.conf
sudo vim /var/log/mongodb/mongod.log
sudo -u mongodb /usr/bin/mongod --auth --config /etc/mongod.conf
sudo systemctl status mongod
● mongod.service - MongoDB Database Server
>    Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
>    Active: failed (Result: exit-code) since Τετ 2022-03-23 20:49:30 EET; 2min 56s ago
>      Docs: https://docs.mongodb.org/manual
>   Process: 27417 ExecStart=/usr/bin/mongod --auth --config /etc/mongod.conf (code=exited, status=1/FAILURE)
>  Main PID: 27417 (code=exited, status=1/FAILURE)
> 
 systemd[1]: Started MongoDB Database Server.
systemd[1]: mongod.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: mongod.service: Unit entered failed state.
systemd[1]: mongod.service: Failed with result 'exit-code'.

I suppose the logs are saved somewhere else so tried to find where with the below, but I am not sure if it correctly used:

grep -r 'mongod.service: Main process exited, code=exited, status=1/FAILURE' *

Ramachandra_Tummala · 2022-03-24T05:08:41.981Z

Please show screenshot of the manual run
Did it complete successfully?
Can you run mongod as normal user on a different port,dbpath,logpath(not using your config file)

Konstantina_Arnaoutaki · 2022-03-24T08:30:00.194Z

The results when running the command you suggested:

image869×164 8.18 KB

Konstantina_Arnaoutaki · 2022-03-24T08:30:34.032Z

If I just try to mongod I get the below:

image1398×291 10.8 KB

Ramachandra_Tummala · 2022-03-24T08:49:59.924Z

When you run mongod without any parameters it will try to start it on default port 27017 and default dirpath /data/db
Do you have this dir existing?

I asked you to run on different dbpath,logpath,port like this to rule out any issues with dirs
Something like below

Mongod --dbpath yourhomedir --logpath yourhomedir/mongod.log --port 29000 --fork

Once your mongod is up you can connect as
Mongo --port 29000

Konstantina_Arnaoutaki · 2022-03-24T09:15:50.794Z

The dir /data/db does exist

image929×309 9.11 KB

Konstantina_Arnaoutaki · 2022-03-24T09:17:24.509Z

Anyway I run this

mongod --dbpath /var/lib/mongodb --logpath /var/log/mongodb/mongod.log --port 29000 --fork

and get the below

image1895×134 8.02 KB

Ramachandra_Tummala · 2022-03-24T09:45:00.457Z

Please use your home dir or someother dir where mongod can write.I specifically mentioned this
Do not use the dirs like /var/* as given in /etc/mongod.conf

Regarding /data/db error what is your os?
Your files are owned as root
They should be owned by mongod

Konstantina_Arnaoutaki · 2022-03-24T10:27:58.160Z

ok, I have the below:

image1085×119 6.37 KB

Konstantina_Arnaoutaki · 2022-03-24T13:13:27.996Z

My OS is Ubuntu 16.04.
And the ownerships along my files are:
for directories in / owned by root, apart from (as per directories around mongodb):

var/lib/mongodb owned by mongodb user
var/log/mongodb owned by mongodb user
I also noticed that /etc/mongod.conf is under root user, is this correct?

image921×543 16 KB

Then in my home directory all the files under /home/user/ are owned by user

steevej · 2022-03-24T13:27:04.726Z

Do not use systemctl status like

Konstantina_Arnaoutaki:

sudo systemctl status mongod

when you start a process manually like

Konstantina_Arnaoutaki:

sudo -u mongodb /usr/bin/mongod --auth --config /etc/mongod.conf

Use the commands

steevej:

ss -tlnp
ps -aef | grep [m]ongod
ls -l /tmp/mongodb-*

Konstantina_Arnaoutaki:

The dir /data/db does exist

But it is owned by root, not good.

Konstantina_Arnaoutaki · 2022-03-24T20:37:53.736Z

I have changed the ownership of the mongod.conf to user mongodb
sudo chown -R mongodb:mongodb /etc/mongod.conf

and then rerun manually with the command:
sudo -u mongodb /usr/bin/mongod --auth --config /etc/mongod.conf

Now I am able to see logs inside

var/log/mongodb

2022-03-24T22:28:15.179+0200 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2022-03-24T22:28:15.179+0200 I  CONTROL  [initandlisten] allocator: tcmalloc
2022-03-24T22:28:15.179+0200 I  CONTROL  [initandlisten] modules: none
2022-03-24T22:28:15.179+0200 I  CONTROL  [initandlisten] build environment:
2022-03-24T22:28:15.179+0200 I  CONTROL  [initandlisten]     distmod: ubuntu1604
2022-03-24T22:28:15.179+0200 I  CONTROL  [initandlisten]     distarch: x86_64
2022-03-24T22:28:15.179+0200 I  CONTROL  [initandlisten]     target_arch: x86_64
2022-03-24T22:28:15.179+0200 I  CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "0.0.0.0", port: 23654 }, processManagement: { timeZoneInfo: "/usr/share/zoneinfo" }, security: { authorization: "enabled" }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2022-03-24T22:28:15.182+0200 I  STORAGE  [initandlisten] Detected data files in /var/lib/mongodb created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2022-03-24T22:28:15.182+0200 I  STORAGE  [initandlisten]
2022-03-24T22:28:15.182+0200 I  STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2022-03-24T22:28:15.182+0200 I  STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2022-03-24T22:28:15.182+0200 I  STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1463M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
2022-03-24T22:28:16.844+0200 E  STORAGE  [initandlisten] WiredTiger error (13) [1648153696:844130][24294:0x7f4bdfc9ab00], wiredtiger_open: __posix_open_file, 712: /var/lib/mongodb/WiredTiger.turtle: handle-open: open: Permission denied Raw: [1648153696:844130][24294:0x7f4bdfc9ab00], wiredtiger_open: __posix_open_file, 712: /var/lib/mongodb/WiredTiger.turtle: handle-open: open: Permission denied
2022-03-24T22:28:16.844+0200 E  STORAGE  [initandlisten] WiredTiger error (13) [1648153696:844481][24294:0x7f4bdfc9ab00], wiredtiger_open: __posix_open_file, 712: /var/lib/mongodb/WiredTiger.turtle: handle-open: open: Permission denied Raw: [1648153696:844481][24294:0x7f4bdfc9ab00], wiredtiger_open: __posix_open_file, 712: /var/lib/mongodb/WiredTiger.turtle: handle-open: open: Permission denied
2022-03-24T22:28:16.845+0200 E  STORAGE  [initandlisten] WiredTiger error (13) [1648153696:845360][24294:0x7f4bdfc9ab00], wiredtiger_open: __posix_open_file, 712: /var/lib/mongodb/WiredTiger.turtle: handle-open: open: Permission denied Raw: [1648153696:845360][24294:0x7f4bdfc9ab00], wiredtiger_open: __posix_open_file, 712: /var/lib/mongodb/WiredTiger.turtle: handle-open: open: Permission denied
2022-03-24T22:28:16.845+0200 W  STORAGE  [initandlisten] Failed to start up WiredTiger under any compatibility version.
2022-03-24T22:28:16.845+0200 F  STORAGE  [initandlisten] Reason: 13: Permission denied

I believe that all the problems were some kind related to the wrong ownership some mongo related files or directories have.

steevej · 2022-03-24T20:46:17.981Z

Output of

ls -l /var/lib/mongodb/

Ramachandra_Tummala · 2022-03-25T02:15:53.082Z

All errors indicate some permissions issue
Something wrong in the way you are starting mongod
How can be your home dir /home/mongod?
How do you login to your system?
What does id command show?
Looks like you are switching to mongod user