Run a mongodb ReplicaSet on a Mac using `brew services`

Thomas_Bednarz · 2022-09-20T15:51:07.168Z

I like to install a MongoDB ReplicaSet on my MacBook Air M2 for development purpose only. I run a similar setup on a MacBook Pro with Intel processor. But there I have 64 GB RAM and run a Linux VM for that. However, I like to get rid of virtualization and run all my stuff directly on the Mac. This saves me a lot of time to maintain linux VMs etc.

Unfortunately I cannot get this thing going with homebrew. There seem to be several problems:

No idea how to run three mongod instances, each with a different port on a Mac. I don’t know how to give each instance a different name.
It starts one instance but is using the default .conf file instead of the one I passed as parameter.

What I have done so far:

created a folder where I store my individual versions of the homebrew.mxcl.mongodb-community.plist property files.
added three plist files named homebrew.mxcl.mongodb-community-rs1-0.plist , homebrew.mxcl.mongodb-community-rs1-1.plist , homebrew.mxcl.mongodb-community-rs1-2.plist each pointing in the ProgrammArguments tag to an individual conf file for each instance. (rs1-0 means ReplicaSet 1 node 0, rs1-1 means ReplicaSet 1 node 1, etc)
created a shell script which should start the entire ReplicaSet and another one to stop it.

#!/bin/zsh

# Start the mongodb replica set as services
brew services run mongodb-community --file=~/services/mongodb/runtime-config/homebrew.mxcl.mongodb-community-rs0.plist
brew services run mongodb-community --file=~/services/mongodb/runtime-config/homebrew.mxcl.mongodb-community-rs1.plist
brew services run mongodb-community --file=~/services/mongodb/runtime-config/homebrew.mxcl.mongodb-community-rs2.plist

The problem is, that this script starts the same formulae mongodb-community and I have no idea how I can give these different names . I should start them like brew service run mongodb-community-rs10 --file.... , brew service run mongodb-community-rs11 --file.... , etc.

The result of the above script is:

[mongodb] ./startMongoDbReplicaSet.zsh
==> Successfully ran `mongodb-community` (label: homebrew.mxcl.mongodb-community)
Service `mongodb-community` already running, use `brew services restart mongodb-community` to restart.
Service `mongodb-community` already running, use `brew services restart mongodb-community` to restart.

And the only started mongod instance is running on port 27017 and therefore completely ignoring my parameter file.

So here is one of my plist files as a sample:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>homebrew.mxcl.mongodb-community</string>
  <key>ProgramArguments</key>
  <array>
    <string>/opt/homebrew/opt/mongodb-community/bin/mongod</string>
    <string>--config</string>
    <string>~/services/mongodb/mongod_rs1-0.conf</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <false/>
  <key>WorkingDirectory</key>
  <string>/opt/homebrew</string>
  <key>StandardErrorPath</key>
  <string>/opt/homebrew/var/log/mongodb/output.log</string>
  <key>StandardOutPath</key>
  <string>/opt/homebrew/var/log/mongodb/output.log</string>
  <key>HardResourceLimits</key>
  <dict>
    <key>NumberOfFiles</key>
    <integer>64000</integer>
  </dict>
  <key>SoftResourceLimits</key>
  <dict>
    <key>NumberOfFiles</key>
    <integer>64000</integer>
  </dict>
</dict>
</plist>

The three files only differ on line 11 where I load different .conf files for each instance (these are the config file parameters, each line in ONE of the three files) :

<string>~/services/mongodb/mongod_rs1-0.conf</string>
<string>~/services/mongodb/mongod_rs1-1.conf</string>
<string>~/services/mongodb/mongod_rs1-2.conf</string>

And here is a sample of a .conf file:

# mongod.conf suitable for MACOSX on APPLE SILICON machines only

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  logRotate: reopen
  path: /opt/homebrew/var/log/mongodb/mongod_rs0-0.log

# Where and how to store data.
storage:
  dbPath: /opt/homebrew/var/mongodb/mongod_rs0-0
  journal:
    enabled: true
    commitIntervalMs: 100
  directoryPerDB: true
  engine: "wiredTiger"
  wiredTiger:
    engineConfig:
      cacheSizeGB: 1
      journalCompressor: none
      directoryForIndexes: false
    collectionConfig:
      blockCompressor: none
    indexConfig:
      prefixCompression: false

#  mmapv1:

# how the process runs
#processManagement:
#  fork: true  # fork and run in background
#  pidFilePath: /var/run/mongodb/mongod_rs0-0.pid  # location of pidfile
#  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 37017
  bindIp: 0.0.0.0 # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.

#security:

#operationProfiling:

replication:
  replSetName: "dev_pinklady_rs0"
  oplogSizeMB: 128

#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:

These config files differ in the following values:

log path
database path
network port

Can anybody help me with that or point me to some documentation how to install three mongod instances to run a ReplicaSet on one single Mac computer with Apple Silicon chip?

Stennie_X · 2022-09-21T01:50:08.462Z

Hi @Thomas_Bednarz ,

I’m not sure about the specifics of setting up a replica set to run via Homebrew services, but for development convenience I use mlaunch (part of the mtools Python scripts collection) to manage local deployments.

For example:

$ mlaunch --repl
launching: "mongod" on port 27017
launching: "mongod" on port 27018
launching: "mongod" on port 27019
replica set 'replset' initialized.

$ mlaunch list

PROCESS    PORT     STATUS     PID

mongod     27017    running    43667
mongod     27018    running    43672
mongod     27019    running    43682

$ mlaunch stop
sent signal Signals.SIGTERM to 3 processes.

$ mlaunch start
launching: "mongod" on port 27017
launching: "mongod" on port 27018
launching: "mongod" on port 27019

mlaunch uses MongoDB binaries found via your default PATH environment variable or a provided --binarypath parameter, so you can use this with binaries installed via Homebrew as well as alternative approaches.

Note: I also happen to be the maintainer for mtools in my spare time :). I use mlaunch with both Intel and Apple Silicon.

Regards,
Stennie

Thomas_Bednarz · 2022-09-21T06:58:47.281Z

Hi @Stennie_X

Thank you very much, that sounds like an interesting alternative! I am not familiar with py but I think I get this one done. Just figured out, that the problem is NOT only with MongoDB but also with Redis, which I use as simple cache.

To me it looks like brew services is doing a lot of magic behind the scene and is taking the configurations from several places. The bad thing is, that it is different on Intel and Apple Silicon Macs. What I need is a shell script that starts all my services and another one which stops them correctly (especially the ReplicaSet). I need to be able to change configuration quickly and then restart again to test very specific things.

So I will give mtools with mlaunch a try later today.

Regards, Tom

Thomas_Bednarz · 2022-09-26T09:18:34.570Z

Hi @Stennie_X

Sorry, I did not have the time to come back to this last week.

But today I played around a bit. I was able to init a RS, to stop, start etc and to connect to it using mongosh and Studio3T. However I do not understand how I can pass config files, and I do not know where the log files are etc.

So to test, I initialized with mlaunch init --replicaset --nodes 3 --name "dev_pinklady_rs0" --port 37017. That works perfect, but as you can see in my initial message, I have config files with a bunch of additional parameters in a YAML conf file.

Is it possible to pass such a file to mlauch init and if yes, do I have to pass ONE file for EACH node?

Thanks a lot for your help.

Regards, Tom

Stennie_X · 2022-09-29T02:20:43.544Z

Thomas_Bednarz:

Is it possible to pass such a file to mlauch init and if yes, do I have to pass ONE file for EACH node?

Hi @Thomas_Bednarz,

You can use -f /path/to/config to pass in the same config file for every replica set member.

If you want to manually tweak parameters after init, these are saved in an .mlaunch_startup JSON file within the top-level of a directory where you init a deployment with mlaunch. The startup_info key has the command link for starting a process.

Regards,
Stennie

Thomas_Bednarz · 2022-10-02T07:21:54.478Z

Hi @Stennie_X

Thanks a lot, just what I was looking for.

I just wanted install mtools on my Intel MacBookPro. I notices that all links on the GitHub Page are dead. Looks like the site https://blog.rueckstiess.com/ does not exist anymore, at least I cannot reach it at all.

Stennie_X · 2022-10-03T04:41:30.900Z

Thomas_Bednarz:

I notices that all links on the GitHub Page are dead. Looks like the site https://blog.rueckstiess.com/ does not exist anymore, at least I cannot reach it at all.

Hi @Thomas_Bednarz,

I’ll get in touch with the domain owner (and original mtools author) to find out what happened with the domain set up.

In the interim, you can find the same documentation in GitHub under the mtools/doc directory: mtools/doc/mlaunch.rst at develop · rueckstiess/mtools · GitHub

Or via the archive on the Wayback Machine.

Regards,
Stennie