2
May 2022
2 / 4
Jun 2022
28d ago
plastikfan_p
May 2022

This may seem to be an odd question but please hear me out. I am building a cli application in Go that needs to work with a relatively small data set, lets say a few documents less than 20 mb. Internally this may still represent many thousands of objects. Given this relatively low data set, I thought it would be best not to have to use a full blown mondodb server. Rather, since the cli will have access to these documents on disk, I would like to be able to load them into memory and query it with a mongodb client/go-driver as though connnected to a server.

At this stage, I’m only concerned with running queries not mutating data (although will most likely become a requirement, but the qeury part is the most important aspect for now).

It this possible? It is akin to using a lite version of mongo db, which exists in memory only. To me this makes sense, because once you have a document, why could you not be able to query it the same way you would issue a query to a mongo db server.

Currently, I have build prototypes with .net/powershell that uses XML with querying implemented via xpath queries, to prove the via-ability of my idea. I now need to drammatically improve its performance with Go and hopefully gain the querying capabilities of mongdb. I have considered google Protocol Buffers, but that does not seem suited to my usecase (data can’t be represented hierarchichally and message size is limited). Also cosidered MsgPack, but this is also unsuitable for same reason as Protocol Buffers. I then discovered BSON which naturally led me to MongoDb.

I suppose another way of asking this question is, is there an in memory only version of mondodb that can be loaded with JSON/BSON documents, then queried with mongodb client/go-driver.

Thanks.

2
2 months later
Matt_DaleMongoDB Employee
Jun 2022

Hey @plastikfan_p, thanks for the interesting question! MongoDB does have an in-memory storage engine that you can use by running:

mongod --storageEngine inMemory --dbpath <path>

or with a configuration file:

storage:
   engine: inMemory
   dbPath: <path>

However, it sounds like you might want an in-process MongoDB server that can be used without starting a separate process, like SQLite. As far as I know, there is no official version of the MongoDB database that can run in-process with a Go application. If you need an in-memory database that runs in a Go process, you could check out HashiCorp’s go-memdb.

3 years later
senthil_manickavasagam
29d

I am interested in the same question. Is it still the case that one cannot embedded MongoDB like SQLite in the running process. i.e. use MongoDB as a micro service within the application?


Want to read more? Browse other topics in Drivers or view latest topics.