Format for post to insert the file contents in a document

Harry_Melamed · 2024-09-20T16:58:07.713Z

I’ve been wracking what’s left of my brain trying to figure out how to insert a binary file, in this case a zip file into a document. Please help me.

Thanks

Michael_Lynn · 2024-09-20T17:44:27.948Z

in short… don’t do that

For most cases, especially with zip files or other large binaries, I’d recommend leveraging cloud object storage (like AWS S3, Azure Blob, or Google Cloud Storage) instead of storing the file directly in MongoDB. It’s more cost-efficient, scalable, and tailored for large files.

Here’s a simple way to handle it:

Store the file in object storage.
Save a reference (URL or file key) to the file in MongoDB.

This keeps your MongoDB collections lean and fast, while your files are stored in a service that’s purpose-built for binary data.

that said… if you really want to…

You can store large files directly in MongoDB using GridFS. It splits files into chunks and stores them across documents, allowing you to store binary files as well as those that exceed MongoDB’s 16MB limit. It’s useful if you want to manage your binaries directly in the database… but better solutions exist for binary data.

Harry_Melamed · 2024-09-20T18:17:36.900Z

These are relatively small files. And it would be easier just to have a quick way to store them.

Can it be done without GridFS?

steevej · 2024-09-20T18:43:52.647Z

Harry_Melamed:

Can it be done without GridFS?

Yes. With the BSON datatype BinData. It is well supported in MongoDB. I just don’t know the python, inferred from your other posts, way to use it. It is like any other field. But beware, that when a document is updated the whole document is written a new in a new block on disk. This means if your documents have other fields that are frequently updated it might be better to store the associated files into a separate collection and use $lookup to retrieve it.

steevej · 2024-09-25T13:15:02.550Z

@Harry_Melamed, if you pursued using BinData for storing your small files, it would be nice if you could share any documentation relating to how to use it with python. If of course python is the programming language that you use.

You could also add python as a tag for your thread.

Thanks