Store Large Files

Overview

In this guide, you can learn how to store and retrieve large files in MongoDB by using GridFS. GridFS is a specification implemented by the MongoDB PHP Library that describes how to split files into chunks when storing them and reassemble them when retrieving them. The library's implementation of GridFS is an abstraction that manages the operations and organization of the file storage.

Use GridFS if the size of your files exceeds the BSON document size limit of 16MB. For more detailed information on whether GridFS is suitable for your use case, see GridFS in the MongoDB Server manual.

How GridFS Works

GridFS organizes files in a bucket, a group of MongoDB collections that contain the chunks of files and information describing them. The bucket contains the following collections, named using the convention defined in the GridFS specification:

• The chunks collection stores the binary file chunks.

• The files collection stores the file metadata.

When you create a new GridFS bucket, the library creates the preceding collections, prefixed with the default bucket name fs, unless you specify a different name. The library also creates an index on each collection to ensure efficient retrieval of the files and related metadata. The library creates the GridFS bucket, if it doesn't exist, only when the first write operation is performed. The library creates indexes only if they don't exist and when the bucket is empty. For more information about GridFS indexes, see GridFS Indexes in the MongoDB Server manual.

When using GridFS to store files, the library splits the files into smaller chunks, each represented by a separate document in the chunks collection. It also creates a document in the files collection that contains a file ID, file name, and other file metadata. You can upload the file by passing a stream to the MongoDB PHP Library to consume or creating a new stream and writing to it directly. To learn more about streams, see Streams in the PHP manual.

View the following diagram to see how GridFS splits the files when uploaded to a bucket:

When retrieving files, GridFS fetches the metadata from the files collection in the specified bucket and uses the information to reconstruct the file from documents in the chunks collection. You can read the file by writing its contents to an existing stream or creating a new stream that points to the file.

Create a GridFS Bucket

To store or retrieve files from GridFS, call the MongoDB\Database::selectGridFSBucket() method on your database. This method accesses an existing bucket or creates a new bucket if one does not exist.

The following example calls the selectGridFSBucket() method on the db database:

$bucket = $client->db->selectGridFSBucket();

$custom_bucket = $client->db->selectGridFSBucket(
    ['bucketName' => 'myCustomBucket']
);

Upload Files

You can upload files to a GridFS bucket by using the following methods:

• MongoDB\GridFS\Bucket::openUploadStream(): Opens a new upload stream to which you can write file contents

• MongoDB\GridFS\Bucket::uploadFromStream(): Uploads the contents of an existing stream to a GridFS file

$stream = $bucket->openUploadStream('my_file', [
    'metadata' => ['contentType' => 'text/plain']
]);
fwrite($stream, 'Data to store');
fclose($stream);

$file = fopen('/path/to/input_file', 'rb');
$bucket->uploadFromStream('new_file', $file);

Retrieve File Information

In this section, you can learn how to retrieve file metadata stored in the files collection of the GridFS bucket. The metadata contains information about the file it refers to, including:

• The _id of the file

• The name of the file

• The length/size of the file

• The upload date and time

• A metadata document in which you can store any other information

To retrieve files from a GridFS bucket, call the MongoDB\GridFS\Bucket::find() method on the MongoDB\GridFS\Bucket instance. The method returns a MongoDB\Driver\Cursor instance from which you can access the results. To learn more about Cursor objects in the MongoDB PHP Library, see the Access Data From a Cursor guide.

$files = $bucket->find();
foreach ($files as $file_doc) {
    echo toJSON($file_doc), PHP_EOL;
}

{ "_id" : { "$oid" : "..." }, "chunkSize" : 261120, "filename" : "my_file",
"length" : 13, "uploadDate" : { ... }, "metadata" : { "contentType" : "text/plain" },
"md5" : "6b24249b03ea3dd176c5a04f037a658c" }
{ "_id" : { "$oid" : "..." }, "chunkSize" : 261120, "filename" : "new_file",
"length" : 13, "uploadDate" : { ... }, "md5" : "6b24249b03ea3dd176c5a04f037a658c" }

function toJSON(object $document): string
{
    return MongoDB\BSON\Document::fromPHP($document)->toRelaxedExtendedJSON();
}

Download Files

You can download files from a GridFS bucket by using the following methods:

• MongoDB\GridFS\Bucket::openDownloadStreamByName() or MongoDB\GridFS\Bucket::openDownloadStream(): Opens a new download stream from which you can read the file contents

• MongoDB\GridFS\Bucket::downloadToStream(): Writes the entire file to an existing download stream

$stream = $bucket->openDownloadStreamByName('my_file');
$contents = stream_get_contents($stream);
echo $contents, PHP_EOL;
fclose($stream);

"Data to store"

$stream = $bucket->openDownloadStream(new ObjectId('66e0a5487c880f844c0a32b1'));
$contents = stream_get_contents($stream);
fclose($stream);

$stream = $bucket->openDownloadStreamByName('my_file', ['revision' => 0]);
$contents = stream_get_contents($stream);
fclose($stream);

$file = fopen('/path/to/output_file', 'wb');
$bucket->downloadToStream(
	new ObjectId('66e0a5487c880f844c0a32b1'),
	$file,
);

Rename Files

Use the MongoDB\GridFS\Bucket::rename() method to update the name of a GridFS file in your bucket. Pass the following parameters to the rename() method:

• _id value of the file you want to rename

• New file name

The following example shows how to update the filename field to 'new_file_name' by referencing a file's _id value:

$bucket->rename(new ObjectId('66e0a5487c880f844c0a32b1'), 'new_file_name');

Alternatively, you can use the MongoDB\GridFS\Bucket::renameByName() method to rename a GridFS file and all its file revisions. Pass the following parameters to the renameByName() method:

• filename value you want to change

• New file name

The following example renames all files that have a filename value of 'my_file':

$bucket->renameByName('my_file', 'new_file_name');

Delete Files

Use the MongoDB\GridFS\Bucket::delete() method to remove a GridFS file's collection document and associated chunks from your bucket. This effectively deletes the file. Pass the _id value of the file you want to delete as a parameter to the delete() method.

The following example shows you how to delete a file by referencing its _id field:

$bucket->delete(new ObjectId('66e0a5487c880f844c0a32b1'));

Alternatively, you can use the MongoDB\GridFS\Bucket::deleteByName() method to delete a GridFS file and all its file revisions. Pass the filename value of the file you want to delete as a parameter to the deleteByName() method, as shown in the following code:

$bucket->deleteByName('my_file');

API Documentation

To learn more about using the MongoDB PHP Library to store and retrieve large files, see the following API documentation:

• MongoDB\Database::selectGridFSBucket()

• MongoDB\GridFS\Bucket::openUploadStream()

• MongoDB\GridFS\Bucket::uploadFromStream()

• MongoDB\GridFS\Bucket::find()

• MongoDB\GridFS\Bucket::openDownloadStreamByName()

• MongoDB\GridFS\Bucket::openDownloadStream()

• MongoDB\GridFS\Bucket::downloadToStream()

• MongoDB\GridFS\Bucket::rename()

• MongoDB\GridFS\Bucket::delete()