Integrating MongoDB Change Streams With Socket.IO

Brian Leonard5 min read • Published Apr 18, 2022 • Updated Aug 28, 2024

MongoDB

Rate this tutorial

Integrating MongoDB Change Streams with Socket.IO

Introduction

The Socket.IO Getting Started guide provides a nice introduction to Socket.IO. The guide bundles the server and client into a single application where messages submitted via an HTML input form are received and displayed on the page.

Since MongoDB supports an exceptional eventing framework of its own, this tutorial will demonstrate how to propagate events emitted from MongoDB through to Socket.IO. To keep things consistent, I will try to mirror the Socket.IO Getting Started guide as much as possible.

Let's get started...

The web framework

As with the Socket.IO Getting Started guide, we're going to set up a simple HTML webpage, however, in our example, it's only going to display a list of messages -- there will be no input form.

First let's create a package.json manifest file that describes our project. I recommend you place it in a dedicated empty directory (I'll call mine mongo-socket-chat-example).

1 {
2  "name": "monngo-socket-chat-example",
3  "version": "0.0.1",
4  "description": "my first mongo socket.io app",
5  "dependencies": {}
6 }

Then use npm to install express:

1 npm install express@4

One express is installed, we can set up an index.js file that will set up our application.

1 const express = require('express');
2 const app = express();
3 const http = require('http');
4 const server = http.createServer(app);
5 
6 app.get('/', (req, res) => {
7  res.send('<h1>Hello world</h1>');
8 });
9 
10 server.listen(3000, () => {
11  console.log('listening on *:3000');
12 });

This means that:

Express initializes app to be a function handler that you can supply to an HTTP server (as seen in line 4).
We define a route handler / that gets called when we hit our website home.
We make the HTTP server listen on port 3000.

If you run node index.js you should see the following:

And if you point your browser to http://localhost:3000:

Serving HTML

So far in index.js we are calling res.send and passing it a string of HTML. Our code would look very confusing if we just placed our entire application’s HTML there, so we're going to create an index.html file and serve that instead.

Let’s refactor our route handler to use sendFile instead.

1 app.get('/', (req, res) => {
2 
3  res.sendFile(__dirname + '/index.html');
4 
5 });

Here's a simple index.html file to display a list of messages with some styling included:

1 <!DOCTYPE html>
2 <html>
3   <head>
4     <title>Socket.IO chat</title>
5     <style>
6       body { margin: 0; padding-bottom: 3rem; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; }
7 
8       #messages { list-style-type: none; margin: 0; padding: 0; }
9       #messages > li { padding: 0.5rem 1rem; }
10       #messages > li:nth-child(odd) { background: #efefef; }
11     </style>
12   </head>
13   <body>
14     <h1>Hello world from file</h1>
15     <ul id="messages"></ul>
16 </body>
17 </html>

If you restart the process (by hitting Control+C and running node index.js again) and refresh the page it should look like this:

Integrating MongoDB

Since you're here, I'm going to assume you already have access to a MongoDB cluster. If you do not, just follow these Create a MongoDB Cluster instructions.

Note, a cluster (replica set) is required because we'll be using change streams, which require an oplog. There's no need to worry, however, as it is easy enough to configure a single node replica set.

Create the Database and Collection

For this example I'm going to create a chat database with a messages collection along with an initial record that I will later use to validate connectivity to MongoDB from my client application:

Install dotenv

To avoid storing MongoDB credentials in our application code, we'll use dotenv to read the MongoDB connection string from our environment. As with the express framework, use npm to install dotenv:

1 npm install dotenv --save

Create a .env file with the following MONGODB_CONNECTION_STRING variable:

1 MONGODB_CONNECTION_STRING='<Your MongoDB Connection String>'

Then add the following to your index.js:

1 require('dotenv').config()
2 console.log(process.env.MONGODB_CONNECTION_STRING) // remove this after you've confirmed it working

If you restart the process (by hitting Control+C and running node index.js again) you can verify that your environment is working properly:

Install the Node.js driver:

Use npm once again to install the Node.js driver:

1 npm install mongodb@4.5

Connect to MongoDB

Add the following code to your index.js:

1 const { MongoClient } = require("mongodb");
2 
3 const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING);  // remove this after you've confirmed it working
4 
5 async function run() {
6 
7   try {
8 
9     await client.connect();
10     const database = client.db('chat');
11     const messages = database.collection('messages');
12 
13     // Query for our test message:
14     const query = { message: 'Hello from MongoDB' };
15     const message = await messages.findOne(query);
16     console.log(message);
17 
18   } catch {
19 
20     // Ensures that the client will close when you error
21     await client.close();
22   }
23 }
24 
25 run().catch(console.dir);

Restart your application and you should see the following

For further information, this MongoDB Node.js Quick Start provides an excellent introduction to incorporating MongoDB into your Node.js applications.

Watching for Changes

We want to be alerted any time a new message is inserted into the database. For the purpose of this tutorial we'll also watch for message updates. Replace the three lines of query test code in index.js with the following:

1        // open a Change Stream on the "messages" collection
2        changeStream = messages.watch();
3 
4        // set up a listener when change events are emitted
5        changeStream.on("change", next => {
6            // process any change event
7            switch (next.operationType) {
8                case 'insert':
9                    console.log(next.fullDocument.message);
10                    break;
11                case 'update':
12                    console.log(next.updateDescription.updatedFields.message);
13            }
14        });

Then edit and/or insert some messages:

Integrating Socket.IO

Socket.IO is composed of two parts:

A server that integrates with (or mounts on) the Node.JS HTTP Server socket.io
A client library that loads on the browser side socket.io-client

During development, socket.io serves the client automatically for us, as we’ll see, so for now we only have to install one module:

1 npm install socket.io

That will install the module and add the dependency to package.json. Now let’s edit index.js to add it:

1 const { Server } = require("socket.io");
2 const io = new Server(server);

Now in index.html add the following snippet before the </body> (end body tag):

1 <script src="/socket.io/socket.io.js">&lt;/script>
2 <script>
3    var socket = io();
4 </script>

Broadcasting

The next goal is for us to emit the event from the server to the rest of the users.

In order to send an event to everyone, Socket.IO gives us the io.emit() method, which looks as follows:

1 io.emit('<event name>', '<event data>')

So, augment our change stream code with the following:

1       switch (next.operationType) {
2                 case 'insert':
3                     io.emit('chat message', next.fullDocument.message);
4                     console.log(next.fullDocument.message);
5                     break;
6 
7                 case 'update':
8                     io.emit('chat message', next.updateDescription.updatedFields.message);
9                     console.log(next.updateDescription.updatedFields.message);
10             }

And on the client side when we capture a 'chat message' event we’ll include it in the page. The total client-side JavaScript code now amounts to:

1 <!DOCTYPE html>
2 <html>
3  <head>
4    <title>Socket.IO chat</title>
5    <style>
6      body { margin: 0; padding-bottom: 3rem; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; }
7 
8      #messages { list-style-type: none; margin: 0; padding: 0; }
9      #messages > li { padding: 0.5rem 1rem; }
10      #messages > li:nth-child(odd) { background: #efefef; }
11    </style>
12  </head>
13  <body>
14    <ul id="messages"></ul>
15 
16    <script src="/socket.io/socket.io.js"></script>
17    <script>
18        var socket = io();
19        var messages = document.getElementById('messages');
20 
21        socket.on('chat message', function(msg) {
22            var item = document.createElement('li');
23            item.textContent = msg;
24            messages.appendChild(item);
25            window.scrollTo(0, document.body.scrollHeight);
26        });
27    </script>
28 </body>
29 </html>

And that completes our chat application, in about 80 lines of code! This is what it looks like on the web client when messages are inserted or updated in our chat.messages collection in MongoDB:

Rate this tutorial

Tutorial

Optimizing $lookup Performance Using the Power of Indexing

Aug 30, 2024 | 7 min read

Article

Implementing Right to Erasure with CSFLE

Mar 08, 2023 | 7 min read