Explore Developer Center's New Chatbot! MongoDB AI Chatbot can be accessed at the top of your navigation to answer all your MongoDB questions.

Join us at AWS re:Invent 2024! Learn how to use MongoDB for AI use cases.
MongoDB Developer
Atlas
plus
Sign in to follow topics
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Productschevron-right
Atlaschevron-right

Introducing Atlas Stream Processing Support Within the MongoDB for VS Code Extension

Robert Walters4 min read • Published Feb 13, 2024 • Updated Mar 05, 2024
Stream ProcessingAtlas
Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Across industries, teams are building applications that need access to low-latency data to deliver compelling experiences and gain valuable business insights. Stream processing is a fundamental building block powering these applications. Stream processing lets developers discover and act on streaming data (data in motion), and combine that data when necessary with data at rest (data stored in a database). MongoDB is a natural fit for streaming data with its capabilities around storing and querying unstructured data and an effective query API. MongoDB Atlas Stream Processing is a service within MongoDB Atlas that provides native stream processing capabilities. In this article, you will learn how to use the MongoDB for VS Code extension to create and manage stream processors in MongoDB Atlas.

Installation

MongoDB support for VS Code is provided by the MongoDB for VS Code extension. To install the MongoDB for VS Code extension, launch VS Code, open the Extensions view, and search for MongoDB to filter the results. Select the MongoDB for VS Code extension.
MongoDB for VS Code extension in the marketplace

Connecting to an Atlas Stream Processing instance

Once installed, you can connect to the Atlas Stream Processing instance by clicking on the “Connect” button, as shown below.
MongoDB extension dashboard showing Connect button
This will pop up a connection string dialog box.
VS Code connection string dialog
To get the MongoDB Atlas Stream Processing instance connection string, navigate to the Atlas portal and then “Stream Processing” in the menu, and click on the “Connect” button of the Stream Processing Instance you’d like to connect from VS Code.
Atlas portal showing the StockTutorial Stream Processing instance
This will launch the Connect dialog. Select “MongoDB for VS Code,” as shown below.
Connect dialog
The connection string will be displayed in Step 3 of this dialog.
Stream Processing instance connection string within the Connect dialog
Copy the “mongodb://…” connection string to the clipboard and paste it into the connection string box in VS Code as shown in Figure 3. Make sure to modify the username : password with your own credentials. Note the colon separation of the username and password.

Exploring the Playground

Once connected, you can explore the Atlas Stream Processor instance and the associated Stream Processors. Since this is a new Stream Processing instance, there are no stream processors. Let’s create one using the Playground.
Connections listed in VS Code sidebar
Click on the “Create New Playground” button. This will launch a window with an Atlas Stream Processing template. The template provides two commands, sp.listConnections() and sp.process().
Atlas Stream Processing template
By highlighting text within the Playground, you can selectively execute parts of the script as shown in the figure above. In our example, by highlighting the sp.listConnections() statement and clicking the “Run” button, the Playground will display the results as shown in Figure 8. In this case, we have two connection registry entries defined for this Stream Processor instance. For commands that return a single result, such as listConnections, the results will be displayed in the Playground Result tab. For commands that stream results, such as with sp.process(), these results will be written to the output tab as shown below.
VS Code output window showing streaming results
Note that each execution of the Playground is independent so if your script has variables, for example, make sure you include those definitions each time you run the Playground.
The .process() command runs the stream processor continually, sending results to the console. This is great for developers since they can build out their stream processors and test them before creating them to run in perpetuity in the background.
In this example, notice when .process is run, there is a popup within VS Code that enables you to cancel/stop the .process. While you might be tempted to hit Control-C to stop the stream, clicking “Cancel” is the way to stop the output. Control-C will not stop the processing.

Developing a stream processor using the Playground

Now that we have explored the Playground, let’s create a simple stream processor that reads data from the solar sample stream, computes a tumbling window, and writes the results out to an Atlas collection. Assume two connection registry entries were created in the Atlas portal, one for sample_stream_solar and one for our Atlas cluster, called “LBSolar.” To develop the new stream processor, the following code was entered in the Playground.
Sample pipeline code for a stream processor
Here we use .process to verify that the results are what we expect. The .process command enables developers to build out their stream processing pipelines and see the results of the stream in the console versus having to write it out to a collection or other stream processing source.
Next, we can create the stream processor with the createStreamProcessor command. This will add the metadata of the stream processor to the Atlas Stream Processing back end. Once created, users will need to issue a .start() to start the processor or launch “Start Stream Processor” from the side bar menu, as shown in Figure 11.
Stream Process context menu
Some things to note: The MongoDB extension for VS Code supports auto-complete. Simply start the “sp.” and hit Tab to get a list of available functions.
Autocomplete
In addition, you can also launch the MongoDB Shell right from the context menu of the Stream Processing instance. This command leverages the credentials used in the connection string, offering quick access to the MongoDB Shell in the VS Code Terminal.
Stream processing context menu showing Launch MongoDB Shell

Summary

Support for Atlas Stream Processing begins with version 1.15 of the MongoDB extension for VS Code. The extension provides basic functionality to connect, build, and test stream processors. Over time, additional features and functionality will become available. Learn more about Atlas Stream Processing or visit the online documentation.
Top Comments in Forums
There are no comments on this article yet.
Start the Conversation

Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Tutorial

How to Implement Agentic RAG Using Claude 3.5 Sonnet, LlamaIndex, and MongoDB


Jul 02, 2024 | 17 min read
Article

Auto Pausing Inactive Clusters


Sep 09, 2024 | 10 min read
Article

5 Ways to Reduce Costs With MongoDB Atlas


Sep 11, 2024 | 3 min read
Tutorial

Build a CRUD API With MongoDB, Typescript, Express, Prisma, and Zod


Sep 04, 2024 | 10 min read
Table of Contents