Build an AWS Lambda Serverless function with PyMongo & MongoDB
Rate this video
00:00:00Introduction to AWS Lambda and MongoDB
- Introduction to the tutorial's objectives.
- Benefits of using AWS Lambda with MongoDB.00:01:49Setting Up MongoDB Atlas Cluster
- Step-by-step instructions on creating an Atlas cluster.
- Configuring the database with a username, password, and IP access.00:03:56Loading Sample Data into Atlas Cluster
- Instructions on how to load a sample data set into the newly created Atlas cluster.00:04:18Setting Up AWS Lambda Function
- Steps to create a new AWS Lambda function.
- Choosing the runtime and creating the function.00:05:57Connecting AWS Lambda to MongoDB Cluster
- Preparing the connection between AWS Lambda and MongoDB.
- Using Visual Studio Code for easier dependency management.00:07:10Packaging and Uploading Code to AWS Lambda
- Instructions on packaging dependencies and Lambda function code.
- Uploading the zipped code to AWS Lambda.00:08:07Testing AWS Lambda Function
- Running the AWS Lambda function to test the database connection.
- Verifying the insertion of a document into the MongoDB cluster.00:09:54Creating and Testing Aggregation Pipeline
- Modifying the Lambda function to include an aggregation pipeline.
- Testing the pipeline to retrieve a sample of American restaurants in Brooklyn.00:11:37Conclusion and Additional Resources
- Recap of the tutorial and its outcomes.
- Encouragement to explore more MongoDB resources and further learning materials.The primary focus of the video is to demonstrate how to set up and use AWS Lambda with MongoDB's Python driver, PyMongo, to run applications without worrying about hardware complexities.
🔑 Key Points
- Setting up an Atlas cluster and connecting it to AWS Lambda using PyMongo.
- Writing and executing an aggregation pipeline within AWS Lambda.
- Managing dependencies and packaging code for AWS Lambda deployment.
- Testing the connection and functionality of the AWS Lambda function with MongoDB.
🔗 Related Links
Full Video Transcript
hey everyone my name is Anaya Ray sangani and I am a developer Advocate over here with manga DB today I have a really great tutorial for you all on how to use AWS Lambda with pie [ __ ] mongodbs python driver so AWS lamb does paper execution model along with mongodb's unparalleled scalability will really help developers understand what it's like to have their applications Thrive without the hardware complexities they might normally be used to this combination is great because instead of fussing over Hardware complexities developers are free to focus entirely on running and executing their applications so this tutorial will take you through how to properly set up an atlas cluster how to connect it to AWS Lambda using pymongo how to write an aggregation Pipeline on our data and then how to return our wanted information so let's get started our first step is to create an atlas cluster we are currently in the atlas UI click the green build a database button for this tutorial we can use the m03 tier please ensure that your provider is AWS and you are in the correct region I'm going to be naming the cluster AWS Lambda demo and then we can hit create we will be taken to this page where we need to set a username and password for this tutorial I will use the username [ __ ] and then the password [ __ ] as well but please ensure that you are using a secure username and password then hit create user I am going to add in Access from anywhere which is just 0.0.0.0 but please keep in mind that this is not very secure for production we do not recommend it this is just for this demo and then I'm going to hit finish and close once your cluster is up and running we need to now load in our sample data set click the ellipses next to the browse Collections and click on load sample data set once our cluster is fully provisioned let's move on and set up our AWS Lambda function sign in to your AWS account and either search for Lambda or click on the button that says Lambda we are going to want to hit this orange create function button and now we can see that we are in our create function area we want to click the first one which is author from scratch and then we want to select a name for our function I'm going to call mine AWS Lambda demo 1. the round time that I'm going to be using is python 3.8 and then I'm going to keep the architecture the same so now I can scroll down and hit the create function button on the bottom of the screen once your function is created you will see a page with your function overview above and then your code Source right below now we are ready to set up our connection from AWS Lambda to our mongodb cluster to make things easier for ourselves because we are going to be using pi [ __ ] a dependency instead of editing directly in the code Source we are actually going to be using visual studio code so AWS Lambda has a limited amount of pre-installed libraries and dependencies so in order to get around this and incorporate Pi [ __ ] we will actually need to package our code in a special way due to this workaround this will not be a typical tutorial with testing at every single step we will first have to download our dependencies and upload our code to Lambda prior to ensuring our code Works instead of using a typical requirements.txt file now we are ready to establish a connection between AWS Lambda and our previously made mongodb cluster to do this let's first access our terminal and then we are going to create a new directory on our local machine so I'm just going to do it on my desktop so CD desktop and then I'm going to do after AWS Lambda demo I want to name it demo one to keep things consistent and so now we need to install Pi [ __ ] and as I said before Lambda doesn't have every Library available so we need to download pymongo at the root of our project so we can do this by first entering the directory we just made which will be CD AWS Lambda demo one and then I'm going to make a new directory in here where all of our dependencies are going to live so it's going to be maketer dependencies and here's where I am going to install Pi [ __ ] directly in this new package so this command is going to be pip install Target dot slash dependencies and then Pi [ __ ] and then just hit enter once everything is downloaded we are now going to open up visual studio code so here I have Visual Studio code open and then the directory that we are working in which once again is AWS Lambda demo one so now that we're in here the first thing we're going to do is create a new python file and we are going to name it Lambda function dot py so this is actually where the heart of our entire connection is going to be so let's create that first and now we are going to be doing import OS and then from PI [ __ ] import [ __ ] clients so the first thing we need to do is establish a connection to our database that we created before so we're going to have [ __ ] client and then host is equal to OS on fire on dot gets and then here we're going to have our Atlas URI so I am going to be showing you all how to insert our environment variables at a later step so even if once we have this file done even if I run it it's not going to work just yet so now if you remember from before in the code Source there's a little Lambda Handler definition so I'm going to copy that in with deaf Lambda Handler and then we have events and context and now I'm just going to do the name of my database so here I'm going to be creating just to test our connection and make sure it's working I'm actually going to be creating a new database and then a new collection and they're both going to have the name of test just to make things a little bit easier for us so this is the name of my database and then we're going to have the name of our collection so this is going to be collection is equal to database test great and then inside of our little test database and collection I'm actually going to be inserting a document so this way when we check we will know for sure if we are able to write to our database after we run all of our code in AWS Lambda so to do this I am just going to document to include and that's going to be documents is equal to and I'm just going to do my first name and then my last name so this is going to be Anaya and then last name raisingani and let's make sure we have our brackets fantastic and then we are going to actually insert the document so we're going to have results is equal to our collection dot insert one because we only have one document and then we need to make sure to specify document perfect and then we can just write a little just to know for a fact in AWS Lambda if it has actually been entered we can do just if result dot inserted ID and then return inserted and then else we can do return not inserted perfect let's save that now that we have our Lambda function what we need to do is zip all of our dependencies together and then upload it to AWS Lambda and run it from there as I mentioned before we still have we haven't inserted our Atlas URI or our connection string yet so even if we run this it's not going to work so in order to run this properly we first need to create a zip file and so we can do this actually at the roots let's go back to our terminal and we can go with the command CD dependencies and then we can do zip our deployment dot zip perfects so we have cd into our dependencies directory and now we're going to do zip r dot dot deployment dot zip and then period to copy in everything great so this creates a deployment.zip file inside of our project directory and so now we actually need to add in our Lambda function.py file to the root of our ZIP file so CD dot dot and then zip deployment dot zip Lambda function.py okay so I spelled that wrong so let's fix that Lambda function Dot py perfect so now let's just check and make sure that we have our nesting properly made great this is what we should see we should see a dependencies folder our deployment dot zip file and then our Lambda function.py file so now we can actually go back to our Lambda function and then we can click on upload from and hit zip file and then click upload and let's search for our AWS Lambda demo one and we want our deployment dot zip so let's open it and then hit save awesome so now we can see all of our code from our Lambda function.py inside of our code source which is awesome so now we can actually go ahead and configure our environment variables and go up here and click on configuration and we can click on environment variables and we can add in our environment variable so our key is atlas URI and then our value is taken from our connection string so we can go back to our database and click on connect and then hit on drivers and we want to copy this in so let's go back and copy in our connection string and then don't forget to exchange out the password for your own once that's in we can hit save so now we are actually ready to see if our connection was successful so we want to hit let's go back to code we want to hit the test button so if it's the first time that you're hitting your test button you're going to actually need to name your event so I can just say um demo one and then we can keep everything else the exact same and we can just hit save awesome so now let's hit test and as we can see we have a response that says inserted let's just go and double check in Atlas and make sure that our document was actually inserted go over here click on browse collections scroll down and as you can see there is a database named test with a collection inside name test and then there is a document with my first name and my last name that has been included so congratulations our connection works now we are ready to take things a step further and actually input a simple aggregation pipeline so for our pipeline we want to change our code to connect to our sample restaurants database and our restaurants collection so we're going to be doing is we're going to be incorporating our aggregation pipeline so just find a sample size of five American Cuisine restaurants that are located in Brooklyn New York so since we have our PI [ __ ] dependency already downloaded we can directly incorporate our aggregation pipeline into our code source so let's change our Lambda function so just open that up and we want to incorporate just a pretty basic aggregation pipeline just to show you all really how powerful AWS Lambda is and mongodb is so let's change the name of our database to be connecting to our sample restaurants database so sample restaurants database I will show you is right over here so sample restaurants and as you can see for example this one is a restaurant in Brooklyn that is American Cuisine and then it gives the name of the restaurant so we want to find out of 25 over 25 000 documents we want to get a sample size of just five that give us American restaurants in Brooklyn New York so let's get started and edit this file first things first I'm just going to change this to connect to make things a little bit more readable and now we're going to change out the name of our database this is going to be connect sample restaurants and then collection is also going to be um restaurants amazing and then we can incorporate our aggregation pipeline so we can delete this and let me just make a comment here for clarity pipeline so first let's find American restaurants in Brooklyn that match what we want so we can do a dollar sign match operator we need to have our quotes and we want it to be burrow Brooklyn and then we want to match it with Cuisine to show American perfect and let's make sure we have upper brackets at the end great so that is our match operator and now we only want to limit it to five we don't want to find all the probably thousands of restaurants that hit our match so we want to just limit it use a limit Operator just to five that's what that looks like and then if you remember from here we don't want to have all of this information we don't want to read back everything we just want to see the cuisine the burrow and then let's also include the name just to make things easier for ourselves so we can do that with a project operator it's going to be dollar sign project and then we are going to do so we don't want the ID so we can actually zero the ID out and then we do want the name so I'm going to keep the name in as one we do want the burrow perfects and then that one is going to be one as well and we also want cuisine Cuisine is going to be one and then let's close that up and then let's actually make sure we can see our we can delete all of this let's make sure we can see our pipeline so results is equal to lists lists collection Dot aggregate pipeline and then we want to print the results so for restaurants in results print restaurant perfect great so now let's as you can see AWS Lambda lets us know that changes have not been deployed yet so we can hit the deploy button first just to save everything make sure everything's updated we can go over here and click the test button and great we have five we have a sample size of five restaurants that are located in Brooklyn New York that have American food this tutorial provided you with the hands-on experience to connect a mongodb atlas database to AWS Lambda we also got an inside look on how to write to a cluster from Lambda how to read back information from an aggregation Pipeline and how to properly configure our dependencies when using AWS Lambda hopefully now you are ready to take advantage of both AWS Lambda and mongodb and create some amazing applications if you enjoyed this tutorial and you would like to learn more please check out our mongodb developer Center our YouTube channel and also our podcast and I will link everything in the description below thank you so much for watching