The video begins by explaining the multi-step process of enabling semantic search on user-specific data. This process includes loading, transforming, embedding, and storing data before it can be queried. The tutorial uses the AT&T Wikipedia page as a data source and demonstrates how to use libraries from Lang chain to load, transform, embed, and store vectors.

Introduction to Semantic Search

The next section of the video focuses on setting up MongoDB Atlas as the Vector store. The tutorial explains how to clone a repo, add an OpenAI API key and MongoDB connection string, and create a new Python environment. It also demonstrates how to install all the necessary requirements and vectorize the data.

Setting Up MongoDB Atlas

The final section of the video explains how to retrieve and query relevant data using Vector search. The tutorial demonstrates how to set up a vector search index on the Vector embeddings field and how to run a search using methods like similarity search or maximum marginal reference search. The video concludes by showing how to run a query and see the results.

Querying and Retrieving Data

"✅ Sign-up for a free cluster at → <a href="https://mongodb.prakticum-team.ru/cloud/atlas/register" target="_blank" rel="noreferrer">https://mongodb.prakticum-team.ru/cloud/atlas/register</a>
✅ Get help on our Community Forums → <a href="https://mongodb.prakticum-team.ru/community/forums/" target="_blank" rel="noreferrer">https://mongodb.prakticum-team.ru/community/forums/</a>
✅ <a href="<a href="<a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a>" target="_blank" rel="noreferrer"><a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a></a>-tutorial" target="_blank" rel="noreferrer"><a href="<a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a>" target="_blank" rel="noreferrer"><a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a></a>-tutorial</a>
✅ <a href="<a href="https://trymongodb.com/3H7kO3L" target="_blank" rel="noreferrer">https://trymongodb.com/3H7kO3L</a>" target="_blank" rel="noreferrer"><a href="https://trymongodb.com/3H7kO3L" target="_blank" rel="noreferrer">https://trymongodb.com/3H7kO3L</a></a>
✅ <a href="<a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a>" target="_blank" rel="noreferrer"><a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a></a>
✅ <a href="<a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a>" target="_blank" rel="noreferrer"><a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a></a>-community
✅ <a href="<a href="https://twitter.com/codestackr" target="_blank" rel="noreferrer">https://twitter.com/codestackr</a>" target="_blank" rel="noreferrer"><a href="https://twitter.com/codestackr" target="_blank" rel="noreferrer">https://twitter.com/codestackr</a></a>
✅ <a href="<a href="<a href="https://mdb.link/subscribe"" target="_blank" rel="noreferrer">https://mdb.link/subscribe"</a> target="_blank" rel="noreferrer">https://mdb.link/subscribe</a>" target="_blank" rel="noreferrer"><a href="<a href="https://mdb.link/subscribe"" target="_blank" rel="noreferrer">https://mdb.link/subscribe"</a> target="_blank" rel="noreferrer">https://mdb.link/subscribe</a></a>


The video is a tutorial on how to enable semantic search on user-specific data using Lang chain and MongoDB Atlas. The process involves loading, transforming, embedding, and storing data before it can be queried. The tutorial uses the AT&T Wikipedia page as a data source and demonstrates how to use libraries from Lang chain to load, transform, embed, and store vectors. The video also explains how to retrieve and query relevant data using Vector search.

📚 RESOURCES 📚
<a href="<a href="<a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a>" target="_blank" rel="noreferrer"><a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a></a>-tutorial" target="_blank" rel="noreferrer"><a href="<a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a>" target="_blank" rel="noreferrer"><a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a></a>-tutorial</a>
<a href="<a href="https://trymongodb.com/3H7kO3L" target="_blank" rel="noreferrer">https://trymongodb.com/3H7kO3L</a>" target="_blank" rel="noreferrer"><a href="https://trymongodb.com/3H7kO3L" target="_blank" rel="noreferrer">https://trymongodb.com/3H7kO3L</a></a>
<a href="<a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a>" target="_blank" rel="noreferrer"><a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a></a>
<a href="<a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a>" target="_blank" rel="noreferrer"><a href="https://mdb.link/ZvwUzcMvKiI" target="_blank" rel="noreferrer">https://mdb.link/ZvwUzcMvKiI</a></a>-community
<a href="<a href="https://twitter.com/codestackr" target="_blank" rel="noreferrer">https://twitter.com/codestackr</a>" target="_blank" rel="noreferrer"><a href="https://twitter.com/codestackr" target="_blank" rel="noreferrer">https://twitter.com/codestackr</a></a>
<a href="<a href="<a href="https://mdb.link/subscribe"" target="_blank" rel="noreferrer">https://mdb.link/subscribe"</a> target="_blank" rel="noreferrer">https://mdb.link/subscribe</a>" target="_blank" rel="noreferrer"><a href="<a href="https://mdb.link/subscribe"" target="_blank" rel="noreferrer">https://mdb.link/subscribe"</a> target="_blank" rel="noreferrer">https://mdb.link/subscribe</a></a>


⏱️ Timestamps ⏱️
Introduction to Semantic Search [00:00:00 - 00:02:35]
The video begins with an explanation of the multi-step process of enabling semantic search on user-specific data. This process includes loading, transforming, embedding, and storing data before it can be queried. The speaker mentions the team at Lang chain, whose goal is to simplify this process. The tutorial will walk through each of these steps using MongoDB Atlas as the vector store and the AT&T Wikipedia page as the data source.

Setting Up the Environment [00:02:36 - 00:05:11]
In this chapter, the speaker guides the viewer through the process of setting up the environment. This includes creating a free MongoDB Atlas account, obtaining an OpenAI API key, and cloning a repository with all the necessary code. The speaker also explains how to add the OpenAI API key and MongoDB connection string to the project, create a new Python environment, and install all the required dependencies.

Vectorizing and Querying Data [00:05:12 - 00:07:47]
The final chapter focuses on vectorizing and querying data. The speaker demonstrates how to load, transform, embed, and store data using the AT&T Wikipedia page as the data source. The process of creating vector representations of data using an LLM is explained, as well as how to store these vectors in a MongoDB database. The speaker then shows how to query this data using vector search, and how to set up a vector search index on the vector embeddings field. The video concludes with a demonstration of a CLI-based query and the results of a semantic search.

------
✅ Subscribe to our channel → <a href="<a href="<a href="https://mdb.link/subscribe"" target="_blank" rel="noreferrer">https://mdb.link/subscribe"</a> target="_blank" rel="noreferrer">https://mdb.link/subscribe</a>" target="_blank" rel="noreferrer"><a href="<a href="https://mdb.link/subscribe"" target="_blank" rel="noreferrer">https://mdb.link/subscribe"</a> target="_blank" rel="noreferrer">https://mdb.link/subscribe</a></a>"

enabling semantic search on user specific data is a multi-step process that includes loading transforming embedding and storing Data before it can be queried now that graphic is from the team over at Lang chain whose goal is to provide a set of utilities to greatly simplify this process in this tutorial we're going to walk through each of these steps using mongodb Atlas as our Vector store and we're going to use the AT&T Wikipedia page as our data source we'll then use libraries from Lang chain to load transform embed and store our vectors and then we can retrieve and query relevant data to get started you'll need a free mongodb Atlas account and an open aai API key and today we're going to be using python but this can also be accomplished using JavaScript and nodejs first we're going to clone a repo that has all the code written for us to get started and link to this repo is in the video description below so I'm going to go over to code and copy the link in vs code let's clone this Repository I'll select my location and open this up the first thing we need to do is open p.p We'll add our open AI API key and our mongodb connection string in these spots and then save it now let's open up the terminal and we're going to create a new python environment and let's activate the environment and we'll install all of the requirements now we're ready to vectorize our data we'll load it trans transform it embed it and store it and again we're going to use the AT&T Wikipedia site as our data source but there's no lack of sources of data it could be slack YouTube git Excel PDFs markdown files Etc and Lang chain provides a growing list of Integrations for these sources so let's take a look at the vectorize dopy file the first thing that we're doing is loading our data and here we're using the webbase loader that we're importing from Lang chain and in this loader we are pointing to the at TNT Wikipedia site and we can also load multiple sources like we're doing here with the Bank of America Wikipedia page as well and next we'll transform the data by splitting it now it needs to be split into smaller chunks so that we can easily find relevant portions based on our search query now for this example we'll use the recommended recursive character text splitter which again we are importing up here from Lang chain make this formatting look a little bit better here we have our chunk size of 1,000 so that's 1,000 characters we have an overlap of zero and our separators this is going to attempt to split on paragraphs with the two new line characters the one new line character and then sentences and words so if a paragraph doesn't fit into 1,000 characters it's going to truncate at the next word that it can find that will fit within that chunk size of under 1,000 characters and you can tune the chunk size to your liking smaller numbers will lead to more documents and vice versa on next is the embedding embedding is the process of creating Vector representations of data using an llm and there are many options to choose from such as open Ai and hugging face and Lang chain provides a standard interface for interacting with all of these so here we're going to use the popular open aai embeddings now again be sure you have your openai API key set in the pam. py file and now that we have the embedder loaded let's store our vectors and data now you will need a vector database to store your embeddings and fortunately for you mongodb is perfect for storing vector embeddings and even better the folks at Lang chain have a mongodb atlas module that will do all of the heavy lifting for you again don't forget to also add your mongod Atlas connection string to the pam. py file so here we're going to connect our client get our collection we'll go ahead and clear out the collection if there's anything there and then this is the Mong Atlas Vector search method from Lang chain that we are importing at the top here so in in this we're going to pass our docs our embeddings our collection and our index name and we only need to run this file once per data source so let's go ahead and run it so back in the terminal let's run python vectorize all right it split it into 288 documents let's go over to Mong Tobe Atlas and check it out so in our database let's go to browse Collections and we can see that a database search DB was created a collection search collection ction and we have all of these documents each of these has the original text Chunk along with the vector embeddings for each one along with some other metadata we see these embeddings are an array of 1 1536 vectors now in order to query this data using Vector search next we need to set up a vector search index on our Vector embeddings field but while we're here in Atlas let's go up to Atlas search and let's create a search index index we're going to choose Atlas Vector search Json editor and next for the index name I'm going to name it vsearch index then be sure to select the collection and we'll set some parameters the path is going to be embedding that's the field where our embeddings are and the number of Dimensions is 1536 this is specific to the embedding model of the llm that we're using and then for similarity we're going to set that to co all right and then let's click next and create that search index and now we're ready to retrieve and query our data using Vector search so let's take a look at the query. py file so this is a CLI based query so we're going to parse our arguments first and we can ask it questions within the CLI here's some sample questions that we could ask first thing it's going to do is print your question and the question that we asked then it's going to connect to our client and get our collection and then initialize our Vector store so again we're using mongod Tob Atlas Vector search from Lang chain passing it our collection and creating a new open AI embeddings for the query we have to vectorize the query in order to compare it with the vectors that we already have in our database we'll print another new line and then we'll run the actual query here now we can run a search using methods like similarity search or maximum marginal reference search which we have here and this would return the relevant slice of data which in our case would be an entire paragraph but we could continue to harness the power of the llm to contextually compress the response so that it more directly tries to answer our question so we'll run it again through the contextual compression retriever which is coming in at the top here from Lang chain and then we'll print that response as well so let's run the query and see what we get go back in our terminal let's run our query so we'll say python query uh pi and then our question is going to be who started AT&T all right let's make this a little bit bigger and we can see here our question was who started AT&T and the semantic search Returns the most relevant chunk of text but then passing that through the AI the ai's response is only the relevant pieces of text from that chunk that most directly answer the question so I hope you see how easy it is to get semantic search working in mongodb using Lang chain now if you prefer to read a written version of this tutorial is linked in the video description below give mongodb Vector search a try in your next AI project and let us know what you think if this video was helpful give it a like And subscribe for more great mongodb content ngodb 

Python

The video is a tutorial on how to enable semantic search on user-specific data using Lang chain and MongoDB Atlas. The process involves loading, transforming, embedding, and storing data before it can be queried. The tutorial uses the AT&T Wikipedia page as a data source and demonstrates how to use libraries from Lang chain to load, transform, embed, and store vectors. The video also explains how to retrieve and query relevant data using Vector search.

Semantic Search Made Easy With LangChain and MongoDB

✅ Sign-up for a free cluster at → <a href="https://mdb.link/free-1ZIYVNvRVsY" target="_blank" rel="noreferrer">https://mdb.link/free-1ZIYVNvRVsY</a>
✅ Get help on our Community Forums → <a href="https://mdb.link/community-1ZIYVNvRVsY" target="_blank" rel="noreferrer">https://mdb.link/community-1ZIYVNvRVsY</a>

** Support for the '$vectorSearch' aggregation pipeline stage is available with MongoDB Atlas 6.0.11 and 7.0.2. **

Welcome to this comprehensive guide on Vector Search, a revolutionary technique that allows you to search data based on meaning rather than just keywords. If you've ever struggled to find exactly what you're looking for in a database, this video is for you. We'll explore how Vector Search works, its benefits, and how to implement it in MongoDB for a seamless and powerful search experience.

🛠 What You'll Learn

What is Vector Search?: Understand the machine learning models behind Vector Search and how they transform text, audio, images, or other types of data into high-dimensional vectors.

Benefits of Vector Search: Discover why semantic understanding, scalability, and flexibility make Vector Search a must-have feature for modern databases.

MongoDB & Vector Search: Learn how to set up Vector Search in MongoDB Atlas, create triggers with OpenAI API, and perform vector search queries. We'll walk you through each step, from setting up your MongoDB Atlas account to writing JavaScript functions for querying.

📚 Resources 📚
✅ Written article & code → <a href="https://mdb.link/semantic-search-mongodb-atlas-vector-search-1ZIYVNvRVsY" target="_blank" rel="noreferrer">https://mdb.link/semantic-search-mongodb-atlas-vector-search-1ZIYVNvRVsY</a>
✅ Vector Search Documentation→ <a href="https://mdb.link/vector-search-1ZIYVNvRVsY" target="_blank" rel="noreferrer">https://mdb.link/vector-search-1ZIYVNvRVsY</a>
✅ OpenAI → <a href="https://platform.openai.com/overview" target="_blank" rel="noreferrer">https://platform.openai.com/overview</a>
✅ Create your first FREE MongoDB Atlas Cluster → <a href="https://youtu.be/jXgJyuBeb_o" target="_blank" rel="noreferrer">https://youtu.be/jXgJyuBeb_o</a>

------
✅ Subscribe to our channel → <a href="https://mdb.link/subscribe" target="_blank" rel="noreferrer">https://mdb.link/subscribe</a>

JavaScript

Vector Search: The Future of Data Querying Explained | Semantic Searching

Join Hubert Nguyen, a leading expert in the PHP/Laravel and MongoDB landscape, as he walks you through the intricacies of leveraging MongoDB within a PHP/Laravel back-end service. This promises to be an informative, hands-on session for PHP Developers of all experience levels. 

This stream is part of the Practice with PHP event week at MongoDB. Jump into the Community Forum (<a href="<a href="https://mdb.link/practice-with-php-community" target="_blank" rel="noreferrer">https://mdb.link/practice-with-php-community</a>)" target="_blank" rel="noreferrer"><a href="https://mdb.link/practice-with-php-community" target="_blank" rel="noreferrer">https://mdb.link/practice-with-php-community</a>)</a> and use the #practice-with-php tag to have your questions answered.

Also... Join us on November 3rd, 2023 for an exclusive Ask Me Anything event with Hubert and some of the best and brightest PHP minds. (<a href="https://www.youtube.com/watch?v=rLkoiZGhUq0)" target="_blank" rel="noreferrer">https://www.youtube.com/watch?v=rLkoiZGhUq0)</a>

Links:
Official Rules: <a href="https://mdb.link/practice-with-php-rules" target="_blank" rel="noreferrer">https://mdb.link/practice-with-php-rules</a>
PHP Quest: <a href="https://mdb.link/practice-with-php-event" target="_blank" rel="noreferrer">https://mdb.link/practice-with-php-event</a>
MongoDB Community Forum: <a href="https://mdb.link/practice-with-php-community" target="_blank" rel="noreferrer">https://mdb.link/practice-with-php-community</a>

Videos

Featured Shows

Semantic Search Made Easy With LangChain and MongoDB

Vector Search: The Future of Data Querying Explained | Semantic Searching

Practice with PHP: Mastering PHP/Laravel with MongoDB Atlas

All Shows