Introducing Semantic Caching and a Dedicated MongoDB LangChain Package for Gen AI Apps

Prakul Agarwal, Erick Friis, and Jacob Lee
March 20, 2024 | Updated: October 25, 2024
#genAI

We are in an unprecedented time in history where developers can build transformative AI applications quickly, without being AI experts themselves. This ability is enabling new classes of applications that can better serve customers with conversational AI for assistance and automation, advanced reasoning and analysis using AI-powered retrieval, and recommendation systems.

Behind this revolution are large language models (LLMs) that can be prompted to solve for a wide range of use cases. However, LLMs have various limitations, like knowledge cutoff and a tendency to hallucinate. To overcome these limitations, they must be integrated with proprietary enterprise data sources to build reliable, relevant, and high-quality generative AI applications. That’s where MongoDB plays a critical role in the modern generative AI stack.

Developers use MongoDB Atlas Vector Search as a vital part of the generative AI technique known as retrieval-augmented generation (RAG). RAG is the process of feeding LLMs the supplementary data necessary to ground their responses, ensuring they're dependable and precise. LangChain has been a critical part of this journey since the public launch of Atlas Vector Search, enabling developers to build better retriever systems powered by vector search and store conversation history in the operational database.

Today, we are excited to announce support for two enhancements:

  1. Semantic cache powered by Atlas vector search, which improves the performance of your apps

  2. A dedicated LangChain-MongoDB package for Python and JS/TS developers, enabling them to build advanced applications even more efficiently

The MongoDB Atlas integration with LangChain can now power all the database requirements for building modern generative AI applications: vector search, semantic caching (currently only available in Python), and conversation history.

Earlier, we announced the launch of MongoDB LangChain Templates, which enable the developers to quickly deploy RAG applications, and provided a reference implementation of a basic RAG template using MongoDB Atlas Vector Search and OpenAI and a more advanced Parent-document Retrieval RAG template using MongoDB Atlas Vector Search. We are excited about our partnership with LangChain and will continue innovating.

Improve LLM application performance with semantic cache

Semantic cache improves the performance of LLM applications by caching responses based on the semantic meaning or context within the queries themselves. This is different from a traditional cache that works based on exact keyword matching. In the era of LLM the value of semantic cache is increasing tremendously, enabling sophisticated user experiences that closely mimic human interactions. For example, if two different users enter two different prompts, “give me suggestions for a comedy movie” and “recommend a comedy movie”, the semantic cache can understand that the intent behind the queries are same and return a similar response, even though different keywords are used, whereas a traditional cache will fail.

Diagram of using semantic cache using MongoDB Atlas Vector Search. Before using cache, each use query sends to LLM, which takes up time and money. With the cache, the system checks if the query has already been processed by the LLM and only new queries are sent to the LLM. Saving time and money
Figure 1: Semantic cache using MongoDB Atlas Vector Search

Check out this video walkthrough for the semantic cache:

Accelerate development with a dedicated package

With a dedicated LangChain-MongoDB package, MongoDB is even more deeply integrated with LangChain. The Python and Javascript packages contain the following LangChain Integrations: MongoDBAtlasVectorSearch (Vector stores) and MongoDBChatMessageHistory (Chat Messages Memory). In addition, the Python package includes the MongoDBAtlasSemanticCache (LLM Caching).

The new package langchain-mongodb contains all the MongoDB-specific implementations and needs to be installed separately from langchain, which includes all the core abstractions. Earlier, everything was in the same package, making it challenging to correctly version and communicate what version should be used and whether any breaking changes were made.

Find out more about the langchain-mongodb package:

Get started today

  • Check out this accompanying tutorial and notebook on building advanced RAG with MongoDB and LangChain, which contains a walkthrough and use cases for using semantic cache, vector search, and chat message history.

  • Check out the “PDFtoChat” app to see langchain-mongodb JS in action. It allows you to have a conversation with your proprietary PDFs using AI and is built with MongoDB Atlas, LangChain.js, and TogetherAI. It’s an end-to-end SaaS-in-a-box app and includes user authentication, saving PDFs, and saving chats per PDF.

  • Read the excellent overview of semantic caching using LangChain and MongoDB.

← Previous

New Data Modeling Learning Path and Certification

This post is also available in: Deutsch , Français , Español , Português , Italiano , 한국어 , 简体中文 . Data modeling is a crucial part of software development. It's what gives structure to the data so that it can be analyzed, used to make decisions, and built into useful applications. It can be complex and challenging, especially when you're juggling performance and maintainability. But here's the good news: a well-designed data model can boost your app's performance. That's why we've put together a new learning path and certification to help you level up your data modeling skills. Whether you're a seasoned developer looking to deepen your understanding or a newcomer eager to learn, this content is designed to give you the tools you need to create efficient and effective data models. Let's dive in and start building better apps together. Deep dive into data modeling with MongoDB The new free, online MongoDB Data Modeling Path will help you build your knowledge from the ground up and prepare you for the certification exam. The curated sequence of videos, hands-on labs, and quizzes provides you with a guided journey to learning data modeling at your own pace. The learning path will show you a step-by-step method for creating effective data models. You'll learn how to identify entities and workloads, map relationships, and when to embed or reference them. In addition, you'll learn different schema design patterns, how to recognize and address MongoDB anti-patterns, and the basics of schema lifecycle management. And, here's a little something extra: Once you complete the learning path, you'll automatically score a 50% discount on the new certification ! Learn fast, certify faster With an extensive array of new educational resources on data modeling for MongoDB, we are thrilled to introduce our new credential, the MongoDB Associate Data Modeler certification , which will allow you to validate and showcase your expertise in the field. Our certifications are officially recognized by professional institutions, validating and acknowledging your MongoDB expertise. They're a valuable asset for advancing in your role and enhancing your marketability for future positions. Certified individuals earn bragging rights, inclusion in the Credly Talent Directory , and a distinguished Credly badge, making it easy to share your achievements. The MongoDB Associate Data Modeler certification is designed for experienced users. Candidates should have some familiarity with JSON, and the MongoDB Query API , including aggregations and data modeling on MongoDB. You should also understand the tradeoffs of simplicity versus performance of data modeling techniques. This certification attests to your competency in designing, building, and evolving effective data models in MongoDB, and incorporating data governance into your work. Explore MongoDB’s new learning path and certification designed to elevate and validate your data modeling skills.

March 20, 2024
Next →

MongoDB and Partners: Building the AI Future, Together

If you’re like me, over the past year you’ve closely watched AI’s developments—and the world’s reactions to them. From infectious excitement about AI’s capabilities, to impatience with its cost and return on investment, every day has been filled with AI twists and turns. It’s been quite the roller coaster. During the ride, from time to time I’ve wondered where AI falls on the Gartner hype cycle, which gives "a view of how a technology or application will evolve over time." Have we hit the "peak of inflated expectations" only to fall into the "trough of disillusionment?" Or is the hype cycle an imperfect guide, as The Economist argues? The reality is that it takes time for any new technology—even transformative ones like AI—to take hold. And every advance, no matter how big, has had its detractors. A famous example is that of Picasso (!), who in 1968 said, “Computers are useless. They can only give you answers.” (!!) For our part, MongoDB is convinced that AI is a once-in-a-generation technology that will enhance every future application—a belief that has been reinforced by the incredible work our partners have shared at MongoDB’s 2024 events. Speeding AI development MongoDB is committed to helping organizations of all sizes succeed with AI, and one way we’re doing that is by collaborating with the MongoDB partner ecosystem to create powerful, user-friendly AI development tools and solutions. For example, Fireworks.ai —which is a member of the MongoDB AI Applications Program ecosystem —created an inference solution that hosts gen AI models and supports containerized deployments. This tool makes it easier for developers to build and deploy powerful applications with a range of easy-to-use tools and customization options. They can choose to use state-of-the-art, open-source language, image, and multimodal foundation models off the shelf, or they can customize and fine-tune models to their needs. Jointly, Fireworks.ai and MongoDB provide a solution for developers who want to leverage highly curated and optimized open-source models and combine these with their organization’s own proprietary data—and to do so with unparalleled speed and security. “MongoDB is one of the most sophisticated database providers, and it’s very easy to use,” said Benny Chen , cofounder of Fireworks.ai. "We want developers to be able to use these tools, and we want to work with providers who enable and empower developers." Nomic , another MAAP ecosystem member, also enables developers with best-in-class solutions across the entire unstructured data workflow. Their Embed offering, available through the Nomic API , allows users to vectorize large-scale datasets for use in text, image, and multimodal retrieval applications, including retrieval-augmented generation (RAG), using only their web browser. The Nomic-MongoDB solution is a highly efficient, open-weight model that developers can use to visualize the unstructured datasets they store in MongoDB Atlas . These insights help users quickly discover trends and articulate data-driven value propositions. Nomic also supported the recently announced vector quantization in MongoDB Atlas Vector Search , which reduces vector sizes while preserving performance. Last—but hardly least!—there’s our new reference architecture with MAAP partners AWS and Anthropic. Announced at MongoDB.local London , the reference architecture supports building memory-enhanced AI agents, and is designed to streamline complex processes and develop smarter, more responsive applications. For more—including a link to the code on Github— check out the MongoDB Developer Center . Making AI work for anyone and everyone The companies MongoDB partners with aren’t just making gen AI easier for developers—they’re building tools for everyone. For example, Capgemini has invested $2 billion in gen AI and is training 100,000 of its employees in the technology. GenYoda, a solution that helps insurance professionals with their daily work, is a product of this investment. GenYoda leverages MongoDB Atlas Vector Search to analyze large amounts of customer data, like policy statements, premiums, claims history, and health information. Using GenYoda, insurance professionals can quickly analyze underwriters’ reports to make informed decisions, create longitudinal health summaries, and streamline customer interactions to improve contact center efficiency. GenYoda can ingest 100,000 documents in just a few hours and respond to users’ queries in two to three seconds—a metric on par with the most widely used gen AI models. And it produces results: in one example, by using Capgemini’s solution an insurer was able to increase productivity by 15%, add new reports 25% faster (thus speeding decision-making), and reduce the manual effort of searching PDFs, increasing efficiency by 10%. Building the future of AI together So, what’s next? Honestly, I’m as curious as you are. But I’m also incredibly excited. At MongoDB, we’re active participants in the AI revolution, working to embrace the possibilities that lie ahead. The future of gen AI is bright, and I can’t wait to see what we’ll build together. To learn more about how MongoDB can accelerate your AI journey, explore the MongoDB AI Applications Program .

November 4, 2024