MongoDB

Thibaut Gourdel

Senior Technical Product Marketing Manager

DeepSeek’s latest release has taken the AI ecosystem by storm, and for good reason. It has introduced state-of-the-art models and open-sourced them, making frontier AI more accessible than ever. One of these models is a reasoning model comparable to those from the best AI labs—until now, such standalone and open-weight models were rare. As if that weren’t enough, DeepSeek claims to have trained these models at a fraction of the cost of the leading model counterparts, challenging the economics of the industry. 

In this article, we will explore what makes DeepSeek’s models unique and how they can be leveraged locally for enhanced retrieval-augmented generation (RAG) with MongoDB.

The full code for the application in this tutorial is located in the [GenAI Showcase Repository](https://github.com/mongodb-developer/GenAI-Showcase/tree/main/apps/local-rag-pdf).

## Why do DeepSeek’s models stand out?

DeepSeek has three models:
* DeepSeek-V3: General-purpose LLM on-par with GPT-4o and Claude 3.5 Sonnet
* DeepSeek Coder: Specialized LLM focused on generating and completing code
* DeepSeek-R1: Reasoning model comparable to OpenAI-o1

The DeepSeek models are unique in several respects. Let’s start with DeepSeek-V3, which is praised for its scalability and cost-efficiency compared to alternatives due to its architecture of Mixture of Experts (MoE). This means that out of its 671 billion parameters, only 37 billion are activated at a time, depending on the tasks. For this reason, DeepSeek-V3 represents a true cost-efficient alternative to frontier lab models for handling general tasks. This model is the result of multiple optimizations and innovative techniques, including the use of knowledge distillation from another model, DeepSeek-R1, during post-training. 

In simple terms, the researchers developed a novel method to enhance a standard model like DeepSeek-V3 by transferring reasoning skills from another reasoning model, DeepSeek-R1. As a result, DeepSeek-V3 becomes significantly better at reasoning while maintaining its general-purpose efficiency and practical usability. This leads us to the DeepSeek-R1 family of models, marking the first generation of reasoning models developed by this AI lab. Like its standard V3 counterpart, R1 is built on an MoE architecture, activating 37 billion parameters out of a total of 671 billion. What sets reasoning models apart is their ability to use chain-of-thought (CoT) “thinking” before generating outputs, significantly improving performance and accuracy in solving logical problems. In addition to open-sourcing the models, [the DeepSeek team transparently provided the different methods and techniques](https://arxiv.org/abs/2501.12948) behind their technical breakthroughs. 

Notably, they successfully trained a model to reason step by step without relying on massive supervised datasets (which are expensive to produce). Instead, they used reinforcement learning with an innovative reward model, enabling the model to autonomously refine its reasoning and allocate the needed computation time based on the query. But that’s not all. While DeepSeek-R1 is a large-scale reasoning model, the team also applied its distillation technique (as described above) to other smaller open-source models, namely Qwen and Llama. By leveraging samples generated by DeepSeek-R1, they fine-tuned smaller models to mimic its reasoning process, significantly enhancing performance. According to DeepSeek benchmarks, these distilled models show remarkable results, with DeepSeek-R1-Distill-Qwen-32B achieving performance comparable to OpenAI-mini-o1. The distilled models, ranging from 1.5 billion to 70 billion parameters, have also been open-sourced by DeepSeek and unlock new use cases by leveraging state-of-the-art reasoning capabilities for a fraction of the cost locally or by self-hosting.

## Local RAG with a reasoning model

Retrieval-augmented generation is an AI approach that combines document retrieval with generative models to generate more accurate and contextually relevant responses. To learn more about RAG, see [Retrieval-Augmented Generation (RAG) with Atlas Vector Search.](https://www.mongodb.com/docs/atlas/atlas-vector-search/rag/#std-label-ai-key-concepts)

A local RAG solution, where the various components of the AI stack run on a local machine or, more broadly, on-premises, is cost-effective and allows all operations to be run locally. 

In many cases, such self-hosted solutions are essential, particularly in regulation-heavy industries like healthcare and defense. These solutions address critical security and privacy concerns and provide greater control over the overall infrastructure for organizations that still run their applications on-premises. By releasing more performant and efficient models, DeepSeek opens new avenues for applications, including powerful on-device applications that can run on consumer hardware such as Macbook M1+.In the case of integrating models like DeepSeek-R1 distilled models, using a reasoning model can enhance problem-solving capabilities over technical or research subjects. Moreover, unlike proprietary models such as the OpenAI-o1 family, the DeepSeek-R1 family shows the chain of thought it used to arrive at its conclusions, providing greater explainability and transparency in its outputs.

## Implementing local RAG with MongoDB and DeepSeek

Let’s now dive into an implementation example. This tutorial aims to build a mini-application that allows you to upload your own PDF files, including your private or sensitive documents, and interact with them through questions and answers. To meet local requirements, the entire stack will be deployed locally using:
* **Ollama** for running the models, Nomic-Text-Embed for the embedding model, and DeepSeek-R1-Distill-8B (depending on your machine, you could go up to the 24B or 32B).
* **A local deployment of MongoDB Atlas** for storing documents and vectors, as well as performing vector search.
* **LangChain** to orchestrate the back end and LLMs.
* **Streamlit** to create the user interface.
* **Data**: Any PDF document.

![Overview diagram illustrating the architecture of a local RAG application, integrating Nomic for embeddings, DeepSeek-R1 as the LLM (running locally via Ollama), Streamlit for the chat interface, LangChain for retrieval, and MongoDB Atlas as the vector database.][1]

While this tutorial features a local-first deployment, you have the option to either create a local Atlas deployment by using the [Atlas CLI](https://www.mongodb.com/docs/atlas/api/#std-label-atlas-programmatic-access-cli-overview) or [deploy a cluster](https://www.mongodb.com/docs/atlas/tutorial/create-new-cluster/#std-label-create-new-cluster) on the cloud. The Atlas CLI is the command-line interface for MongoDB Atlas, and you can use the Atlas CLI to interact with Atlas from the terminal for various tasks, including creating local Atlas deployments. To learn more, see [Manage Local and Cloud Deployments from the Atlas CLI.](https://www.mongodb.com/docs/atlas/cli/current/atlas-cli-local-cloud/)Additionally, the different models used in this tutorial are downloaded and run using [Ollama](https://ollama.com/), an open-source ecosystem for local LLM development.

### Prerequisites

To complete this tutorial, you must have the following: 

* The [Atlas CLI](https://www.mongodb.com/docs/atlas/cli/current/) installed and running v1.14.3 or later
* [MongoDB Command Line Database Tools](https://www.mongodb.com/try/download/database-tools) installed
* [Ollama](https://ollama.com/download) installed and running
* Python v3.10 or later

### Set up the environment

For this tutorial, you’ll want to create a directory where you can save the different files—for example, local-rag-mongodb.
Then, download the following two models from Ollama:

```
ollama pull nomic-embed-text
ollama pull deepseek-r1:8b
```

### Create a local deployment by using the Atlas CLI

Run the following command and follow the prompts to create a local deployment.
```
atlas deployments setup
```
![atlas deployment setup][2]

For detailed instructions, see [Create a Local Atlas Deployment.](https://www.mongodb.com/docs/atlas/cli/current/atlas-cli-deploy-local/#create-a-local-atlas-deployment-1)

Alternatively, you can also use Docker to run a local deployment of MongoDB Atlas:
```
docker run -d -e JAVA_TOOL_OPTIONS="-XX:UseSVE=0" -p 27017:27017 mongodb/mongodb-atlas-local
```

Once running, you can connect to your deployment using:
```
atlas deployments connect
```

Multiple options are available to connect and manage deployments, including MongoDB Compass for visually interacting with MongoDB Atlas deployments. For more commands to manage Atlas deployments, see [Manage a Local Atlas Deployment](https://www.mongodb.com/docs/atlas/cli/current/atlas-cli-deploy-local/#manage-a-local-atlas-deployment).

Then, using either MongoDB Shell or MongoDB Compass, let’s create a database, `knowledge_base`, and a collection, `documents`.

Using MongoDB Shell, run the mongosh command, which should automatically find and connect to your local deployment.

```
mongosh
```
```
> use knowledgebase
switched to db knowledgebase
> db.createCollection("documents")`
```
This step is optional since both the database and collection are created automatically when the first document is added.

### Run the RAG application

Once the environment is set up, and with the models downloaded locally using Ollama and MongoDB Atlas running locally, we can proceed to run the application. 

As we noted in the initial list of components required, the application is developed using LangChain for orchestrating LLM requests and integrated with MongoDB Atlas Vector Search and Streamlit for the interface.


The full code is available in the [GitHub repository](https://github.com/mongodb-developer/GenAI-Showcase/tree/main/apps/local-rag-pdf) for you to run. 

```
git clone https://github.com/mongodb-developer/GenAI-Showcase.git cd GenAI-Showcase/apps/local-rag-pdf`
```

First, we recommend running the application within a Python virtual environment to isolate the project from the global Python environment. 
You can use the ``` venv``` package, which comes with your Python installation. 
```
python -m venv local-rag-env
source local-rag-env/bin/activate
```

Depending on your operating systems, the Python executable might be `python3` instead of `python`.

Then, install the required libraries:
```pip install -r requirements.txt```

Once installed, you can launch the application:
```streamlit run app.py```

Upload your own PDF or download the [DeepSeek-R1 research paper](https://arxiv.org/pdf/2501.12948) as an example, and start asking questions directly about its content:

![Streamlit interface of a local RAG application powered by MongoDB Atlas and DeepSeek showing a PDF upload and a chat history displaying user queries and responses][3]

### Key components explained
#### 1. Initialization

The main parameters are defined in the config.yaml file:
```
llm_model: "deepseek-r1:8b"
embedding_model: "nomic-embed-text"
mongo_connection_str: "mongodb://localhost:27017/?directConnection=true"
database_name: "knowledge_base"
collection_name: "documents"
```

The application is then initialized by retrieving the configuration file variables, such as the different models used via Ollama—here, [nomic-embed-text](https://ollama.com/library/nomic-embed-text) for the embedding model and [deepseek-r1:8b](https://ollama.com/library/deepseek-r1) for the large language model, as well as the MongoDB connection information, including the database name and collection name.

```python
config = load_config(config_file)

 # Read values from config
 llm_model = config["llm_model"]
 embedding_model = config["embedding_model"]
 mongo_connection_str = config["mongo_connection_str"]
 database_name = config["database_name"]
 collection_name = config["collection_name"]

```

The splitting parameters for chunking documents and the prompt used to request the LLM are as follows:

```python
self.text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=100)
self.prompt = ChatPromptTemplate.from_template(
            """
            You are a helpful assistant answering questions based on the uploaded document and the conversation.
            Conversation History:
            {conversation_history}
            Context from Documents:
            {context}
            Question:  {question}
            """
```

Then, the vector store connection and its options are set, followed by the creation of the vector search index. 
This is needed to perform vector search queries against the vector embeddings based on semantic similarity.

```python
# Initialize the vector store with MongoDB Atlas
self.vector_store = MongoDBAtlasVectorSearch(
     collection=self.collection,
     embedding=self.embeddings,
     index_name="vector_index",
     relevance_score_fn="cosine"
)
self.vector_store.create_vector_search_index(dimensions=768)
```

#### 2. Data preparation

```python
docs = PyPDFLoader(file_path=pdf_file_path).load()
chunks = self.text_splitter.split_documents(docs)
```

PDF documents are read using LangChain PyPDFLoader and split according to the parameters chunk size and overlap size defined during initialization. Finally, the documents, along with their embeddings, are stored in MongoDB Atlas.


#### 3.  Question answering 

The last step is to send the user prompt alongside the retrieved documents based on the relevancy of the query. First, embeddings for the query are generated so that the vector search is performed and relevant documents are retrieved.

```python
# Generate and log query embeddings
query_embedding = self.embeddings.embed_query(query)

retrieved_docs = self.retriever.invoke(query)
```
The number of documents retrieved and the similarity score threshold are set to 5 and 0.2 by default; feel free to tweak them and assess the results.
The main prompt is structured using LangChain, including the conversation history, retrieved context documents, and the user prompt.

```python
# Format the input for the LLM, including conversation history
formatted_input = {
"conversation_history": "\n".join(conversation_history) if conversation_history else "",
"context": "\n\n".join(doc.page_content for doc in retrieved_docs),
"question": query,
}

# Build the RAG chain
chain = (
RunnablePassthrough()   # Passes the input as-is
     | self.prompt           # Formats the input for the LLM
     | self.model            # Queries the LLM
     | StrOutputParser()     # Parses the LLM's output
)

```
## Conclusion

This detailed guide has explored the latest DeepSeek models, including the DeepSeek-R1 family of reasoning models and their smaller distilled versions, which have shown impressive performance while running on relatively modest consumer hardware. 

These models not only deliver exceptional results given their minimal footprint, but they also democratize access to advanced reasoning with some of the first open-weight, high-performance reasoning models available. These advancements are a testament to the highly competitive nature of foundational models and set the stage for a rapid pace of innovation in the near future. Notably, this progress paves the way for the deployment of local-first RAG applications leveraging both MongoDB as the knowledge base and these advanced reasoning models to achieve performance comparable to frontier lab models. This tutorial demonstrated how accessible and practical it is to implement a local solution with DeepSeek and MongoDB, even on consumer-grade hardware.

### Learn more
Learn how MongoDB Atlas takes RAG applications to the next level by exploring advanced RAG and agentic systems in MongoDB's expert webinar: [Expanding LLM Applications: From RAG to Agentic Systems](https://www.mongodb.com/resources/solutions/use-cases/webinar-expanding-llm-applications-from-rag-to-agentic-systems)


  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt5447f5dd3852656b/679d7dce4405de0bc44af0df/unnamed_(4).png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blta7540eecdd5b2473/679d7e154bee0b5c61183b68/unnamed_(5).png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blte117df95516b81ce/679d7e6c4bee0b3888183b6e/unnamed_(6).png

Discover how DeepSeek’s innovative open-source models—DeepSeek-V3, DeepSeek Coder, and the groundbreaking reasoning model DeepSeek-R1—are revolutionizing local-first, retrieval-augmented generation (RAG) applications with MongoDB. This comprehensive guide details how cutting-edge chain-of-thought reasoning and model distillation techniques enhance AI performance and explainability. Learn how integrating tools like Ollama, Nomic-Text-Embed, LangChain, and Streamlit with MongoDB Atlas enables secure, scalable, and privacy-focused deployments ideal for industries such as healthcare and defense. Unlock advanced technical insights and step-by-step implementation strategies to build dynamic on-device AI applications that process and analyze sensitive documents with unparalleled accuracy and flexibility.

Thibaut Gourdel

Content by Thibaut Gourdel

Local-first and Reasoning-enhanced RAG With DeepSeek and MongoDB