DocsGPT: Migrating One of the Industry’s Most Popular Open Source AI Assistants to Atlas Vector Search

Since its founding in 2019, Arc53 has focused on building predictive AI/ML solutions for its clients, with use cases ranging from recommendation engines to fraud detection. But it was with OpenAI’s launch of ChatGPT in November 2022 that the company saw AI rapidly take a new direction.

As Arc53 co-founder Alex Tushynski explains, “It was no surprise to see generative AI suddenly capture market attention. Suddenly developers and data teams were being challenged to bring their companies’ own proprietary data to gen AI models, in what we now call retrieval-augmented generation (RAG). But this involved them building new skills and disciplines. It wasn’t easy as they had to stitch together all of their different databases, data lakes, file systems, and search engines, and then figure out how to feed data from those systems into their shiny new vector stores. Then they had to orchestrate all of these components to build a complete solution. We identified an opportunity to abstract this complexity away from them. So DocsGPT was born.”

DocsGPT is an open-source documentation assistant that makes it easy for developers to build conversational user experiences with natural language processing (NLP) directly on top of their data. That can be a chatbot on a company website for customer support or as an interface into internal data repositories to help boost employee productivity.

Developers simply connect their data sources to DocsGPT to experiment with different embedding and large language models to optimize for their specific use case. LLM options currently include ChatGPT 3.5 and 4, along with DocsGPT-7B, based on Mistral.

In addition to the choice of models, developers can choose where they deploy DocsGPT. They can download the open source code to run in their own environment or consume DocsGPT as a managed service from Arc53.

The freedom developers enjoy with DocsGPT is reflected in its levels of adoption. Since its release last year, the project has accumulated close to 14,000 GitHub stars and built a vibrant community with over 100 independent contributors. Tushynski says, “DocsGPT counts the UK government’s Department of Work and Pensions, pharmaceutical industry solution provider NoDeviation, and nearly 20,000 other users.”

Tushynski and team selected MongoDB Atlas as the database for the DocsGPT managed service. “We’ve used MongoDB in many of our prior predictive AI projects. Its flexibility to store data of any structure, scale to huge data sets, and ease of use for both developers and data scientists means we can deliver richer AI-driven solutions faster. Using it to underpin DocsGPT was an obvious choice. As developers connect their documentation to DocsGPT, MongoDB stores all of the metadata, along with chat history and user account information.”

Migrating from Elasticsearch to MongoDB Atlas Vector Search

With the release of Atlas Vector Search, the DocsGPT team is now migrating its vector database from Elasticsearch into MongoDB Atlas. Tushynski says, “MongoDB is a proven OLTP database handling high read and write throughput with transactional guarantees. Bringing these capabilities to vector search and real-time gen AI apps is massively valuable. Atlas is able to handle highly dynamic workloads with rapidly changing embeddings in ways Elasticsearch cannot. The latency Elasticsearch exhibits as it merges updates into existing indexes means the app is often retrieving stale data, impacting the quality and reliability of model outputs.”

Tushynski goes on to say, “We’ve experimented with a number of standalone vector databases. There are some good technologies there, but again, they don’t meet our needs when working with highly dynamic gen AI apps. We often see users wanting to change embedding models as their apps evolve — a process that means re-encoding the data and updating the vector search index. For example, we’ve migrated our own default embedding models from OpenAI to multiple open-source models hosted on Hugging Face and now to BGE. MongoDB’s OLTP foundations make this a fast, simple, and hassle-free process.”

Tushynski discusses the importance of embedding models in his blog post, Amplify DocsGPT with optimal embeddings. The post includes an example of how one customer was able to improve measured user experience by 50% simply by updating their embedding model.

“One of the standout features of MongoDB Atlas in this context is its adeptness in handling multiple embeddings. The ability to link various embeddings directly with one or more LLMs without the necessity for separate collections or tables is a powerful feature," Tushynski says. "This approach not only streamlines the data architecture but also eliminates the need for data duplication, a common challenge in traditional database setups. By facilitating the storage and management of multiple embeddings, it allows for a more seamless and flexible interaction between different LLMs and their respective embeddings.”

Being part of AI Innovators program, the DocsGPT engineering team gets free Atlas credits as well as access to technical expertise to help support their migration. The AI Innovators program is open to any startup that is building AI with MongoDB.