Zomato Manages High-Volume Data and Delivers High-Speed Success

MongoDB
December 6, 2023 | Updated: December 8, 2023

Zomato is one of India’s largest consumer technology companies. The restaurant aggregator and food delivery operator provides restaurant information, menus and user reviews, and food delivery options from partner restaurants in over 1,000 Indian cities and towns.

With over 17.5 million customers, 220,000 restaurant partners and 350,000 delivery partners, Zomato deals with data in huge quantities. However, in 2017 Zomato became increasingly aware of the pain points of database management.

“Zomato was expanding at a remarkable pace,” says Abhishek Jain, senior software engineer, Zomato at MongoDB.local New Delhi earlier this year. “We realized that it would be prudent to migrate all our self-hosted clusters to a managed platform, which is when we integrated with MongoDB Atlas. As Zomato and its portfolios kept on expanding, so did our use cases for MongoDB.”

Using features such as index suggestions, geospatial queries and analytics nodes, MongoDB is now the driving force behind many of Zomato’s key operational systems:

  • The Order Tracking system covers live order statuses as well as live locations of delivery partners.

  • The Order Assignment system is one of the most critical components, and one of the most complex.

  • The Order Details system displays items and quantities for each order to ensure smooth fulfillment.

  • The Delivery Partner Expected Earnings feature combines key information such as a rider’s location, their distance from the partner restaurant and customer, and the type of transport used.

  • The Featured Restaurants system is what most people see when using Zomato app. A key marketing tool, it creates bespoke campaigns and offers, and suggests local restaurants to users.

  • The Delivery Partner Onboarding system simplifies the onboarding process by collecting and storing drivers’ names, addresses, driving licenses, vehicle type and more.

Another critical tool, the Delivery Partner Location system, is central to Zomato. With thousands of riders on the streets of India continuously publishing their locations at very high rates, flexible and responsive persistent storage is vital.

“We get hundreds of thousands of requests per minute,” he explains. “We need a database capable of handling this load without any issues or latency, even during extremely high-volume periods such as New Year’s Eve. Again, MongoDB was the answer.”

A final core element is Chat SDK, a central messaging hub that enables customers and delivery partners to stay in contact via email or text message and can include images and video.

“All this data is non-relational, so we need a NoSQL data store that provides strong consistency under heavy read and write workloads,” says Soumil Kanwal, Software Engineer at Zomato. “MongoDB's document-oriented database fits our requirements perfectly.”

He concludes: “We’re in the process of migrating features from other databases to MongoDB and building new features on top of those. MongoDB manages them all very comfortably.”

Check out the full presentation now

← Previous

阿里云数据库MongoDB版助力掌阅平滑上云，撬动数据红利

掌阅科技：深耕优质内容 优化数字阅读体验 掌阅科技股份有限公司成立于2008年9月，专注于数字阅读，是全球领先的数字阅读平台之一。 掌阅主营业务为互联网数字阅读服务及增值服务，同时从事网络原创文学版权运营，以及基于自有互联网平台的流量增值服务，服务覆盖自研产品、阅读服务、原创矩阵、网文出海、终端产品等五大方面。 掌阅的自研产品包括掌阅App、掌阅精选App、掌阅课外书App、得间小说App、阅爱聊等。其中，掌阅App是掌阅科技自研的数字阅读平台，拥有国学经典、严肃出版、原创文学、有声读物、漫画杂志等海量阅读内容，在业内率先实现了3D仿真翻页、护眼模式等技术的创新与引用，并在文档识别、转化、续读技术以及精装排版等方面形成了核心技术优势，处于行业领先水平。 业务挑战 数字阅读渐成主流 数据拓展需要数据管理同步升级 潜心数字阅读领域十余年，掌阅积累了海量阅读数据，如何做到这些数据的平滑升级和扩展、如何将这些数据充分利用起来并指导企业工作？是掌阅最大的痛点，也是掌阅多年来一直在研究和探索的问题。 依据阅读场景下的数据特点，掌阅在数据方面主要面对以下挑战： 双向数据 – 人和书这个关系相对复杂。如一个人喜欢哪些书、同时这些书又被哪些人同时喜欢，这在关键性数据库里需要建两张表，人一张表，书一张表。更为重要的是，要在两张表的数据关系中找到潜在读者、客户。这就需要一个能够有效管理非结构化数据的复合型索引。 字段的不确定性 – 在具体阅读的过程中，掌阅旗下不同APP要适应多重场景和运营活动，随着定位和画像的不同，数据标签随时更新、变化。传统的关系型数据库不再适合厘清这些不确定字段的数据信息。 数据升级 – 围绕掌阅的几大主营业务，涉及各个领域的APP矩阵不断扩大，数据量级已达到TB级，这种高并发海量数据需要数据管理的同步升级。 解决方案 阿里云数据库MongoDB版落地掌阅多元业务场景 大幅提升服务性能 2022年4月，掌阅科技在年报内显示将其核心服务进行云原生的改造，在大幅提升研发效率的同时，保证良好的服务可用性，持续提升服务性能30%以上。对客户端进行大幅包体瘦身和稳定性加固，崩溃率降低50%以上，冷启动打开速度提升30%以上，包体积降低30%以上。 这与掌阅选择阿里云数据库MongoDB版不无关系。掌阅平台研发部负责人张博表示，这主要是考虑到MongoDB在非结构化数据管理方面的显著优势，一是文档结构，字段可灵活扩展，可实现双向数据查询；二是支持Sharding、多节点容灾，实现海量数据的存储和可用性；三是事务支持，可实现多文档事务；四是阿里云的支持，为未来混合云架构打好基础。 张博还对阿里云数据库MongoDB版在云书架、书评、流水帐单、类积分管理四个业务场景中的落地实践进行了介绍。 云书架 – 每一位掌阅用户都会拥有一个个人书架，可以自助添加、删除书目。书、人之间会产生错综复杂的对应关系，而且这种对应数据呈持续变化，MongoDB介入其中进行动态管理。 书评 – 书评是阅读的一项“附加产品”。做书评是用户在掌阅APP上进行的重要交互行为，对每一本书、每一章节、每一句话，都可以上传个人感悟、评价。在平台后端形成了多事务多文档，这也是MongoDB善于处理的数据文档形态。 流水帐单 – 掌阅的流水数据很长一段时间内存在互联网数据中心 (IDC) 内的 HBase 系统 中，由于时间和历史原因，Hbase可维护性、可扩展性已到瓶颈阶段。作为掌阅与阿里云合作的第一个项目，掌阅将所有流水帐单从HBase迁移到MongoDB系统中，完成财务共享中心建设，突破瓶颈，轻松应对流水数据的不断增加。 类积分管理 – 积分机制是掌阅的一种重要“用户管理”方式，但多种积分活动相互交叉，积分规则各不相同，积分数值变化相对琐碎。大量积分数据在平台后端形成单事务多文档，MongoDB能够帮助实现积分管理清晰化、标准化。 客户价值 以数据为驱动 掌阅科技不断提升精细化能力 年报显示，掌阅科技2021年营收为20.7亿元，较上年同期的20.6亿元增长0.49%。其中，数字阅读平台收入占营业收入的比例分别为75.13%。可以说，以数据为驱动，掌阅不断提升精细化能力，运营效率进一步提高。聚焦在阿里云数据库MongoDB版为掌阅带来的重要价值，张博更喜欢用这样的三个词来概括：效率、成本、稳定性。 提高效率 – 在与阿里云和 MongoDB合作过程中，MongoDB技术团队会直接进入项目，并对重点问题全程追踪，通力解决问题，双方达成良性合作，提高工作效率。 加强稳定性 – 数据弹性大是云原生的特色能力。阿里云数据库MongoDB版为掌阅数据带来了稳定的数据弹性支持，尤其针对掌阅不同的运营活动、不同的爆款产品、不同的热点书目，随着数据量波动，MongoDB都能很好地帮助掌阅应对数据库扩容和缩容。 降低成本 – 效率的提高、稳定性的提高都可以直接带来成本降低。据掌阅统计，引入MongoDB系统后，运营成本节省10%以上。 客户证言 掌阅平台研发部负责人 张博 “阿里云数据库MongoDB版为我们提供了一系列操作工具，并作为我们研发平台上的重要组件，帮助我们在统一的、开放的研发环境下，不断提升研发成效。随着互联网行业格局的逐步成型以及数字阅读行业的市场竞争日益激烈，掌阅将与阿里云和MongoDB继续深度合作，变现技术红利，继续深化存量用户精细化运营和加大增量用户贡献。”

December 5, 2023
Next →

AI-Powered Call Centers: A New Era of Customer Service

Customer satisfaction is critical for insurance companies. Studies have shown that companies with superior customer experiences consistently outperform their peers. In fact, McKinsey found that life and property/casualty insurers with superior customer experiences saw a significant 20% and 65% increase in Total Shareholder Return , respectively, over five years. A satisfied customer is a loyal customer. They are 80% more likely to renew their policies, directly contributing to sustainable growth. However, one major challenge faced by many insurance companies is the inefficiency of their call centers. Agents often struggle to quickly locate and deliver accurate information to customers, leading to frustration and dissatisfaction. This article explores how Dataworkz and MongoDB can transform call center operations. By converting call recordings into searchable vectors (numerical representations of data points in a multi-dimensional space), businesses can quickly access relevant information and improve customer service. We'll dig into how the integration of Amazon Transcribe, Cohere, and MongoDB Atlas Vector Search—as well as Dataworkz's RAG-as-a-service platform— is achieving this transformation. From call recordings to vectors: A data-driven approach Customer service interactions are goldmines of valuable insights. By analyzing call recordings, we can identify successful resolution strategies and uncover frequently asked questions. In turn, by making this information—which is often buried in audio files— accessible to agents, they can give customers faster and more accurate assistance. However, the vast volume and unstructured nature of these audio files make it challenging to extract actionable information efficiently. To address this challenge, we propose a pipeline that leverages AI and analytics to transform raw audio recordings into vectors as shown in Figure 1: Storage of raw audio files: Past call recordings are stored in their original audio format Processing of the audio files with AI and analytics services (such as Amazon Transcribe Call Analytics ): speech-to-text conversion, summarization of content, and vectorization Storage of vectors and metadata: The generated vectors and associated metadata (e.g., call timestamps, agent information) are stored in an operational data store Figure 1: Customer service call insight extraction and vectorization flow Once the data is stored in vector format within the operational data store, it becomes accessible for real-time applications. This data can be consumed directly through vector search or integrated into a retrieval-augmented generation (RAG) architecture, a technique that combines the capabilities of large language models (LLMs) with external knowledge sources to generate more accurate and informative outputs. Introducing Dataworkz: Simplifying RAG implementation Building RAG pipelines can be cumbersome and time-consuming for developers who must learn yet another stack of technologies. Especially in this initial phase, where companies want to experiment and move fast, it is essential to leverage tools that allow us to abstract complexity and don’t require deep knowledge of each component in order to experiment with and realize the benefits of RAG quickly. Dataworkz offers a powerful and composable RAG-as-a-service platform that streamlines the process of building RAG applications for enterprises. To operationalize RAG effectively, organizations need to master five key capabilities: ETL for LLMs: Dataworkz connects with diverse data sources and formats, transforming the data to make it ready for consumption by generative AI applications. Indexing: The platform breaks down data into smaller chunks and creates embeddings that capture semantics, storing them in a vector database. Retrieval: Dataworkz ensures the retrieval of accurate information in response to user queries, a critical part of the RAG process. Synthesis: The retrieved information is then used to build the context for a foundational model, generating responses grounded in reality. Monitoring: With many moving parts in the RAG system, Dataworkz provides robust monitoring capabilities essential for production use cases. Dataworkz's intuitive point-and-click interface (as seen in Video 1) simplifies RAG implementation, allowing enterprises to quickly operationalize AI applications. The platform offers flexibility and choice in data connectors, embedding models, vector stores, and language models. Additionally, tools like A/B testing ensure the quality and reliability of generated responses. This combination of ease of use, optionality, and quality assurance is a key tenet of Dataworkz's "RAG as a Service" offering. Diving deeper: System architecture and functionalities Now that we’ve looked at the components of the pre-processing pipeline, let’s explore the proposed real-time system architecture in detail. It comprises the following modules and functions (see Figure 2): Amazon Transcribe , which receives the audio coming from the customer’s phone and converts it into text. Cohere ’s embedding model, served through Amazon Bedrock , vectorizes the text coming from Transcribe. MongoDB Atlas Vector Search receives the query vector and returns a document that contains the most semantically similar FAQ in the database. Figure 2: System architecture and modules Here are a couple of FAQs we used for the demo: Q: “Can you explain the different types of coverage available for my home insurance?” A: “Home insurance typically includes coverage for the structure of your home, your personal belongings, liability protection, and additional living expenses in case you need to temporarily relocate. I can provide more detailed information on each type if you'd like.” Q: “What is the process for adding a new driver to my auto insurance policy?" A: “To add a new driver to your auto insurance policy, I'll need some details about the driver, such as their name, date of birth, and driver's license number. We can add them to your policy over the phone, or you can do it through our online portal.” Note that the question is reported just for reference, and it’s not used for retrieval. The actual question is provided by the user through the voice interface and then matched in real-time with the answers in the database using Vector Search. This information is finally presented to the customer service operator in text form (see Fig. 3). The proposed architecture is simple but very powerful, easy to implement, and effective. Moreover, it can serve as a foundation for more advanced use cases that require complex interactions, such as agentic workflows , and iterative and multi-step processes that combine LLMs and hybrid search to complete sophisticated tasks. Figure 3: App interface, displaying what has been asked by the customer (left) and how the information is presented to the customer service operator (right) This solution not only impacts human operator workflows but can also underpin chatbots and voicebots, enabling them to provide more relevant and contextual customer responses. Building a better future for customer service By seamlessly integrating analytical and operational data streams, insurance companies can significantly enhance both operational efficiency and customer satisfaction. Our system empowers businesses to optimize staffing, accelerate inquiry resolution, and deliver superior customer service through data-driven, real-time insights. To embark on your own customer service transformation, explore our GitHub repository and take advantage of the Dataworkz free tier .

November 27, 2024