Built With MongoDB: Milky Way AI

Siya Raj Purohit and Adam Hughes
May 26, 2021 | Updated: October 17, 2022
#BuiltWithMongoDB#Startups

Sagar Setu received his PhD in helicopter flight dynamics and has a fascination with deep learning and integration within the field of aerospace. However, helicopter flight dynamics is not what Sagar is involved in today.

Through Entrepreneur First, an international program that helps entrepreneurs launch companies, Sagar met Eunice Wong, a fellow aspiring entrepreneur, who introduced Sagar to the world of retail, which he calls a “fantastic playground” for any engineer to be involved in. The pair founded Milky Way AI, with Wong as CEO and Sagar as CTO.

Milky Way AI is designed to empower the largest retailers and brands globally with real-time visibility into how their products and their competitors’ products are being merchandised across thousands of stores.

In this edition of #BuiltWithMongoDB, we chat with Sagar about the ways Milky Way AI creates opportunities for retailers, his favorite MongoDB features, and how the competitive AI industry keeps him motivated.

MongoDB: What does the product look like now, and how does it work?

Sagar: Our flagship product is called InstaShelf. It’s a mobile app that we put in the hands of distributors and merchandisers. When there is a person stocking the shelves and we put the app in the hands of that person, they are able to snap a photo, which then goes through our proprietary computer vision engine. This generates a variety of insights that are valuable for both the distributor and the brands. We are looking into how we can share this same data with and make it equally valuable to the retailers.

We developed this product over the last year and launched our beta three months ago. Since then we have gotten quite good traction in terms of users in a number of countries that are deploying the product.

MongoDB: Let's talk about that traction. How far along are you?

Sagar: We started in January with a 15-store pilot for Kelloggs in Singapore. We have deployed across 150 stores now, and we are set to deploy across a few hundred more in Malaysia and the Philippines. By the end of next year, we hope to be in three more countries — just with Kelloggs. The typical number of users for each of our pilots is around 15 to 20 merchandisers visiting between 50 and 60 stores. In a typical audit, the user takes 10 to 15 photographs and our AI identifies what's on the shelf from these pictures, reporting on key metrics such as a brand’s share of shelf compared with a competitor brand, products that are out of stock, product placement compliance, and so forth.

MongoDB: What does your tech stack consist of?

Sagar: The web and mobile components of our solution are built using MongoDB, React Native, React, Node.js, and Python Flask. The computer vision pipeline is built on both TensorFlow and Pytorch. We use MongoDB for all our database requirements — transactional and analytical.

Our top criteria for choosing the tech stack were proven scalability and stability, and a wide developer pool. It was important in the early stages to keep the team lean and the product flexible, and the choice of MongoDB Atlas turned out to be a great one. The support for being schemaless was crucial in allowing us to stay nimble as we learned the nitty-gritty of the domain. With features such as triggers and BI Connector, we could orchestrate various components of the solution right from the Atlas GUI, saving us hundreds of working hours.

MongoDB: What are some of your favorite features in MongoDB?

Sagar: My favorite feature is the support with autoscaling, which is the primary concern if you’re building anything into production. I’ve never had to worry about that. I don’t even think about it; I have just turned the features on, and it allows so much creativity. With MongoDB Atlas, I have peace of mind.

MongoDB: What is something that you are learning right now?

Sagar: Learning is a constant, working in the field of AI. A wonderful plus point is you always have so much competition: there might be a paper tomorrow that basically undoes everything you’ve done — something comes out that is far superior to the method you just took two months to deploy. So I’m always reading, learning, and trying to improve our solutions.

MongoDB: What’s one of your favorite books?

Sagar: The Selfish Gene. It’s not exactly technical, but more on the scientific side. That’s more of my kind of read. I really like the thought process the book instills in you. It gives you an understanding of the world — the good, the bad, and learning not to take things personally.

Looking to build something cool? Get started with the MongoDB for Startups program.

← Previous

Why I Wrote the New MongoDB Aggregations Book

In early May 2021, I published my book, Practical MongoDB Aggregations, which I released electronically and free for anyone to read . I love the MongoDB database and the uniqueness and power of its aggregation framework to analyse and manipulate massive amounts of data intuitively and efficiently. The opportunity to share this passion with others spurred me to write the book, with which I aim to support developers, architects, data analysts, data engineers, and data scientists to better understand how to maximise their productivity and effectiveness when building aggregation pipelines, as well as how to optimise these pipelines. Like many people over the past year during the pandemic, I’ve struggled to keep myself occupied when not busy doing my day job. Hence, my book was born not just from a desire to improve people’s knowledge but as my pandemic project, written over many weekends, to stave off the boredom. I believe aggregation pipelines provide a powerful domain-specific language for data processing in a way I’ve not seen before in other data-oriented tools, languages, or standards. SQL is a good data query language that caters to some analytical use cases via “group-by/having” statements. However, it typically has to be paired with a procedural language (e.g., Oracle’s PL/SQL ) to encompass an ordered set of complex data transformation rules. In the big data world of Hadoop , I find the MapReduce approach is too complex to develop with efficiently. Higher-level tools like Spark help alleviate some of this. However, by the necessity of still having to be general-purpose and versatile, the amount of Spark code required to process data sitting in any type of database is still too high for my liking. Many ETL tools provide proprietary data transformation capabilities, but these have to cater to the lowest common denominator capabilities across all the different types of databases they interact with. For these reasons and from experience, I consider MongoDB Aggregations to be the best tool for processing large data sets because it combines performance with productivity. Nevertheless, I sense the aggregation framework is shrouded in mystery for many people, hence my desire to demystify it with this book. I believe I identified a knowledge gap that many users wanted to be filled. MongoDB Inc. provides excellent reference documentation about aggregations in the MongoDB Manual , and MongoDB University provides a tremendous free online training course on aggregations . What I felt was still to be addressed was an opinionated yet informed perspective on how best to assemble aggregation pipelines from the well-documented parts—something that points the way to achieve optimal productivity and performance, accompanied by fully formed example pipelines to help put these approaches into practice. I hope readers of my book will learn some new things of value and enjoy reading it. A good test of the relevance of my book, in time, will be if people come back to it repeatedly as they continue with their journey of developing aggregations. Read the book for free now!

May 25, 2021
Next →

MongoDB Database Observability: Integrating with Monitoring Tools

This post is the final in a three-part series on leveraging database observability. Welcome back to our series on Leveraging Database Observability! Our previous post showcased a real-world use case highlighting how MongoDB Atlas’s observability tools effectively tackle database performance challenges. Whether you’re a developer, DBA, or DevOps engineer, our mission is to empower you to harness the full potential of your data through our observability suite . Integrating Atlas metrics with your central enterprise observability tools can simplify your operations. By seamlessly working with popular observability tools, our approach helps teams streamline workflows and enhance visibility across systems. Integrating MongoDB Atlas with third-party monitoring tools MongoDB’s developer data platform combines all essential data services for building modern applications within a unified experience. Our purpose-built observability tools for Atlas environments offer automatic monitoring and optimization, guiding diagnostics tailored specifically for MongoDB. Additionally, we extend Atlas metrics into your existing enterprise observability stack, enabling seamless integration without replacing your current tools. This creates a consolidated, single-pane view that unifies Atlas telemetry with other tech and application metrics, ensuring comprehensive visibility into both database and full-stack performance. This integration empowers you to monitor, receive alerts, and make data-driven decisions within your existing workflows, driving greater efficiency. Below is a quick guide to modifying integration settings through the Atlas UI and the popular integrations we support: Navigate to the Project Integrations page in Atlas. Choose the organization and project you want to configure from the navigation bar. On the Project Integrations page, select the third-party services you’d like to integrate. Configure the chosen services with the required API keys and regions. Critical integrations for your observability platform With Atlas’s Datadog and Prometheus integrations, you can send critical MongoDB metrics to these platforms, empowering detailed, real-time monitoring. Through Datadog , you can track database operation counts, query efficiency, and resource usage, ideal for pinpointing bottlenecks and managing resources. Similarly, Prometheus enables you to monitor essential metrics like query times, connection rates, and memory usage, supporting flexible tracking of database health and performance. Both integrations facilitate proactive detection of issues, alert configuration for resource thresholds, and a cohesive view of Atlas data when visualized in Grafana. Atlas’s integration with PagerDuty streamlines incident management by sending metrics like performance alerts, billing anomalies, and security events directly to PagerDuty. This integration records incidents automatically, notifies teams upon alerts, and supports two-way syncing, ensuring resolved alerts in Atlas are reflected in PagerDuty. It enables efficient incident response and resource allocation to maintain system stability. With Atlas integrations for Microsoft Teams and Slack, you can route key metrics—such as query latency, disk usage, and throughput—to these channels for timely updates. Teams can use these insights for real-time performance monitoring, incident response, and collaboration. Notifications through these platforms ensure your team stays informed on database performance, storage health, and user activity changes as they occur. Use case: Centralized observability with MongoDB Atlas, Datadog, and Slack Let’s walk through a hypothetical scenario for ShopSmart, an e-commerce company that leverages MongoDB Atlas to manage its product catalog and customer data. As traffic surges, the DevOps team faces challenges in monitoring application performance and database health effectively. To tackle these challenges, the team leverages MongoDB Atlas’ integration with Datadog and Slack, creating a powerful observability ecosystem. Integrating MongoDB Atlas with Datadog: The team pushes key MongoDB Atlas metrics into Datadog, such as query performance, connection counts, and Atlas Vector Search metrics. With Datadog, they can visualize these metrics and correlate overall MongoDB performance with their other applications. Out-of-the-box monitors and dedicated dashboards allow the team to track metrics like throughput, average read/write latency, and current connections. This visibility helps pinpoint bottlenecks in real time, ensuring optimal database performance and improving overall application responsiveness. Setting up alerts in Datadog: The team configures alerts for critical metrics like high query latency and increased error rates. When thresholds are breached, Datadog instantly notifies the team. This proactive approach allows the team to address potential performance issues before they impact customers. Integrating Datadog with Slack: To ensure fast communication, alerts are sent directly to the dedicated Slack channel, “ShopSmart-Alerts.” This integration fosters seamless collaboration, enabling the team to discuss and resolve issues in real-time. With these integrations, ShopSmart’s engineering team can monitor performance quickly and address issues efficiently. The unified observability approach enhances operational efficiency, improves the customer experience, and supports ShopSmart’s competitive edge in the e-commerce industry. By leveraging MongoDB Atlas, Datadog, and Slack, the team ensures scalable performance and drives continuous innovation. Conclusion MongoDB Atlas empowers developers and organizations to achieve unparalleled observability and control over their database environments. By seamlessly integrating with central enterprise observability tools, Atlas enhances your ability to monitor performance metrics and ensures you can do so within your existing workflows. This means you can focus on building modern applications confidently, knowing you have the insights and alerts necessary to maintain optimal performance. Embrace the power of MongoDB Atlas and transform your approach to database management—because your applications can thrive when your data is observable. And that wraps up our Leveraging Database Observability series! We hope you learned something new and found value in these discussions. Sign up for MongoDB Atlas , our cloud database service, to see database observability in action. To dive deeper and expand your knowledge, check out this learning byte for more insights on the MongoDB observability suite and how it can enhance your database performance.

November 14, 2024