Solution Overview

Dynamic pricing, the art of adjusting prices in real time based on market conditions, has become a crucial strategy for businesses aiming to maximize revenue and gain a competitive edge. To effectively implement dynamic pricing, a robust technological infrastructure is essential. This solution integrates MongoDB Atlas and Google Cloud Vertex AI to create a real-time dynamic pricing microservice. By utilizing Google Cloud Pub/Sub for rapid data ingestion, Vertex AI Notebooks and TensorFlow models analyze customer behavior to optimize pricing strategies. MongoDB Atlas serves as a flexible feature store for intricate pricing data, while Google Cloud's robust computational resources power complex calculations and hosting.

The outcome is a scalable and adaptable pricing system that delivers instant price adjustments based on the latest market intelligence. This integration enhances operational efficiency by effectively handling large datasets and complex pricing scenarios. Continuous model retraining guarantees ongoing accuracy and market competitiveness.

Reference Architectures

The architecture for a dynamic pricing microservice integrates MongoDB and Google Cloud Vertex AI to facilitate real-time data processing and responsive pricing adjustments. At the core, Google Cloud Pub/Sub handles the ingestion and distribution of customer behavior data, allowing for scalable and efficient message processing. This data is then cleaned and processed in Vertex AI Notebooks, where machine learning models are developed using TensorFlow to predict optimal prices based on patterns identified in historical data.

MongoDB Atlas serves as the central data repository and feature store, storing complex pricing data and supporting the machine learning models. The document model of MongoDB provides the flexibility needed to manage and update pricing data dynamically. Google Cloud Functions orchestrate the entire workflow, processing customer events, converting them into tensors, and ensuring that the model predictions are updated in real time in the MongoDB Atlas product catalog. In the architecture diagram, the blue data flow illustrates how customer event data is ingested into a Pub/Sub topic, leading to a push subscription that triggers the Cloud Function. This function transforms raw events into tensors and updates the predicted prices in the MongoDB product catalog.

This architectural approach allows for the isolation of raw event threads, enabling the development of various services that can react in real time for dynamic pricing or operate asynchronously for model training. By maintaining loose coupling between components, the system is resilient and avoids complete failures if one part experiences issues. Publishers and subscribers can independently continue processing their logic, ensuring a robust and flexible system that supports continuous operation and seamless updates.

Building the Solution

• Step 1: Setting up MongoDB Atlas
  • Sign in to MongoDB Atlas and create a new cluster.
  • Choose a region closest to your user base for optimal performance.
  • Configure security by:
    • Creating database users with specific roles.
    • Enabling IP access to secure the database connection.
    • Connect to your cluster using the connection string provided by Atlas for application integration.
• Step 2: Setting up Google Cloud:
  • Log into Google Cloud console and create a new project for your microservice.
  • Ensure that the necessary APIs are enabled for your project.
  • Install and initialize the Google Cloud CLI.
    • Authenticate with your Google Cloud account.
    • Set your project as the default.
• Step 3: Develop the microservice and model
  • Clone the repository and navigate to the dynamicPricing microservice directory.
  • Install required Python packages using pip.
  • Set up environment variables for MongoDB Atlas, Google Cloud credentials, and Pub/Sub.
  • Configure a Pub/Sub topic in Google Cloud.
  • Develop the pricing logic for the dynamicPricing service.
  • Use Vertex AI to create a new notebook for connecting to MongoDB and training the model.
  • Train a TensorFlow model by cleaning data, building the model, and training it.
  • Save and upload the trained model to Google Cloud Storage.
  • Register the model in Vertex AI for management and deployment.
• Step 4: Deploy a model to an endpoint:
  • Deploy the model:
    • Access the Vertex AI section in Google Cloud console.
    • Locate and select the model you want to deploy.
    • Click on "Deploy to endpoint" under the "Deploy & Test" tab.
    • Complete the deployment setup by configuring model settings and monitoring.
  • Retrieve endpoint ID:
    • Go to "Endpoints" in Vertex AI and note the Endpoint ID for use in Cloud Function configuration.
  • Configure Cloud Function:
    • Create a new Cloud Function in Google Cloud.
    • Set the function's trigger to the Pub/Sub topic you created.
    • Implement code in the function to preprocess event data, invoke the Vertex AI model for predictions, and update MongoDB with the results.
  • Prepare dependencies:
    • Ensure the requirements.txt file includes all necessary libraries for the Cloud Function.
  • Simulate customer events:
    • Use a provided script to generate synthetic customer behavior data.
    • Push this data to MongoDB and Pub/Sub to test the model and microservice integration.
  • Run the simulation:
    • Execute the script to validate the complete end-to-end system.

Advantage of MongoDB for modeling events:

Adding a tensor allows for an event-driven architecture with all features in a single collection, which can streamline data retrieval and processing but may lead to large, complex documents that require careful management to maintain performance. This flexibility enables MongoDB to house both operational data and a feature store within the same collection, facilitating integrated data analytics and real-time decision-making while potentially increasing storage costs and complexity in data management.

For more-detailed instructions on how to build your own dynamic microservice, please refer to our blog post, Building a Dynamic Pricing Microservice with Vertex AI and MongoDB Atlas.

Key Learnings

• Centralized feature store: MongoDB serves as a feature store, acting as a centralized repository specifically designed for storing, managing, and serving features for machine learning (ML) models. Its polymorphic capabilities enable the utilization of a single interface to represent various types of data. This implies that as new features are introduced or pricing models evolve, MongoDB can adeptly manage diverse data types within the same system. In the context of dynamic pricing, this capability facilitates the seamless incorporation of new pricing factors or variables without causing disruptions to existing data structures or operations.
• Scalability and efficiency: Google Cloud Pub/Sub can handle massive volumes of customer data efficiently, ensuring scalability for real-world applications. While this microservice simulates only 25 customer events every three seconds, Pub/Sub is capable of processing much larger data streams.
• Real-time price updates: Cloud functions trigger TensorFlow models to generate dynamic prices based on customer behavior. These generated prices are then inserted or updated (upserted) back into the product catalog collection in MongoDB. This enables real-time adjustments in the e-commerce application because the application's front end retrieves data directly from the same collection.

Technologies and Products Used

MongoDB developer data platform:

• MongoDB Atlas
• Atlas Collections
• Atlas Clusters

Partner technologies:

• Google Cloud
• Vertex AI
• Cloud Functions

Author

• Francesco Baldissera, MongoDB
• Sebastian Rojas Arbulu, MongoDB