Transform the Driver Experience with MongoDB & Google Cloud

On-board Components

These run in the vehicle, close to the driver, and enable real-time voice interaction.

• Car console: The in-car interface where users speak to the assistant and get responses. This demo uses a web application which represents the embedded system in a real vehicle.

• Local data storage: Vehicles store key signals locally using PowerSync SDK, a lightweight edge database built on SQLite. This ensures fast access to diagnostic data and keeps data synced with MongoDB Atlas.

• Assistant backend: This component manages the conversation. It handles voice transcription using Google Cloud Speech-to-Text. Depending on the query, it either responds directly or calls tools to fetch more data or take action. This demo includes four sample actions:

  • Consult manual: Uses Atlas Vector Search to retrieve relevant info from the car manual.

  • Run diagnostics: Fetches current diagnostic codes from local vehicle data.

  • Recalculate route: Adjusts the trip if the driver adds a stop.

  • Close chat: Ends the conversation gracefully.

This solution uses the following object to define tools for the assistant backend. The solution passes the object to Google Cloud when it starts the chat functionality.

const functionDeclarations = [
  {
    functionDeclarations: [
      {
        name: "closeChat",
        description:
          "Closes the chat window when the conversation is finished. By default it always returns to the navigation view. Ask the user to confirm this action before executing.",
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {
            view: {
              type: FunctionDeclarationSchemaType.STRING,
              enum: ["navigation"],
              description:
                "The next view to display after closing the chat.",
            },
          },
          required: ["view"],
        },
      },
      {
        name: "recalculateRoute",
        description:
          "Recalculates the route when a new stop is added. By default this function will find the nearest service station. Ask the user to confirm this action before executing.",
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {},
        },
      },
      {
        name: "consultManual",
        description:
          "Retrieves relevant information from the car manual.",
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {
            query: {
              type: FunctionDeclarationSchemaType.STRING,
              description:
                "A question that represents an enriched version of what the user wants to retrieve from the manual. It must be in the form of a question.",
            },
          },
          required: ["query"],
        },
      },
      {
        name: "runDiagnostic",
        description:
          "Fetches active Diagnostic Trouble Codes (DTCs) in the format OBD II (SAE-J2012DA_201812) from the vehicle to assist with troubleshooting.",
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {},
        },
      },
    ],
  },
];

Cloud Components

These components are stored in Google Cloud or MongoDB Atlas and provide AI intelligence, scalable storage, and data processing capabilities.

• Data ingestion: Unstructured content like car manuals is uploaded to Google Cloud Storage. This triggers a pipeline using Pub/Sub, Cloud Run, and Document AI to split the PDFs into chunks. Vertex AI generates embeddings for these chunks, which are then stored in MongoDB Atlas for semantic search.

• Serving speech APIs: Google Cloud's Text-to-Speech and Speech-to-Text handle natural voice interaction. Vertex AI provides text embeddings for search queries and powers Gemini, which is the LLM used by the assistant.

• Data storage and retrieval: MongoDB Atlas stores:

  • Manual chunk embeddings for retrieval via Atlas Vector Search.

  • User preferences and session data.

  • Vehicle signals—both latest values and full time series telemetry.

Atlas Vector Search is used to match user questions with the most relevant manual sections, enabling a Retrieval-Augmented Generation (RAG) flow. MongoDB's native support for structured, semi-structured, and vector data in one place simplifies the assistant logic and speeds up development.

• Data sync: This solution uses PowerSync for two-way sync between the vehicle and cloud:

  • Vehicle to cloud: Vehicle sends telemetry data like diagnostic codes, speed, or acceleration. A cloud run function processes and stores this in Atlas.

  • Cloud to vehicle: Enables updates or actions sent remotely to the car—like OTA updates or remote locking.

MongoDB in Conversational AI

MongoDB Atlas improves this solutions architecture in the following ways:

• Unifies operational and vector data: Vehicle signals, vector embeddings, and user sessions are stored together in a single platform.

• Enables more relevant responses: Atlas Vector Search retrieves the correct chunks from large documents instantly, fueling accurate and context-rich responses.

• Built for the enterprise scale: Whether it's one model or a global fleet, MongoDB Atlas offers built-in horizontal scalability, high-availability and enterprise-grade security.

• Simplifies edge and cloud sync: PowerSync and MongoDB work together to bridge in-car and cloud environments without friction.

This architecture is designed to scale, evolve, and adapt, just like the vehicles it supports. With MongoDB at the core, automakers can focus less on data plumbing and more on delivering smart, helpful in-car experiences that truly make a difference on the road.

Built for Innovation and Speed

The document model is designed for developers. MongoDB's flexible schema enables you to easily change and update your data model. As new vehicle features roll out or user expectations shift, teams can evolve the data model on the fly without expensive migrations or app downtime. Additionally, because each document is self-contained, queries are faster and simpler.

The Natural Choice for AI Workloads

Generative AI thrives on rich, diverse, and unstructured data. Embeddings, contextual metadata, structured references all contribute to improving AI systems. You can perform the following actions with MongoDB:

• Store vector embeddings, metadata, and source content in a single document.

• Combine structured and vector data without jumping between systems.

• Query vector and non-vector fields together for contextual, accurate results.

Example 1: Car Manual Embeddings

When using a retrieval-augmented generation (RAG) approach, the quality of the chunking and embeddings directly impacts the quality of the AI's responses. Poorly segmented content or missing context can lead to vague or inaccurate answers. Technical manuals often contain dense text, diagrams, and domain-specific terminology, making it challenging to retrieve the right information.

This solution represents each chunk of the manual as a document. The document includes not just the text and its vector embedding, but also metadata such as content type (e.g. safety and diagnostics), page numbers, chunk length, and links to related chunks. This additional context helps the system understand how pieces of information relate to one another, which is especially important in highly technical or interdependent topics.

MongoDB's flexible document model makes it straightforward to capture this complexity. As the manual evolves or as new needs emerge, you can incrementally add fields or adjust structure without requiring a full schema migration. This enables more precise retrieval and more helpful AI responses.

The following example document represents a manual chunk:

{
  "_id": {
    "$oid": "67cc4b09c128338a8133b59a"
  },
  "text": "Oil Pressure Warning Lamp. If it illuminates when the engine is running this indicates a malfunction. Stop your vehicle as soon as it is safe to do so and switch the engine off. Check the engine oil level. If the oil level is sufficient, this indicates a system malfunction.",
  "page_numbers": [
    23
  ],
  "content_type": [
    "safety",
    "diagnostic"
  ],
  "metadata": {
    "page_count": 1,
    "chunk_length": 1045
  },
  "id": "chunk_0053",
  "prev_chunk_id": "chunk_0052",
  "next_chunk_id": "chunk_0054",
  "related_chunks": [
    {
      "id": "chunk_0048",
      "content_type": [
        "safety"
      ],
      "relation_type": "same_context"
    },
    {
      "id": "chunk_0049",
      "content_type": [
        "safety"
      ],
      "relation_type": "same_context"
    },
    ...
  ],
  "embedding": [
    -0.002636542310938239,
    -0.005587903782725334,
    ...
  ],
  "embedding_timestamp": "2025-03-08T13:50:00.887107"
}

Example 2: Vehicle Signal Data

For vehicle signals, this solution models data by using the COVESA Vehicle Signal Specification (VSS). VSS provides a standardized, hierarchical structure to describe real-time signals like speed, acceleration, or diagnostic trouble codes (DTCs). It's an open, extensible format that enables easier collaboration, system integration, and data reuse across vehicle platforms.

Because MongoDB's document model natively handles nested structures, representing the VSS hierarchy is straightforward. Signals can be grouped logically, just like they appear in the VSS model, which aligns with the tree-based structure of the spec.

Figure 3. The VSS data model is a hierarchical tree structure built with modules that can be flexibly combined. Source: https://covesa.global/vehicle-signal-specification/

This structure accelerates development and ensures that AI tools and workflows have consistent access to clean, structured, and meaningful data.

The following document is an example representation of a vehicle signal that conforms to VSS.

{
  "_id": {
    "$oid": "67e58d5f672b23090e57d478"
  },
  "VehicleIdentification": {
    "VIN": "1HGCM82633A004352"
  },
  "Speed": 0,
  "TraveledDistance": 0,
  "CurrentLocation": {
    "Timestamp": "2020-01-01T00:00:00Z",
    "Latitude": 0,
    "Longitude": 0,
    "Altitude": 0
  },
  "Acceleration": {
    "Lateral": 0,
    "Longitudinal": 0,
    "Vertical": 0
  },
  "Diagnostics": {
    "DTCCount": 0,
    "DTCList": []
  }
}

MongoDB's document model doesn't just store your data. It mirrors the complexity of the real world, making it easier to build smarter systems that respond in real time, adapt to user needs, and grow with your platform. Whether you're storing vehicle diagnostics or vector-encoded manuals, MongoDB gives you the tools to build intelligent experiences faster.