Integrate Atlas Application Services Logs Into Datadog on AWS
BS
AR
Babu Srinivasan, Abilash Ravikumar2 min read • Published Nov 02, 2022 • Updated Sep 09, 2024
Rate this tutorial
Datadog is a well-known monitoring and security platform for cloud applications. Datadog’s software-as-a-service (SaaS) platform integrates and automates infrastructure monitoring, application performance monitoring, and log management to provide unified, real-time observability of a customer’s entire technology stack.
MongoDB Atlas on Amazon Web Services (AWS) already supports easy integration with Datadog for alerts and events right within the Atlas UI (select the three vertical dots → Integration → Datadog). With the Log Forwarding feature, it's now possible to send Atlas Application Services logs to Datadog. This blog outlines the configuration steps necessary as well as strategies for customizing the view to suit the need.
Atlas Application Services (formerly MongoDB Realm) is a set of enhanced services that complement the Atlas database to simplify the development of backend applications. App Services-based apps can react to changes in your MongoDB Atlas data, connect that data to other systems, and scale to meet demand without the need to manage the associated server infrastructure.
App Services provides user authentication and management, schema validation and data access rules, event-driven serverless functions, secure client-side queries with HTTPS Endpoints, and best of all, synchronization of data across devices with the Realm Mobile SDK.
With App Services and Datadog, you can simplify the end-to-end development and monitoring of your application. Atlas App Services specifically enables the forwarding of logs to Datadog via a serverless function that can also give more fine-grained control over how these logs appear in Datadog, via customizing the associated tags.
We assume that you already have an Atlas account. If not, you can sign up for a free account on MongoDB or the AWS Marketplace. Once you have an Atlas account, if you haven't had a chance to try App Services with Atlas, you can follow one of our tutorials to get a running application working quickly.
To initiate custom log forwarding, follow the instructions for App Services to configure log forwarding. Specifically, choose the “To Function” option:
Within Atlas App Services, we can create a custom function that provides the mapping and ingesting logs into Datadog. Please note the intake endpoint URL from Datadog first, which is documented by Datadog.
Here’s a sample function that provides that basic capability:
1 exports = async function(logs) { 2 // `logs` is an array of 1-100 log objects 3 // Use an API or library to send the logs to another service. 4 await context.http.post({ 5 url: "https://http-intake.logs.datadoghq.com/api/v2/logs", 6 headers: { 7 "DD-API-KEY": ["XXXXXX"], 8 "DD-APPLICATION-KEY": ["XXXXX"], 9 "Content-Type": ["application/json"] 10 }, 11 body: logs.map(x => {return { 12 "ddsource": "mongodb.atlas.app.services", 13 "ddtags": "env:test,user:igor", 14 "hostname": "RealmApp04", 15 "service": "MyRealmService04", 16 "message" : JSON.stringify(x) 17 }}), 18 encodeBodyAsJSON: true 19 }); 20 }
One of the capabilities of the snippet above is that it allows you to modify the function to supply your Datadog API and application keys. This provides the capability to customize the experience and provide the appropriate context for better observability. You can change ddtags, the hostname, and service parameters to reflect your organization, team, environment, or application structure. These parameters will appear as facets helping with filtering the logs.
Note: Datadog supports log ingestion pipelines that allow it to better parse logs. In order for the MongoDB log pipeline to work, your ddsource must be set to
mongodb.atlas.app.services.
Once log forwarding is configured, your Atlas App Services logs will appear in the Datadog Logs module.
You can click on an individual log entry to see the detailed view:
In this blog, we showed how to configure log forwarding for Atlas App Services logs. If you would like to try configuring log forwarding yourself, sign up for a 14-day free trial of Datadog if you don’t already have an account. To try Atlas App Services in AWS Marketplace, sign up for a free account.
Rate this tutorial
Related
Tutorial
Building Generative AI Applications Using MongoDB: Harnessing the Power of Atlas Vector Search and Open Source Models
Sep 18, 2024 | 10 min read
Article
Capturing and Storing Real-World Optics With MongoDB Atlas, OpenAI GPT-4o, and PyMongo
Sep 04, 2024 | 7 min read