Forward Logs to a Service
On this page
Overview
You can configure a log forwarder to automatically store your application's server-side logs in a MongoDB collection or send them to an external service. Atlas App Services can forward logs individually as they're created or batch logs together to reduce overhead.
A log forwarder consists of the following components:
An action that controls how and where App Services forwards logs.
A filter that controls which logs App Services forwards.
A policy that controls whether App Services batches logs or forwards them individually.
Why Should I Configure Log Forwarding?
Consider setting up a log forwarder if you need to do any of the following:
Store logs for longer than App Services's retention period of 10 days.
Integrate logs into an external logging service
Access logs in Atlas Search, Online Archive, and Charts
How is Log Forwarding Billed?
Each log forward action invocation (on either an individual log or a batch) is billed as one App Services request.
Set Up a Log Forwarder
1. Create a Log Forwarder
To create a new log forwarder, navigate to the Logs page and select the Forwarding tab. Then, click the Create a Log Forwarder button.
On the next screen, specify a unique name for the log forwarder.
To create a new log forwarder, add a new configuration file to the
log_forwarders directory of your app. The file name should
match the value in the configuration's name field.
{ "name": "<name>" }
2. Choose Which Logs to Forward
App Services can forward all of your app's logs or send only a subset to the target collection or service. You control this subset for each log forwarder by defining filters for the log type (e.g. functions, sync, etc.) and status (i.e. success or error) that invoke the forwarder's action.
Choose one or more types of log to forward in the Log Type dropdown:
Choose one or more statuses to forward in the Log Status dropdown:
Specify one or more types and one or more statuses for the forwarder to match and forward:
{ "name": "<name>", "log_types": [ "<type>", ... ], "log_statuses": [ "<status>", ... ] }
App Services supports forwarding the following log types:
authendpointfunctiongraphqlpushschemaservicesynctriggertrigger_error_handler
App Services supports forwarding the following log statuses:
errorsuccess
Important
App Services only forwards a given log if both its type and status are specified in the filter.
For example, consider a forwarder that filters for sync logs with
an error status.
The filter would forward the following log:
{ "type": "sync", "status": "error", ... }
The filter would not forward the following logs:
{ "type": "sync", "status": "success", ... } { "type": "schema", "status": "error", ... }
3. Configure Log Batching
App Services can combine multiple logs in to a single batched request to reduce overhead. The way that App Services groups logs into batches is controlled by a batching policy.
App Services supports the following batching policies:
No Batching: App Services forwards logs individually as their corresponding requests occur.
Batching: The forwarder groups documents into a batch as they happen. Each batch may include up to 100 log entries. When a batch is full, App Services forwards the entire batch in a single request. App Services forwards logs at least once a minute regardless of the number of logs in the current batch.
To configure batching, select either the No batch or Batching policy.
To configure batching, specify the policy type, either single
or batch, in the policy field:
{ "name": "<name>", "log_types": [ "<type>", ... ], "log_statuses": [ "<status>", ... ], "policy": { "type": "<single|batch>" } }
5. Define an Action
A log forwarder can automatically store logs in a linked MongoDB collection or call a custom function that sends the logs to an external service.
Store Logs in a MongoDB Collection
To store logs in a collection, select the To Collection action and enter the names of the linked cluster, database, and collection that should hold the forwarded logs.
To store logs in a collection, specify an action of type
collection that includes the names of the linked cluster,
database, and collection that should hold the forwarded logs.
{ "name": "<name>", "log_types": [ "<type>", ... ], "log_statuses": [ "<status>", ... ], "policy": { "type": "<single|batch>" }, "action": { "type": "collection", "data_source": "<data source name>", "database": "<database name>", "collection": "<collection name>" } }
Forward Logs with a Custom Function
To forward logs to an external service, write a function that accepts an array of log objects and calls the service through an API, SDK, or library.
Note
Depending on your batching policy and log frequency, App Services may call a log forwarding function with an array of up to 100 log objects.
The function should have the same signature as the following example:
exports = async function(logs) { // `logs` is an array of 1-100 log objects // Use an API or library to send the logs to another service. await context.http.post({ url: "https://api.example.com/logs", body: logs, encodeBodyAsJSON: true }); }
Once you've written the log forwarding function, you can assign it to a log forwarder by name.
To assign a function to a log forwarder, select the To Function action and then select the function from the dropdown input.
To assign a function to a log forwarder, specify an action of
type function that includes the name of the log forwarding
function.
{ "name": "<name>", "log_types": [ "<type>", ... ], "log_statuses": [ "<status>", ... ], "policy": { "type": "<single|batch>" }, "action": { "type": "function", "name": "<function name>" } }
6. Save and Deploy your Changes
Once you've configured the log forwarder, click Save. If you have deployment drafts enabled, make sure to deploy your changes.
Once you've configured the log forwarder, save the configuration file and then push your updated app configuration:
appservices push
Restart a Suspended Log Forwarder
A log forwarder may suspend in response to an event that prevents it from continuing, such as a network disruption or a change to the underlying cluster that stores the logs. Once suspended, a forwarder cannot be invoked and does not forward any logs.
You can restart a suspended log forwarder from the Logs > Forwarding screen of the App Services UI.
Note
If a log forwarder is suspended, App Services sends the project owner an email alerting them of the issue.