Alerts tab

The Alerts tab in the Notification Center displays the alerts that Snaplex nodes generated during the specified time range. From this tab, you can view alerts and download alert logs and Snaplex logs to troubleshoot issues. You can address issues with self-managed Snaplexes (Groundplexes). SnapLogic manages issues with Cloudplexes.

Alerts have two states, open and closed . The system closes some alerts when the cause of the alert resolves. You can manually close alerts from the three dot menu.

Snaplex nodes raise alerts for various conditions, including:
  • Problems during initialization.
  • Congestion, when pipelines queue because the node is busy with other requests.
  • Node crashes, which the next node to start on the same path detects, so the alert timestamp reflects when detection happened rather than the moment the node actually went down.
  • Rejected connections.
  • A node interrupts a pipeline that's using too much memory.
  • A task duration notification is set and any task exceeds the specified execution duration.
  • The daily API usage exceeds 75% of the limit.
  • The concurrent API usage exceeds 75% of the limit.
Tip: Create a Snaplex notification describes how to configure email and slack notifications for node resource utilization. Some of those configurable notifications don't display on the alerts tab.

Snaplex alerts include a state, event name, description, and date as shown in the image below:


Alert history page in Monitor

Use table controls to:

  • Select the date range.
  • Download alerts log: Download formatted alert data in CSV file format suitable for viewing and reporting.
  • Download raw data: Download raw data with unformatted API fields in CSV file format for custom processing or integration with external systems.
  • Refresh table data.
  • Configure the table, by clicking . You can set row spacing to expanded or compact, hide or show columns, and filter the State column.
  • To download Snaplex logs, click the link in node crash alerts.
  • For pipeline interrupter alerts, click the link to open the pipeline in Designer.
  • To filter alerts for a specific Snaplex, click the Snaplex name in a node alert.

Snaplex congestion alerts

A Snaplex raises an alert in the following scenarios:

  • One or more Pipelines are in the Queued state on a Snaplex for more than:
    • 75% of their respective Time To Live (TTL), or
    • the maximum time that SnapLogic attempts to execute a Pipeline.

    At most, a Snaplex raises one alert regardless of the number of Pipelines over the threshold. The default Pipeline TTL is 5 minutes and the default threshold is 75%.

Important: Always investigate the cause of Snaplex congestion alerts. Frequent alerts indicate that a Snaplex needs more capacity or the pipelines running on it need a redesign.
Note: Open Alerts:

A scheduled job runs every 5 minutes to monitor the queued Pipelines. If it detects any Pipeline above 75% of the TTL threshold at that instant, it generates an Open Alert. However, if an Open Alert already exists for that Snaplex, a new Alert is not generated. The previous Open Alert remains in the same state.

After generating an Open Alert, the scheduled job repeats, monitoring the Pipelines queued for more than 75% of their respective TTLs. If there are no Pipelines above the threshold for that Snaplex, the scheduled job resolves the Alert, and its state changes to Closed.

Snaplex node threshold watermarks

You can adjust the behavior of Snaplex node alerts with threshold watermarks. In some scenarios, something other than a pipeline raises a CPU, memory, or disk alert. For example, files accumulating in a /tmp directory mounted on the Snaplex node can raise a disk alert. To address this case, an Environment admin can put the Snaplex in maintenance mode, and clear the space in the directory.

Threshold watermarks enhance the reporting accuracy of Snaplex node alerts and can reduce the frequency of alerts. With watermarks enabled, a memory or CPU alert closes when usage falls below the low watermark, which is 40% of the threshold. For example, an alert set to trigger at 80% memory usage closes when memory falls below 0.4 * 80, or 32%. A disk alert closes when free space rises above the high watermark, which is 104% of the threshold.

Watermarks aren't set by default. You can ask your CSM to configure the following threshold watermarks by enabling the feature flag for the environment and setting the watermarks:

  • com.snaplogic.cc.service.UsageMonitorServiceImpl.LOW_WATERMARK = 0.4
  • com.snaplogic.cc.service.UsageMonitorServiceImpl.HIGH_WATERMARK = 1.04