Monitor data pipelines

SnapLogic AutoSync users can view summary information about their own data pipelines. Org admins can view information about all data pipelines in the Org.

Monitor provides infrastructure metrics and execution details for both AutoSync data pipelines and pipelines created in the IIP. Pipelines with warnings can complete successfully.

This page describes how Org admins can find information about AutoSync data pipelines. Refer to the Monitor documentation for information on:

  • Analyzing the execution of pipelines created in the IIP.
  • Viewing metrics for the runtime system.

View data pipeline summaries in the dashboard

In addition to searching for a specific data pipeline, you can filter the AutoSync dashboard to display cards for:

  • Data pipelines you created.
  • Data pipelines you marked as favorites.
  • All data pipelines.

As shown in the following example, a data pipeline card includes:

  • A toggle to enable or disable synchronization.
  • The next scheduled run.
  • The last run and elapsed time.
  • A toolbar with controls to mark as a favorite, edit, and options to run now or delete.
  • Icons that identify the source, destination, and purpose of the pipeline.
  • The creation date.

Data pipeline card example

To see more details, click the card. A panel opens that includes additional information:

  • The full description.
  • The number of tables loaded.
  • The number of records newly uploaded, updated, or failed.

Example data pipeline details panel

View data pipeline executions in Monitor

AutoSync data pipelines are created for you by the system. You cannot edit them. However, viewing information about execution can be useful to you and might be helpful when contacting Support.

Tip: When you drill down into details about data pipeline execution, you might see warnings and errors that recommend some action. Because AutoSync composes the data pipelines and tasks that run them, you will not be able to perform the recommended actions on pipelines, tasks, or the underlying Snaps. For example, since there is no way to set primary key chunking for AutoSync, you could ignore the following warning:
Warning

To view details about data pipeline execution in Monitor, select Execution overview from the Monitor navigation pane. AutoSync data pipelines look similar to the following:

Note that:
  • An AutoSync data pipeline is composed of a parent and several child pipelines.
  • The number of documents processed for the parent data pipeline can be 0 or 1, it does not refer to the number of records loaded.
  • The number of documents processed for child pipelines is cumulative, not per object or per table.

Click a pipeline to view the details panel:


Data pipeline details in Monitor

View loading summary

The first time AutoSync runs a data pipeline, it loads all records from the specified objects or tables. On synchronization, it updates any changed records and adds new ones. You can see the records processed organized by the number of new, updated, or failed. Refer to How AutoSync loads data for more information.

To view the loading summary in the AutoSync dashboard, click the data pipeline card to open the details panel:


Example data pipeline details panel