Pipeline lifecycle metrics reference

SnapLogic emits pipeline lifecycle metrics (plexnode.pipeline.*) through OpenTelemetry so you can monitor pipeline execution state, phase durations, document throughput, and concurrent load in your third-party observability tool.

Pipeline lifecycle metrics give you granular, near real-time visibility into the pipeline execution lifecycle on each Snaplex node: execution starts and stops, prepare and execute phase durations, document throughput, error counts, and the number of currently active pipelines. The metrics are emitted through Third-party Observability and appear in your configured monitoring backend (for example, Datadog, New Relic, or a Prometheus/Grafana stack), where you can filter and aggregate them by tag.

Use these metrics to troubleshoot pipeline failures, optimize performance, plan Snaplex capacity, and verify SLA compliance. For a catalog of the metrics organized by category, see Pipeline metric catalog. For worked monitoring scenarios and sample queries, see Pipeline metrics use cases.

What these metrics capture

Pipeline lifecycle metrics capture the execution lifecycle after a pipeline has successfully started. They do not capture failures that occur before a pipeline begins execution. Use task-level metrics — the plexnode.state.tasks.loading.* and plexnode.state.task.scheduled.* families — to monitor the pre-start phase. See the task loading metrics in the Snaplex metrics reference.

Task preparation (pre-start phase) Pipeline execution (post-start phase)
Not captured by plexnode.pipeline.*. Use the task-level metrics (plexnode.state.tasks.loading.* and plexnode.state.task.scheduled.*) instead:
  • Misfire (scheduled too late)
  • Resource unavailable
  • Pipeline not found
  • Permission denied
  • Missing accounts or dependencies
  • Validation or configuration errors
  • One-at-a-time conflict
 Captured by plexnode.pipeline.*:
  • Execution errors
  • Pipeline timeout
  • Snap failures
  • Data processing errors
  • External dependency failures
  • Document throughput issues
Tip: For complete task monitoring, supplement pipeline metrics with task-level metrics (plexnode.state.task.scheduled.*) to capture preparation failures, misfires, and scheduling drift.

Relationship to existing pipeline metrics

Pipeline lifecycle metrics do not replace the existing pipeline metrics (for example, pipeline.slots.leased). Both sets coexist, each providing a different level of detail: the existing metrics continue to be emitted and serve their current purpose, while the lifecycle metrics add granular, tag-based dimensions for deeper analysis. Your existing dashboards and alerts continue to work without change; adopt the lifecycle metrics where you need finer-grained analysis, or combine both in a single dashboard.

Prerequisites

  • Your organization (Org) must have Third-party Observability enabled.
  • An OpenTelemetry (OTel) collector must be configured for each Snaplex node and routed to your monitoring backend.
  • The LIFECYCLE feature flag must be enabled. This flag is disabled by default (see Feature flags).

Metrics

SnapLogic emits the following pipeline lifecycle metrics. Metric names use the plexnode.pipeline.* prefix.

Metric Type Description
plexnode.pipeline.execution.start.total Counter Number of pipeline executions started.
plexnode.pipeline.execution.stop.total Counter Number of pipeline executions stopped. Combine with the outcome tag to compute success and failure rates.
plexnode.pipeline.prepareDuration.* Summary Duration of the pipeline prepare phase, reported as avg, p95, p99, and max.
plexnode.pipeline.executeDuration.* Summary Duration of the pipeline execute phase, reported as avg, p95, p99, and max.
plexnode.pipeline.errors.total Counter Number of pipeline execution errors. Slice by the error tag to identify the most common failure types.
plexnode.pipeline.documents.processed.oneMinRate Gauge Document processing rate over a one-minute window. Slice by the direction tag to separate input, output, and error flow.
plexnode.pipeline.active.value Gauge Number of pipelines currently active. Use to monitor concurrent load per Snaplex.

Tags

Each metric is tagged so you can filter and group results. The following tags are available.

Tag Description Values
path Pipeline path. For example, /snaplogic/projects/shared/child
invoker_type How this pipeline was started. scheduled, trigger, ultra, nested, error, manual
root_invoker_type How the top-level workflow was started. scheduled, trigger, ultra, manual
root_invoker_path Path of the top-level task or trigger. For example, /snaplogic/projects/shared/parent-one-per-doc
snaplex Snaplex where the pipeline executes. For example, cloud
outcome Final execution result. completed, failed, stopped
error Error description. For example, No script to execute, unknown
direction Document flow direction. in, out, error, feed

Dimensional model

You can slice pipeline metrics by three primary dimensions:

Dimension Tags Primary use
Infrastructure snaplex Where is it running? Capacity planning and node health.
Business context root_invoker_type, root_invoker_path Who or what triggered the workflow? Task monitoring and SLA tracking.
Execution path, invoker_type What is executing? Pipeline optimization and nested-call analysis.
Tip: root_invoker_path is typically the most valuable dimension for business-process monitoring because it represents the entire workflow context, regardless of nesting depth. Every pipeline in a nested chain — the parent and all of its nested children — shares the same root_invoker_path.

Feature flags

Two feature flags control pipeline metric emission:

Flag Default Description
LIFECYCLE Disabled Emits the core pipeline lifecycle metrics described on this page.
DETAILED Disabled Adds high-cardinality tags (such as ruuid) for deeper per-execution analysis.
CAUTION: The DETAILED flag adds high-cardinality tags that can significantly increase storage and query load in your monitoring backend. Evaluate the backend storage impact before enabling DETAILED on high-volume Snaplexes.

Correlating metrics with traces

When your monitoring backend supports Prometheus Exemplars, latency and error metrics carry a trace_id exemplar that links a data point to the corresponding OpenTelemetry trace span. This lets you jump from an anomalous metric — such as a latency spike or an error — directly to the distributed trace for that execution, including across nested pipelines and external service calls. See OpenTelemetry for more information about traces and exemplars.

Limitations

  • Pipeline metrics capture only the post-start execution phase. Pre-start failures are reported by task-level metrics (plexnode.state.tasks.loading.* and plexnode.state.task.scheduled.*; see the task loading metrics in the Snaplex metrics reference). See What these metrics capture.
  • Nested pipeline executions (invoker_type="nested") are excluded from the default metric views to avoid inflating execution counts. You can still filter for nested executions explicitly when analyzing a workflow chain.