Snaplex state transitions

Snaplex nodes have different states that affect pipeline executions. A number of these are user-initiated and some are system-initiated transitional states based on user actions. You can view these states in the Monitor app in the System Overview table view.

  • Default state
  • Cooldown transition
  • Max restart wait time
  • Maintenance mode

Default state

All Snaplex nodes can accept new pipeline execution requests load balanced automatically across the nodes.

How Cooldown Works

A Snaplex node enters a cooldown state as part of a state transition, which can be triggered by a version update, a configuration change, or a system restart. This process is designed to allow a node to shut down gracefully without interrupting active tasks.

When a Snaplex requires a state transition, one or more nodes will enter the cooldown state. Here’s what happens during that time:

  • New pipeline requests are paused. The node will not accept any new requests to execute a pipeline from the system's queue.
  • Current pipelines finish their work. The node waits for any pipelines that are already running to complete.

The cooldown state ensures that work is not lost or abruptly terminated during routine updates and maintenance. While in this state, the node may still accept specific requests sent directly from a client, but it will not pull new pipelines from the general execution queue.

How Max restart wait time works

In the Node properties tab in the Snaplex Settings dialog, the Max Restart Wait Time setting determines how long a Snaplex node will wait for active pipelines to finish during a restart or update. This waiting period is known as the cooldown state. The default wait time is 15 minutes. If pipelines are still running when this time limit is reached, they will be automatically stopped so the node can complete its update.

When a Snaplex node needs to restart (for example, during a version upgrade), it enters a cooldown state. The node stops accepting new pipeline requests and waits for a set amount of time for running pipelines to finish.

Consider the following exceptions:

  • Long-Running Pipelines: You can increase the value beyond the default value if you regularly run pipelines that take longer than 15 minutes to complete.
  • Polling and Ultra Pipelines: You do not need to increase the wait time for these pipelines. They are automatically signaled to stop polling for new data. This allows these pipelines to shut down cleanly without waiting for the timeout.

Best practices

  • Keep the timeout under one hour: We strongly recommend keeping the Max Restart Wait Time below 60 minutes. Longer timeouts can significantly delay configuration changes and version updates. During mandatory upgrades, the SnapLogic control plane only waits one hour for nodes to update before it stops sending them new pipeline requests.
  • Avoid very low values: If you set the timeout to a very low value (such as, 2 minutes), pipelines that need more time will be terminated before they can finish.
  • Using the "Forever" Setting: You can set the wait time to Forever, which prevents pipelines from ever being terminated automatically during a restart. However, this means a node will never restart on its own if a pipeline is running. You will be responsible for manually stopping pipelines to allow the update to proceed.
  • Load balancer requirement: If you use alternate URLs for Triggered Tasks or Ultra Pipelines, you need to configure a load balancer to monitor the health of your Snaplex nodes. Without a properly configured load balancer, tasks will fail because they will be sent to a node that is shutting down and not accepting new work.

Snaplex maintenance mode

When you place a Snaplex node in maintenance mode, the node enters the Cooldown transition and waits for running pipelines to complete. Each node begins failing the health check. After all running pipelines finish executing, the node enters maintenance mode, where pipeline execution requests are not accepted. The Max Restart Wait Time setting value is not related to maintenance mode.

During an upgrade, the behavior of the maintenance mode depends on the Snaplex type:

Note:
  • We do not recommend manually placing the nodes in maintenance mode during a Snaplex version update or configuration change.
  • For Ultra low-latency pipelines, make sure that the FeedMaster does not receive client requests while it is in maintenance mode. If you put a FeedMaster node in maintenance mode, you need to configure your load balancer to stop client requests from directing to the FeedMaster when the health check to the node is failing.

Snaplex version change

When you change a Snaplex version, each node goes into the Cooldown transition one at a time. After the node enters the Cooldown transition and running pipelines finish executing, the JCC service on the node terminates and restarts with the new version of the Snaplex binaries. Since the Snaplex version change happens in a rolling manner without downtime required (as long as the Snaplex has more than one node), your intervention is not required during the rolling restart — all node transitions are completed automatically.

Cloudplex: If a node is in maintenance mode and the Snaplex version is updated, the Cloudplex exits maintenance mode after the version update.

Groundplex: If a node is in maintenance mode and the Snaplex version is updated, the Groundplex remains in maintenance mode after the update.

Snaplex State Transitions Reference

State/Action Purpose Triggered by
Default state For pipeline executions N/A
Version change For changing the Snaplex JCC service to a different version.
  • User initiated via Admin Manager
  • Rolling upgrade via the Auto upgrade Snaplexes switch in Admin Manager or quarterly mandatory automatic upgrade.
Configuration change For changing Snaplex default node properties.
  • User initiated via Admin Manager
Node Shutdown For powering down the node to make changes to the host machine.

CLI command or script depending on the host OS/environment type. Refer to the Groundplex article for your host.

Node Restart For bringing the node online after a shutdown.

User initiated in Monitor > Infrastructure > System Overview page via the Restart node button.

Alternatively, you can use the CLI. The command depends on the host OS type. Refer to the Groundplex article for you host.

Maintenance mode For taking the node offline, usually for configuration changes.

User initiated in Monitor > Infrastructure > System Overview page via the Maintenance mode button.

Cooldown Transitional state between default and a restart, version update, configuration change, maintenance mode. System initiated