Load balancers and Groundplexes

Use case for a load balancer

A load balancer facilitates the efficient distribution of network or application traffic between client devices and backend servers. In the SnapLogic environment, a load balancer accepts incoming requests to the Snaplex from client applications.

Note: Load balancers are the responsibility of the customer's IT or system administration team.

This purpose differs from that of an HTTP proxy, which might be required for outbound requests from the Snaplex to the control plane or other endpoints. (Typically, the HTTP proxy is required when Groundplex nodes are on client servers with a restricted network configuration.)

Load balancers should also not be confused with SnapLogic FeedMaster nodes, which might reside in a Snaplex, and serve as the message queue broker for Ultra Tasks.

Use cases for a load balancer

You should provision a load balancer for a Snaplex when external client API calls are sent directly to the Snaplex nodes. In a SnapLogic-managed Snaplex (Cloudplex), a load-balancer might be configured already when the Snaplex is deployed. We recommend a load balancer in the following use cases:

  • Snaplex-triggered Pipeline executions—Since the control plane triggering mechanism imposes additional Org-level API limits, we recommend using the Snaplex triggering mechanism for high-volume API usage.
  • REST requests to Ultra Tasks—For direct API calls to the Snaplex, the requests must pass through a load balancer. Without a load balancer, request failures occur, and the Snaplex eventually goes offline during Snaplex maintenance or upgrades.
  • API Policies—We recommend that you set up a load balancer for API policies to APIs in APIM or Triggered and Ultra Tasks in Manager for your Self-managed Snaplex. With a SnapLogic-managed Snaplex, we provision the load balancer.

Using a load balancer with Snaplex ensures high availability and minimizes downtime. You can configure the load balancer to run health checks on the node and manage rolling upgrades or restarts seamlessly, ensuring requests are always directed to active nodes. For more information, refer to Snaplex transitions.

After configuring the load balancer, you must add the load balancer URL to the Snaplex properties.

  • For Triggered Tasks, you only need to configure the Load balancer field.
  • If a load balancer points to a FeedMaster node, then you only need to configure the Ultra load balancer field.
  • If the load balancer points to JCC nodes, then you should only configure the Load balancer field in the Snaplex Settings tab.
  • The full URL of the load balancer needs to be configured in the Snaplex properties.

    For example, https://mylb.mydomain.com:443

Use cases where a load balancer is not required

Load balancers are not required for the following types of activities:

  • Pipeline executions triggered through the control plane.
  • Scheduled pipelines, pipeline and account validation, and pipeline development.
  • Headless Ultra (since the Ultra Task processing is not driven by REST API calls).
  • Child Pipeline runs triggered through the Pipeline Execute Snap.

Groundplex load balancer best practices

  • For a Snaplex with FeedMaster nodes, the load balancer should point to these FeedMaster nodes:

    • https://fm-node1.example.com:8084
    • https://fm-node2.example.com:8084

A FeedMaster node can process Triggered and Ultra Tasks, but a JCC node can only process Triggered Tasks. You can use the FeedMaster node as the load balancer endpoint for a Triggered Task. For an Ultra Task, you need to update the load balancer field value in the Snaplex Settings dialog.

  • If your Snaplex does not contain FeedMaster nodes, the load balancer can point to these JCC nodes:

    • https://jcc-node1.example.com:8081
    • https://jcc-node2.example.com:8081

For Triggered Tasks and Low latency Ultra Tasks, you need to update the Load balancer field value in the Snaplex settings.

  • You should configure the load balancer to run health checks on the Snaplex node on the /healthzURL. Any HTTP response code besides 200 indicates a health check failure.
  • The load balancer should perform SSL offloading/termination so that the certificate and cipher management can be done on the load balancer without updating the Snaplex nodes. The connection between the client and the load balancer is over HTTPS with your signed certificate. The connection between the load balancer and the Snaplex nodes is also over HTTPS with the default SnapLogic-generated certificate.
  • You must set the HTTP request timeout to a value of 900 or higher to allow for long-running requests. This timeout setting is different from the keep-alive timeouts that are used for connection management, like the following examples:
    • The proxy_read_timeout for Nginx.
    • The ProxyTimeout for Apache.
    • The idle timeout for AWS ELB.The following image from the AWS UI shows a sample health check configuration for the AWS ELB.
Warning: If you change the default ports of the JCC and FeedMaster nodes in your Groundplex, then you should reconfigure your load balancer to use the new port assignments, which are 8081 for JCC and 8084 for FeedMaster.