Groundplex requirements: Load balancer

A load balancer efficiently distributes network and application traffic between client devices and backend servers. In the SnapLogic environment, a load balancer is used for incoming requests to the Snaplex from client applications. This purpose differs from that of an HTTP proxy, which might be required for outbound requests from the Snaplex to the control plane or other endpoints. Typically, the HTTP proxy is required when Groundplex nodes are on client servers with a restricted network configuration.

When load balancers are required

SnapLogic requires a load balancer for a Snaplex when external client API calls are sent directly to the Snaplex nodes for the following reasons:

  • Snaplex-triggered pipeline executions: Because the control plane triggering mechanism imposes additional Org level API limits, use the Snaplex triggering mechanism for high-volume API usage.
  • REST requests to Ultra Task: For direct API calls to the Snaplex, the requests must pass through a load balancer. Without a load balancer, request failures occur, and the Snaplex eventually goes offline to recover by performing Snaplex maintenance or upgrades.
  • API Policies: A load balancer is required to apply API policies to APIs or to Triggered and Ultra Tasks on a Cloudplex.
    Important: On a Cloudplex, a load balancer is provisioned by SnapLogic, typically only when Ultra Task subscription feature is enabled. The Cloudplex load balancer has a snaplogic.io domain endpoint that points to the FeedMaster nodes. You can provision a load balancer for both Ultra Task-triggered and Snaplex-triggered executions.

When load balancers are not required

Load balancers are not required for the following types of activities:

  • Pipeline executions triggered through the control plane.
  • Scheduled pipelines, Pipeline/account validation, and pipeline development.
  • Headless Ultra (since the Ultra Task processing is not driven by REST API calls).
  • Child pipeline executions triggered through the Pipeline Execute Snap.

Adding the load balancer

After the load balancer is configured in a Groundplex (or provisioned in a Cloudplex), you must add the load balancer URL to the Ultra Task properties or the Snaplex properties.

  • If a load balancer points to a FeedMaster node, then you need to configure the Load balancer field in the Ultra Task properties only.
    Tip: Configuring the Load balancer field for Ultra Tasks also enables load balancing for Triggered Tasks.
  • If the load balancer points to worker nodes, then you need to configure the Load balancer field in the Snaplex properties only.

Best practices for the Groundplex load balancer

  • For Snaplexes with FeedMaster nodes, the load balancer must point to the FeedMaster nodes. Examples: https://fm-node1.example.com:8084 and https://fm-node2.example.com:8084. A FeedMaster node can process Triggered and Utra Task requests, a JCC can process only Triggered Task requests. In the latter case, it is easier to use the FeedMaster node as the load balancer endpoint. The Ultra Task load balancer field value needs to be updated in the Snaplex settings with the load balancer URL.
  • If there are no FeedMaster nodes, the load balancer can point to the JCC nodes. Examples: https://jcc-node1.example.com:8081 and https://jcc-node2.example.com:8081. A JCC node can process only Triggered Task requests. Make sure you update the load balancer field value in the Snaplex settings with the correct load balancer URL.
  • Configure the load balancer to run health checks on the JCC node using the /healthz URL. This ensures that the node going offline for maintenance does not receive any new requests. Any other response code besides 200 indicates a health check failure.
    Example: Configure Health Checks in the AWS UI:
    AWS - Configure Health Check

  • The load balancer must perform SSL offloading/termination so that the certificate and cipher management can be done on the load balancer without updating the Snaplex nodes. The connection between the client and the load balancer is over HTTPS with your signed certificate. The connection between the load balancer and the Snaplex nodes are also over HTTPS with the default SnapLogic generated certificate.
  • Set the HTTP request timeout to a value of 900 or higher to allow for long-running requests. This timeout setting is different from the keep-alive timeouts that are used for connection management, such as the following:
    • The proxy_read_timeout for Nginx.
    • The ProxyTimeout for Apache.
    • The idle timeout for AWS ELB.