Load balancers and Snaplexes

Use case for a load balancer

A load balancer facilitates the efficient distribution of network or application traffic between client devices and backend servers. In the SnapLogic environment, a load balancer accepts incoming requests to the Snaplex from client applications.

This purpose differs from that of an HTTP proxy, which might be required for outbound requests from the Snaplex to the control plane or other endpoints. (Typically, the HTTP proxy is required when Groundplex nodes are on client servers with a restricted network configuration.)

Load balancers should also not be confused with SnapLogic FeedMaster nodes, which might reside in a Snaplex, and serve as the message queue broker for Ultra Tasks.

Use cases for a load balancer

You should provision a load balancer for a Snaplex when external client API calls are sent directly to the Snaplex nodes. In a SnapLogic-managed Snaplex (Cloudplex), a load-balancer might be configured already when the Snaplex is deployed. We recommend a load balancer in the following use cases:

  • Snaplex-triggered Pipeline executions—Since the control plane triggering mechanism imposes additional Org-level API limits, we recommend using the Snaplex triggering mechanism for high-volume API usage.
  • REST requests to Ultra Tasks—For direct API calls to the Snaplex, the requests must pass through a load balancer. Without a load balancer, request failures occur, and the Snaplex eventually goes offline during Snaplex maintenance or upgrades.
  • API Policies—We recommend that you set up a load balancer for API policies to APIs in APIM or Triggered and Ultra Tasks in Manager for your Self-managed Snaplex. With a SnapLogic-managed Snaplex, we provision the load balancer.

You can configure the load balancer to run health checks on the node, which ensure that the node going offline for maintenance does not receive any new requests.

After configuring the load balancer, you must add the load balancer URL to the Snaplex properties.

  • For Triggered Tasks, you only need to configure the Load balancer field.
  • If a load balancer points to a FeedMaster node, then you only need to configure the Ultra load balancer field.
  • If the load balancer points to worker nodes, then you should only configure the Load balancer field in the Snaplex Settings tab.
  • The full URL of the load balancer needs to be configured in the Snaplex properties.

    For example, https://mylb.mydomain.com:443

Use cases where a load balancer is not required

Load balancers are not required for the following types of activities:

  • Pipeline executions triggered through the control plane.
  • Scheduled pipelines, pipeline and account validation, and pipeline development.
  • Always-on Ultra (since the Ultra Task processing is not driven by REST API calls).
  • Child Pipeline runs triggered through the Pipeline Execute Snap.

Cloudplex load balancer

On a Cloudplex, a load balancer is provisioned by SnapLogic, typically only when the Ultra subscription feature is enabled. The Cloudplex load balancer has a snaplogic.io domain endpoint that points to the FeedMaster nodes. You can use a load balancer for both Ultra and Snaplex-triggered Tasks.

As an Environment admin (known as Org admins in Classic Manager), you must update the Snaplex settings with the load balancer URL after the load balancer is provisioned.

Groundplex load balancer best practices

  • For a Snaplex with FeedMaster nodes, the load balancer should point to these FeedMaster nodes:

    • https://fm-node1.example.com:8084
    • https://fm-node2.example.com:8084

A FeedMaster node can process Triggered and Ultra Tasks, but a JCC node can only process Triggered Tasks. You can use the FeedMaster node as the load balancer endpoint for a Triggered Task. For an Ultra Task, you need to update the load balancer field value in the Snaplex Settings dialog.

  • If your Snaplex does not contain FeedMaster nodes, the load balancer can point to these JCC nodes:

    • https://jcc-node1.example.com:8081
    • https://jcc-node2.example.com:8081

For Triggered Tasks and Always-On Ultra Tasks, you need to update the Load balancer field value in the Snaplex settings.

  • You should configure the load balancer to run health checks on the Snaplex node on the /healthzURL. Any HTTP response code besides 200 indicates a health check failure.
  • The load balancer should perform SSL offloading/termination so that the certificate and cipher management can be done on the load balancer without updating the Snaplex nodes. The connection between the client and the load balancer is over HTTPS with your signed certificate. The connection between the load balancer and the Snaplex nodes is also over HTTPS with the default SnapLogic-generated certificate.
  • You must set the HTTP request timeout to a value of 900 or higher to allow for long-running requests. This timeout setting is different from the keep-alive timeouts that are used for connection management, like the following examples:
    • The proxy_read_timeout for Nginx.
    • The ProxyTimeout for Apache.
    • The idle timeout for AWS ELB.The following image from the AWS UI shows a sample health check configuration for the AWS ELB.
Warning: If you change the default ports of the JCC and FeedMaster nodes in your Groundplex, then you should reconfigure your load balancer to use the new port assignments, which are 8081 for JCC and 8084 for FeedMaster.