Deploy FeedMaster node

A FeedMaster is a specialized type of Snaplex node that manages the distribution of Ultra Task requests across the JCC nodes within the Snaplex. It operates as a Java Virtual Machine (JVM) acting as an intermediary between the JCC nodes (which execute the pipelines) and the client.

  • The FeedMaster controls and distributes Ultra Task requests to ensure efficient load balancing and task execution across JCC nodes.
  • In a Cloudplex, FeedMaster nodes are deployed by SnapLogic.
  • In a Groundplex (self-managed Snaplex), your organization sets up and manages the FeedMaster nodes.

A FeedMaster node runs on a Groundplex host and has the same hardware, operating system, and network requirements as any JCC (execution) node. Before you deploy, confirm the following:

  • To use a FeedMaster node for Ultra Task pipelines, your environment (Org) must be subscribed to the Ultra feature.
  • You must have a Groundplex. For Cloudplex FeedMasters, contact your CSM.
  • The host meets the same hardware, operating system, and network requirements as a JCC node. See Groundplex requirements: Hardware and operating systems and Groundplex requirements: Network.
  • Because a FeedMaster queues Ultra Task requests on disk, it needs additional storage beyond the JCC disk requirement. The default is 10 GB, configurable with the broker.disk_limit property. See Deployment architecture for Ultra Tasks.
  • Install the FeedMaster node physically near the Snaplex instances and the data the pipeline passes, to support low-latency processing.

The FeedMaster node uses several dedicated ports: the FeedMaster HTTPS port (default 8084), the embedded message broker port for Ultra Tasks (default 8089), and the FeedMaster HTTP port (default 8091). For the complete port list and firewall requirements, see Groundplex requirements: Network. If other resources use these ports, you can change the port number on the Node Properties tab.

A node determines its server type from its local configuration when the node process starts, before it registers with the SnapLogic Control Plane. If you start a node before it has been designated as a FeedMaster, it comes up as a regular JCC node and can begin receiving pipeline executions. Designate the FeedMaster role before you start the node for the first time (see the steps that follow).

For details about server requirements, see Install a Groundplex on Linux for Groundplex installation and Groundplex Deployment Planning for specific configurations.

Note: Load balancing multiple FeedMasters with the /healthz URL: You can configure more than one FeedMaster. In that configuration, you put a load balancer in front of the FeedMasters so that requests are automatically distributed. The load balancer uses the /healthz URL to check the health status of each FeedMaster. For load balancer requirements, see Groundplex requirements: Load balancer.
  1. In SnapLogic Manager, navigate to the target Project, then click and select Snaplex from the drop-down list.
  2. Enter the information in the Create a Snaplex dialog on the Settings tab.
  3. Click the Node Properties tab; under Snaplex node types, click , then enter the name of the hosting server in the Hostname field.
  4. From the Server type drop-down list, select FeedMaster.
    Note: Specifying a DNS host: To use a DNS host server name or other custom host name, enter that hostname in the Snaplex properties to define the FeedMaster node. The actual host name is the one returned by the hostname command on your Linux or Windows machine.
    Important:

    The node's server type is carried in its .slpropz configuration file, which packages the Snaplex global.properties. How the change reaches the node depends on whether the node already exists:

    • New host: Click Update (next step) first so that the FeedMaster server type is saved to the .slpropz file. Then download that updated .slpropz file and place it on the host before you start the node for the first time, so that it starts as a FeedMaster. For where to place the file, see Install a Groundplex on Linux or Install a Groundplex on Windows.
    • Node already running with a .slpropz configuration: Clicking Update (next step) causes each node to download the updated .slpropz and perform a rolling restart automatically; no manual file deployment is required. See Snaplex configuration changes.

    For older Snaplex installations still configured through the global.properties file (not .slpropz), Manager changes do not take effect until you migrate the nodes to the .slpropz mechanism. On such a node, designate the FeedMaster role before the first start by setting server_type=feed_master in global.properties. See Migrating older Snaplex nodes for the legacy setting and migration steps.

  5. Click Update.

    In the Monitor application, the FeedMaster node is designated by the FeedMaster Node icon. See Snaplex details panel.

Troubleshooting

  • The node starts as a regular JCC node instead of a FeedMaster. The node was started before the FeedMaster server type was applied to its local configuration. Apply the server type before the first start, as described in the Server type step above. For nodes still configured through global.properties, see Node migration.
  • The FeedMaster rejects Ultra Task requests with HTTP 503 Service Unavailable and a message such as Request storage is low, try again later. The message-spool disk is at or near its limit. Free disk space or increase broker.disk_limit. See Deployment architecture for Ultra Tasks.
  • A restarting FeedMaster keeps receiving requests. The load balancer is not health-checking the /healthz URL, so it continues routing requests to a node that is offline or restarting. See Monitor Snaplex health and Groundplex requirements: Load balancer.
  • You cannot tell which node is the FeedMaster. In the Monitor application, FeedMaster and execution (JCC) nodes use different icons. See Monitor FAQs.
  • You need to investigate FeedMaster errors. Check the FeedMaster log files (feed_master.json, feed_master_error.json, and related logs). See Snaplex logs.