Groundplex deployment plan

Planning a Groundplex deployment

Several factors have to be considered when deploying a Groundplex (self-managed Snaplex). Most of them are IT requirements in your computing environment however some also depend on the types of pipelines you plan to run in production.

To install a Groundplex in any environment check the following requirements:

In addition to the listed requirements, ensure that the following requirements are also considered.

Groundplex name and associated nodes

Every Snaplex requires a name, for example, ground-dev or ground-prod. In the SnapLogic Designer, you can choose the Snaplex where pipelines are executed.

Your nodes are associated with a Groundplex through the Environment variable: For example, dev or prod. When you configure the nodes for your Groundplex, you must set the jcc.environment to the environment value that you provided in the Create Snaplex page.

Important: The host name of the system used by a Groundplex cannot have an underscore (_) in its name as per DNS standards. Avoid special characters as well.

You can change this variable in the Update Snaplex page. After the Snaplex service is started on a node, the service connects to the SnapLogic Cloud service. Runtime logs from the Snaplex are written to the following folder:

Linux: /opt/snaplogic/run/log
Windows: c:\opt\snaplogic\run\log

The Monitor shows the currently connected nodes for each Snaplex.

Distribution of data processing across Snaplex nodes

When a pipeline or task is executed, the work is assigned to one of the JCC nodes in the Snaplex. The scheduling of pipelines across nodes in a Snaplex is based on an algorithm that is least-loaded, with priority on memory usage. If there are similarly loaded nodes, the algorithm randomizes the pipeline execution across them.

Tip: To ensure that requests are being shared across JCC nodes, we recommend that you set up a Load balancers to distribute the work across JCC nodes in the Snaplex.

Node cluster

Starting multiple nodes with the JCC service pointing to the same Snaplex configuration automatically forms a cluster of nodes if you follow these requirements for nodes in a Snaplex:

The nodes need to communicate with each other on the following ports: 8081, 8084, and 8090.
The nodes should have a reliable, low-latency network connection between them.
The nodes should be homogeneous in that they should have the same CPU and memory configurations, as well as access to the same network endpoints.
All JCC nodes should be the same size. All FeedMaster nodes should be the same size for load balancing. Worker and FeedMaster nodes can be of different sizes.

Node diagnostics

Snaplex diagnostics helps you verify your Snaplex host environment and troubleshoot any issues. Each Snaplex node is a JCC instance running on a host, and the node diagnostic test highlights the hardware and thread limits requirements. It checks for minimum hardware requirements such as RAM and disk storage. The details of the test are listed in the table below. For more information on how to view the node details panel, refer to Node details panel.


Diagnostic test	Recommended value	Examples of current value displayed in the diagnostic test
Nodes have insufficient swap space	If the maximum value is not present, the system displays the value of 50% of the RAM configured or 8 GB, whichever is greater.	If the value is not as per the recommended value the current value is displayed in red. Example: 1 GiB
Max Slots	If there is no minimum value, then the recommended value is calculated as follows: RAM configured or 8 GB) * 2000 (max value) rounded to the nearest 500.	Example: If the maximum value is 3840, the current value displayed is 4000
Thread limit	Minimum value = 65000	Displays the thread limit in red if the value is below the recommended value. Example: 4000
Max file descriptors	If there is no maximum value and the minimum value is 65000, then the recommended value should be 65000.	Example: 65535
Max jvm heap	The minimum and the recommended value is calculated as follows: RAM configured *.85 Minimum value = 12 GiB Recommended value = 12 GiB	Example: 12.44 GiB
RAM configured	Minimum value = 4 GiB Recommended value = 4 GiB	Example: 16.32 GiB
RAM available	More than 15 minute period per day where memory utilization is > 75% or average memory utilization is > 60%	Example: 2.78%
Disk storage configured	Minimum value: 40 GiB Recommended value: 100 GiB	Example: 39.98 GiB

JCC node communication requirements

Each JCC node publishes its IP addresses to the control plane. DNS is not required for communication between nodes. We recommend setting up all the nodes inside a Snaplex in the same network and data center. Communication between JCC nodes in the same Snaplex is required for the following reasons:

The Pipeline Execute Snap communicates directly with neighboring JCC nodes in a Snaplex to start child pipeline executions and send documents between parent and child pipelines.
The data displayed in Data Preview is written to and read from neighboring JCC nodes in the Snaplex.
A FeedMaster node distributes Ultra Task requests among all JCC nodes in a Snaplex.
A Ground Triggered Task (invoked from a Groundplex) can be executed on a neighboring JCC node because of load-balancing, in which case, the Pipeline prepare request, and the bodies of the request and response, are transferred between nodes.

Any extra latency or network hops between neighboring JCC nodes can introduce performance and reliability problems.