Snaplex configuration changes
Configure Snaplex nodes
Snaplex nodes are typically configured using the slpropz configuration file located in the $SL_ROOT/etc folder.
If you use the slpropz file as your Snaplex configuration, then you can expect the following:
-
After a Snaplex node is started with the
slpropzconfiguration, subsequent configuration updates are applied automatically. - Changing the Snaplex properties in Admin Manager causes each Snaplex node to
download the updated
slpropzand do a rolling restart. If pipelines are still running, the node waits for the pipeline executions to complete before the Max. restart wait time expires. -
Some configuration changes, such as an update to the logging properties, do not require a restart and are applied immediately.
-
Some configuration properties, like the Environment value, cannot be changed without doing manual updates for the
slpropzfiles on the Snaplex nodes.
The Groundplex nodes must be set up to use a slpropz configuration file before changes to these properties take effect. If you change that affect the software configuration, but there are nodes in the Snaplex that are not set up to use a slpropz configuration file, a warning dialog appears with a listing of the unmanaged nodes.
Restart Groundplex nodes
To restart the Snaplex process on your Groundplex nodes, run the following commands, depending on your OS:
Linux: /opt/snaplogic/bin/jcc.sh restart
Windows: c:\opt\snaplogic\bin\jcc.bat restart.
Manage older Groundplex installations
If you have an older Snaplex installation with a configuration defined in the global.properties file, then the Environment value must match the jcc.environment value In the JCC global.properties file. To migrate your Snaplex configuration to the slpropz mechanism, see Migrating Older Snaplex Nodes.
You should always configure your Snaplex instances using the slpropz file because you do not have to edit the configuration files manually. Changes to the Snaplex done through Manager are applied automatically to all nodes in that Snaplex.
Customize version updates
When the Snaplex service is started, two Java processes are started. First is the
Monitor process, which then starts the actual Snaplex process. The monitor keeps
track of the Snaplex process state, restarting it in case of failure. The Snaplex
by default upgrades itself to run the same binaries that are running on the
SnapLogic cloud. The monitor process continues running always, running from the
binary which was used when the Monitor was originally started. Running
jcc.sh restart or jcc.bat restart restarts
both the monitor and the Snaplex processes.
In case a custom Snaplex patch is provided for a customer, it is provided as a
jcc.war file. If this is copied into the
/opt/snaplogic/run/lib directory and the JCC node is
restarted, the JCC again downloads the latest version of the war file. To prevent
the Snaplex from going back to the default binaries, you can set up the Snaplex to
disable downloading the current version from the SnapLogic cloud. To do this, check
the value of build_tag as returned by https://elastic.snaplogic.com/status. For example, if the build tag is
mrc27, adding the following line in
etc/global.properties prevents the Snaplex from downloading
the mrc27 version from the cloud:
jcc.skip_version = mrc27
When the next release is available on the cloud and the custom patch is no longer required, the Snaplex automatically downloads the next version.
The download of latest binaries and automatic updates can be disabled on the
Snaplex. If so, the next time there is a new version available on the SnapLogic
cloud, the Snaplex is no longer usable since there would be a version mismatch. The
automatic download would have to be re-enabled for the Snaplex to be usable again.
To avoid such issues, the jcc.skip_version approach above is the
recommended method to run with a custom version of the Snaplex binaries.
To prevent the new binaries from being downloaded, set (default is True):
jcc.enable_auto_download = False
To disable the automatic restart of the Snaplex after new binaries are downloaded, set (default is True):
jcc.auto_restart = False
Temporary folder
The temporary folder stores unencrypted data. These temporary files are deleted when the Snap/Pipeline execution completes. You can update your Snaplex to point to a different temporary location in the Global properties table by entering the following:
jcc.jvm_options = -Djava.io.tmpdir=/new/tmp/folder
The following Snaps write to temporary files on your local disk:
-
Anaplan: Upload, Write
-
Binary: Sort, Join
-
Box: Read, Write
-
Confluent Kafka: All Snaps that use either Kerberos and SSL accounts
-
Database: When using local disk staging for read-type Snaps
-
Data Science (Machine Learning) Snaps: Profile, AutoML, Sample, Shuffle, Deduplicate, Match
-
Email: Sender
-
Hadoop: Read, Write (Parquet and ORC formats)
-
JMS: When the user provides a JAR file
-
Salesforce: Bulk Query, Snaps that process CSV data
-
Script: PySpark
-
Snowflake: When using internal staging
-
Teradata: TPT FastExport
-
Transform: Aggregate, Avro Parser, Excel Parser, Join, Unique, Sort
-
Vertica: Bulk Load
-
Workday Prism Analytics: Bulk Load