Max: 1

• JSON Formatter
• HDFS Reader
• File Reader

Max: 1

• Mapper
• HDFS Reader

• Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.
• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.
• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Required. Specify a unique name for the Snap. Modify this to be more appropriate, especially if more than one of the same Snaps is in the pipeline.

Default value: HDFS ZipFile Writer

Example: HDFS ZipFile Writer

The URL for the data source (directory). The Snap supports both HDFS and ABFS(S) protocols.

Syntax for a typical HDFS URL:

hdfs://hadoopcluster.domain.com:8020/<user>/<folder_details>

Syntax for a typical ABFS and an ABFSS URL:

abfs:///<filesystem>/<path>/
abfs://<filesystem>@<accountname>.<endpoint>/<path>
abfss:///<filesystem>/<path>/
abfss://<filesystem>@<accountname>.<endpoint>/<path>

When you use the ABFS protocol to connect to an endpoint, the account name and endpoint details provided in the URL override the corresponding values in the Account Settings fields.

Default value: [None]

The relative path and name of the file that must be created post execution.

• sample.zip
• tmp/another.zip
• $filename

Example:

Default value: [None]

Select this check box to enable user impersonation.

Default value: Not selected

Required. Use this field to specify what you want the Snap to do if the file you want it to create already exists. Available options are: Overwrite, Ignore, and Error.

• Overwrite: If the target file exists, the Snap overwrites the file.
• Ignore: If the file already exists, the Snap neither throws an exception nor does it overwrite the file, but creates an output document indicating that the new data has been ignored.
• Error: The error displays in the Pipeline Run Log if the file already exists.

Default value: Overwrite

File permission sets to be assigned to the file. To assign file permissions:

1. Click the + button against File permissions. This adds a row to the fieldset.
2. Click the Suggestible icon in the User type field and select the user type for which you want to enable access. This drop-down offers the following options:
   • Owner: This is the user account under whose name the new file will be created.
   • Group: This is the user group to which the user being impersonated belongs.
   • Others: These are all other users who have at least Read access to the concerned directory.
3. Click the Suggestible icon in the File permissions field and select the permission you want to enable for the user type selected in the User type field.

If selected, the input view label is used for all names of the files added to the zip file. Otherwise, the input view ID is used instead, when input the binary stream does not have its content-location in its header. When this option is selected, if there are more than one binary input streams in an input view, for the second input stream and after, the file names will be the input view label appended with '_n'. If the label is in the format of 'name.ext', '_n' will be append to the 'name', e.g. name_2.ext for the second input stream.

Example: If this option is selected, if Base directory is testFolder and the input view label is test.csv, the file name for the first binary input stream in that input view will be testFolder/test.csv, and the second, testFolder/test_2.csv, and the third, testFolder/test_3.csv, and so on.

Default value: Not selected

Default value: Execute only

Example: Validate & Execute