CSV Formatter

Overview

You can use this Snap to read a document at the input, format it as CSV based on the specified parameters, and write CSV data to the output.


Overview

Prerequisites

None.

Known issues

  • The Snap ignores escape characters when used along with any quote characters. When this happens, the resultant data cannot be parsed correctly using CSV Parser Snap which can cause an error. See Apache CSV Format Issue for details.

  • The Snap prepares the column header for the output view (and hence the CSV file) using the keys defined in the first record. This may result in ignoring any additional keys passed in the subsequent records. We recommend that you pass values for a comprehensive set of all keys used in the input view, for the first record.

Snap views

View Description Examples of upstream and downstream Snaps
Input Any document that contains data structured as key-value attributes.

Format: Document
Output CSV data formatted using specifications provided in the Snap's settings.

Format: Document
Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:

  • Stop Pipeline Execution Stops the current pipeline execution when an error occurs.
  • Discard Error Data and Continue Ignores the error, discards that record, and continues with the remaining records.
  • Route Error Data to Error View Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap settings

Legend:
  • Expression icon (): Allows using pipeline parameters to set field values dynamically (if enabled). SnapLogic Expressions are not supported. If disabled, you can provide a static value.
  • SnapGPT (): Generates SnapLogic Expressions based on natural language using SnapGPT. Learn more.
  • Suggestion icon (): Populates a list of values dynamically based on your Snap configuration. You can select only one attribute at a time using the icon. Type into the field if it supports a comma-separated list of values.
  • Upload : Uploads files. Learn more.
Learn more about the icons in the Snap settings dialog.
Field / Field set Type Description
Label String

Required. Specify a unique name for the Snap. Modify this to be more appropriate, especially if more than one of the same Snaps is in the pipeline.

Default value: CSV Formatter

Example: CSV_Formatter_temp
Binary header properties

Use this field set to add binary-header properties to the output data. These properties contain data related information that enables the system to interpret data.

Binary headers in a document can be accessed and used in the expression-enabled properties of downstream Snaps. For example, you can use binary headers to specify custom statuses associated with the output data.

Similarly, a 'content-location' property added to the binary header in this Snap can be referenced in the File name property of a File Writer Snap with the expression: $['content-location'].
Note: Expression-enabled fields in Binary header properties are evaluated against the first input document only.
Expression String

Specify the value to be associated with a specific binary header property.

Default value: N/A

Example: text/csv
Target Path String/Suggestion

Specify the target JSON path where the value in the expression is written.

Default value: N/A

Example: $['content-location']
Quote character String/Expression

The character that you want to use as the escape character in the CSV document. For example, if you use double quotes (") as the escape character, then commas in the actual data will need to be escaped using double-quotes on both sides.

This property can be an expression. However, if the value associated with the expression contains more than one character, only the first character is used as the quote character.

Default value: "

Example: "
Quote mode Dropdown list
Select an option to specify how the quote character should be used in formatting the CSV data. Available options are:
  • ALL: All values are quoted.
  • MINIMAL: Values are minimally quoted.

    For example, if a value includes a delimiter character, only that character is enclosed in the quote character.

  • NON_NUMERIC: All values except numbers are quoted. If values in the input documents are number objects, they are not quoted. If they are numeric strings, they are quoted.

  • NONE: No value is quoted. If a value includes a delimiter character, it is escaped with a '\' (backslash).

    Note: If the Quote character property is empty, the selection of this property is ignored and the Snap uses NONE for the Quote mode.

Default value: ALL

Example: MINIMAL
Delimiter String/Expression Required. Specify the string or the character to be used as a delimiter in formatting the delimited data. Any combination of characters may be used, adhering to the following guidelines.

The input must be submitted with any control characters escaped.

For example, \t (tab), \n (new line), or \\ (single backslash) must be escaped accordingly. Unicode characters must be specified using the Unicode escape sequence \uXXXX, where each X represents a hexadecimal digit (0-9, a-f) with all four hexadecimal digits defined.

Important: When using a single backslash (\) as a delimiter, it does not need to be escaped (\\). However, if you are using a multi-character delimiter that contains one or more backslashes (\), you must escape all backslashes (\\).

Default value: ,

Example:

  • \t

  • \u0001
Use header Checkbox

Select this checkbox to indicate whether the column names in the Header fields property should be used to format the CSV data. If this checkbox is deselected, the key set of the first document data is used as a CSV header.

Important: Ensure that the CSV document header contains non-empty values for all columns.

Default status: Deselected
Header fields Use this field set to define the header values you want to use in the CSV output data.
Field names String

Specify the field name for the header.

Default value: N/A

Example: $name
Write CSV header Checkbox Select this checkbox to indicate whether the header strings listed in the Header fields properties should be written to the output CSV data.

Default status: Selected
Header size error policy Dropdown list Select an option to handle any header size errors.

  • If the Use header field is not selected, then the Snap determines the CSV header from the key set of the first input document.

  • If the Write CSV header field is selected, then the Snap writes the CSV header to the output view binary stream.

The header size error condition occurs when any subsequent input document has additional column names which are not present in the header. To handle header size errors, you can select any of the following options:
  • Default: The Snap appends the additional values to the CSV line.
  • Fail if record is larger than header: The Snap writes the error document to the error view and continues to process the next input document.
  • Trim record to fit header: The Snap writes the trimmed CSV line to the output view and ignores all additional values in the input document.

Default value: Default

Example: Fail if record is larger than header
Ignore empty stream Checkbox This checkbox determines whether the Snap must or not ignore empty streams received at the input view during pipeline execution.

Select this checkbox to ignore empty input data and not produce any output stream.

Deselect this checkbox to write an empty array to the output stream.

Default status: Deselected
Character set Dropdown list Select the character set in which the input CSV data is encoded. The available options are:

  • UTF-8

  • UTF-16LE

  • UTF-16BE

  • ISO-LATIN-1

  • CESU-8

Default value: UFT-8

Example: UTF-16LE
Write BOM Checkbox This checkbox determines the behavior of the Snap when it starts to write the CSV output data.

Select this checkbox to write the BOM (Byte Order Mark) for the character set selected in the Character-set encoding property; else, the Snap skips writing BOM.

Default status: Deselected

Example: Selected
Newline Dropdown list Select an option to specify the newline characters that you want to use as a line break. The available options are: LF, CR+LF and CR.

Default value: LF

Example: CR + LF
Escape character String/Expression Specify the escape character that is to be used when formatting rows. Only single characters are supported. You can also use the expression enabler to enter a value, which is evaluated with the values from the pipeline parameters.
Note: Leave this field empty if no escape character is used in the input CSV data.

Default value: \

Example: \
Snap execution Dropdown list
Choose one of the three modes in which the Snap executes. Available options are:
  • Validate & Execute: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.
  • Execute only: Performs full execution of the Snap during pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Default value: Execute only

Default value: Validate & Execute

Examples