File Delete

Overview

You can use this Snap to delete a file in the SLDB, S3, FTP, SFTP, FTPS, WASB, WASBS, GS, and SMB protocols.

Note:

You must install the AzCopy utility, if you use the ABFS (Azure Blob File Storage) file protocol Azure Data Lake Gen 2 for bulk operation. The utility must be installed in Snaplex to fetch the file path. If the path is null, the native Azure Storage SDK is used for all operations. Learn more about the AzCopy command. If AzCopy Utility is not installed for ABS file transfer, the file transfer will not be as fast as using AzCopy because a REST call will be invoked for each file content instead of a bulk operation.

The SnapLogic Platform does not support the installation of utilities or processes on Cloudplexes. Learn more.

Important: We plan to introduce additional S3 features exclusively in Amazon S3 Snaps, while Binary Snaps with S3 support will not contain these updates. Therefore, we recommend you to use the Amazon S3 Snap Pack for all your S3 operations within your pipelines. However, Binary Snaps will be retained as is to maintain backward compatibility, but be aware that we will no longer provide S3 support for the Binary Snaps.

Learn more: Migrate from Binary to S3 Snaps.



Prerequisites

Connect to FTP server:

To connect to the FTP server that needs to reuse the session for data transfer over TLS protocol, add:

-DFTPS_SSL_TLS_PROTOCOL=TLSV1.2 (or) TLSV1.3property as a JVM option under the Global properties of the Node Properties tab:

Known issues

This Snap Pack does not natively support SHA1-based algorithms to connect to SFTP endpoints. With the August 2023 GA release, you can now leverage the properties specified in the Configuration settings for Snaps to add support for ones that are disabled on your Snaplex.

Snap views

View Description Examples of upstream and downstream Snaps
Input The Snap expects to find values for the JSON paths used in the File property, if any.
Output The file name and the delete status are written to the output view. The following example shows the output document map data:
{
        "file": "sldb:///sample.json",
        "status": "200 OK"
    }
Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:

  • Stop Pipeline Execution Stops the current pipeline execution when an error occurs.
  • Discard Error Data and Continue Ignores the error, discards that record, and continues with the remaining records.
  • Route Error Data to Error View Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap settings

Legend:
  • Expression icon (): Allows using pipeline parameters to set field values dynamically (if enabled). SnapLogic Expressions are not supported. If disabled, you can provide a static value.
  • SnapGPT (): Generates SnapLogic Expressions based on natural language using SnapGPT. Learn more.
  • Suggestion icon (): Populates a list of values dynamically based on your Snap configuration. You can select only one attribute at a time using the icon. Type into the field if it supports a comma-separated list of values.
  • Upload : Uploads files. Learn more.
Learn more about the icons in the Snap settings dialog.
Field / Field set Type Description
Label String

Required. Specify a unique name for the Snap. Modify this to be more appropriate, especially if more than one of the same Snaps is in the pipeline.

Default value: File Delete

Example: File Delete
File String/Expression

Required. Specify the URL for the file to be deleted. It might start with a file protocol. The supported file protocols are SLDB, S3, FTP, SFTP, FTPS, WASB, WASBS, GS, and SMB protocols.

This Snap supports S3 Virtual Private Cloud (VPC) endpoint.
Warning:

When deleting a directory, ensure to empty its contents first (delete the files within the directory and then the directory folder itself).

Ensure the file name or the file path does not contain '?' character because it is not fully supported and when present, the Snap might fail.

Note: Acceptable File Paths
  • Relative paths

    • filename.json: Saves the file in the project.

    • ../shared/filename.json: Saves the file in the Project Shared Space.

    • ../../shared/filename.json: Saves the files in the Org Shared project.

  • Absolute path
    • /<org>/<projectSpace>/<project>/filename.json
Warning: Lint Warning
The Snap displays a Lint Warning in your Pipeline in the following scenarios:
  • INCORRECT_FILE_PATH: When you provide an incorrect file path to write a file, such as not following the given relative paths pattern or absolute path patterns.
  • INCORRECT_ORG_PATH: When you create files or directories in a different org other than the one where the Pipeline is executing.

Therefore, we recommend that you confirm to any of the acceptable relative paths. Otherwise, use an absolute path—that is provide a file path that belongs to the same org where you want to write the file, or use the File Upload icon to specify the File path.

Default value: None.

Example: SLDB:

sldb:///usr/john/json_files/accounts.json

VPC:

  • s3:///snaplogic-bucket@bucket.vpce-0e340d8d5b0eae90a-k9wbom46-us-west-2a.s3.us-west2.vpce.amazonaws.com
File delete action Dropdown list
Specify the action that the Snap must take when the target file is not found during a file delete attempt. Select from the following options:

  • ERROR if file not found: The Snap writes an error to the error view.

  • IGNORE if file not found: The Snap writes the status 200 OK to the output view.

Default value: ERROR if file not found

Example: IGNORE if file not found
Validate deletion Checkbox

Select this check box to enable the Snap to check if the deleted file still exists.

Default status: Deselected
Number of retries Integer/Expression Specify the maximum number of retry attempts in case of a network failure.
Tip:

If the value is larger than 0, the Snap first downloads the target file into a temporary local file. If any error occurs during the download, the Snap waits for the time specified in the Retry interval and attempts to download the file again from the beginning. When the download is successful, the Snap streams the data from the temporary file to the downstream Pipeline. All temporary local files are deleted when they are no longer needed.

Ensure that the local drive has sufficient free disk space to store the temporary local file.

Minimum value: 0

Default value: 0

Example: 3
Retry interval (seconds) Integer/Expression Specify the minimum number of seconds for which the Snap must wait before attempting recovery from a network failure.

Minimum value: 1

Default value: 0

Example: 3
Advanced properties Use this field set to define additional advanced properties.
  • Properties
    • SAS URI
 Dropdown list Specify the URI of the Shared Access Storage (SAS) you need to access. You can generate the SAS URI either from the SAS Generator Snap or from the Azure portal → Shared access signature.
The supported SAS types are:
  • Service SAS on a container
  • Service SAS on blob
Warning:
  • Ensure that the URI is specified in the format described here.
  • If you provide SAS URI in this field, then:
    • the Primary access key given in the account settings is overridden while authentication. If you do not provide the SAS URI, the Snap considers the Primary access key in the account settings.
    • only this URL is used and the Snap ignores the SAS URI settings that you have configured in the associated account.
Warning: If the SAS URI value is provided in the Snap settings, then the settings provided in the account (if any account is attached) are ignored.

Default value: N/A

Example: https://myaccount.blob.core.windows.net/sascontainer/sasblob.txt?sv=2015-04-05&st=2015-04- 29T22%3A18%3A26Z&se=2015-04-30T02%3A23%3A26Z&sr=b&sp=rw&sip=168.1.5.60 -168.1.5.70&spr=https&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D
Values String/Expression Specify the value for SAS URI.
Snap execution Dropdown list
Choose one of the three modes in which the Snap executes. Available options are:
  • Validate & Execute: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.
  • Execute only: Performs full execution of the Snap during pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Default value: Execute only

Example: Validate & Execute

Examples