Overview

You can use the File Writer Snap to read a binary data stream from the input view and write it to a specified file destination. Possible file destinations include: SLDB, HTTP, S3, FTP, SFTP, FTPS, or HDFS. If you provide file permissions for the file, the Snap sets those permissions to the file.

• Write-type Snap
• Works in Ultra Tasks

Prerequisites

IAM Roles for Amazon EC2

The 'IAM_CREDENTIAL_FOR_S3' feature is used to access S3 files from EC2 Groundplex, without Access-key ID and Secret key in the AWS S3 account in the Snap. The IAM credential stored in the EC2 metadata is used to gain access rights to the S3 buckets. To enable this feature, set the Global properties (Key-Value parameters) and restart the JCC:jcc.jvm_options = -DIAM_CREDENTIAL_FOR_S3=TRUE

This feature is supported in the EC2-type Groundplex only. Learn more.

Connect to FTP server:

To connect to the FTP server that needs to reuse the session for data transfer over TLS protocol, add:

-DFTPS_SSL_TLS_PROTOCOL=TLSV1.2 (or) TLSV1.3property as a JVM option under the Global properties of the Node Properties tab:

Limitations

Known issues

Snap views

{
        "filename": "ftp://ftp.Snaplogic.com/home/qatest/user_manual.pdf",
        "result": "overwritten",
        "original": {
            "content-type" : "application/json"
        }
    }

Snap settings

Syntax for writing files

Acceptable File Paths

• Relative paths
  • filename.json: Saves the file in the project.
  • ../shared/filename.json: Saves the file in the Project Shared Space.
  • ../../shared/filename.json: Saves the files in the Org Shared project.
• Absolute path

  • /<org>/<projectSpace>/<project>/filename.json

Writing Files in SLDB

File Patterns

Reading files from Project and Shared Project Spaces

• If a Pipeline is created in a project other than the shared project and you want to read the "asset.json" file from the same project, enter "asset.json" or "sldb:///asset.json".
• If a Pipeline is created in the shared project and you want to read the "asset.json" file from the shared project, enter "asset.json" or "sldb:///asset.json".
• If a Pipeline is created in a project other than the shared project and you want to read the "asset.json" file from the shared project, enter "shared/asset.json" or "sldb:///shared/asset.json".
• Ensure the file name, folder name, or the file path does not contain '?' character because it is not fully supported and when present, the Snap might fail.

Encoding of characters in a URL

Following are some of the common characters that are automatically encoded.

Following are some of the characters that are not automatically encoded by the Snap:

Video Demonstration

Additional Information

You can also access the fields in a binary header when specifying a file name. For example, if you have a File Reader Snap upstream of a File Writer Snap, you can access the "content-location" header field to get the original path of the file. You can then use a new file name based on the old one, for instance, to make a backup file:$['content-location'].match('/([^/]+)$')[1] + '.backup'

For http: and https: protocols, the Snap uses http PUT method only. This property should have the following syntax:

[protocol]://[host][:port]/[path]

Troubleshooting

Examples

• File Writer - HDFS File Access
• SLDB File Write