CSV file

You can use comma-delimited CSV files as a source for a data pipeline. SnapLogic® AutoSync uploads the files to temporary storage, loads them to the destination, and deletes them from temporary storage. Each source file loads into a destination table of the same name. The columns have the destination's default type for a string.

AutoSync supports the Full load and Append load types for CSV files. The first run of the data pipeline uses Full load. You can change the load type to Append by editing the data pipeline and selecting the Auto synchronize tab. Learn more about how AutoSync loads files and load types.

You cannot schedule synchronization for a data pipeline with CSV files as a source. To use the data pipeline again, select more files. The Upload CSV file action is available from the card, the three-dot menu, the list view, and the details panel.

Tip: This endpoint uploads files from a network location. You can also upload CSV files from an Amazon S3 bucket using the S3 endpoint.

Known limitations

  • Uploading or updating files in the flyout panel of an existing pipeline with CSV as a source causes an error. As a work around, upload files in the card view.

Requirements

File names cannot include single or double quotes or a special character, such as #, $, or %. Files must not be empty. To load correctly, each file must:

  • Have a header row.
  • Be uncompressed.
  • Be less than 100 MB in size.
  • Use single commas as delimiters.
  • Use quotes to enclose meaningful spaces and commas that are part of a field value. AutoSync trims leading and trailing spaces.
  • Use quotes to enclose fields that include commas in the value. For example, in a source that lists vehicles where the header specifies the year, make, model, and description columns, the quotes preserve the comma in the description column: 1997, Ford, E350, "Extended cab, luxury truck"