Merge columns

The Merge columns transformation concatenates columns or fields of type string. Merging columns can simplify complex data structures and result in more efficient storage and queries. Merge any number of columns, select a separator, and name the new concatenated column. After defining a merge, you can use it in another merge.

The Merge columns screen opens after Replace empty values in the data pipeline wizard. Only columns of type string show in the Merge columns data preview table. The data preview reflects all transformations applied in the earlier screens. For example, columns removed in the Filter columns screen aren't visible in the data preview table. To concatenate a filtered column, you need to go back to the Filter columns screen and add it back. You can have AutoSync remove the column after concatenation if you choose to filter the original columns.

A space is the default delimiter. Available separator options include: comma and space, hyphen, custom, or none. To enter a custom separator, select Custom and enter the value directly in the field. A custom separator can be up to five characters, including spaces.

You can also edit an existing data pipeline with the Merge columns option in the Transformations tab.

Target column precision

When you merge two columns, the target column must be large enough to accommodate the merged data. Column precision determines the maximum length of string values. Some source columns don't have a precision setting, have a null value for precision, or when merged, would have a larger precision than the target.

For the columns selected for a merge transformation:

  • When any of the source columns don't have a set precision, AutoSync sets the new column to the target's maximum precision.
  • When all source columns have a set precision, AutoSync uses their value up to the target's maximum precision.

Configure a merge in the wizard

To merge columns in the data pipeline wizard:

  1. In the Merge columns screen, click the columns to concatenate.
    A new column overlays the table with a merge preview:
    Merge columns two columns selected

  2. Optionally, drag the column tokens to change their order.
  3. Enter a name that satisfies any target endpoint restrictions.
  4. Optionally, select Filter original columns after merge to avoid loading the original columns to the target.
  5. Pick a separator from the dropdown list.
  6. Click Save Merge. The preview shows the new column and a table below the preview displays a summary of the merge:
    Merge columns finished

  7. When you finish defining merges, click Next.