This example demonstrates how to use a Mapper Snap to transform and prepare data before inserting it into a Redshift table using the Redshift - Insert Snap. This pattern is useful when you need to map fields, apply transformations, or use pipeline parameters to dynamically configure the insert operation.
The pipeline uses a Mapper Snap to transform input data, mapping fields from the source format to the target Redshift table format, before passing the data to the Insert Snap.
- A valid Redshift account
- A Redshift table to insert data into
- Understanding of SnapLogic expressions and pipeline parameters
- Appropriate permissions to insert records into the target table
-
Configure the Mapper Snap to transform input data.
Map fields from the source format to the target Redshift table format. You can:
- Rename fields to match the target column names
- Apply transformations using expressions
- Add computed fields or constants
- Filter or restructure the data
For example, if your source has employee_id but the target table uses empno, create a mapping:
- Target path:
$empno
- Mapping expression:
$employee_id
-
Configure the Redshift - Insert Snap.
Set the following properties:
- Schema name: Specify the database schema (for example,
public).
- Table name: Specify the table name. You can use pipeline parameters for dynamic table names. For example:
$_table_name
Using Pipeline Parameters:
You can pass the table name and other properties as pipeline parameters, making your pipeline reusable across different execution contexts. For example:
- Set Table name to
$_table_name
- Define the pipeline parameter
_table_name with a default value or pass it at runtime
-
Execute the pipeline.
When the pipeline runs, the Mapper transforms the input data and passes it to the Insert Snap, which inserts the records into the specified Redshift table.
The Snap successfully inserts transformed records into the Redshift table. The output shows the status of each insert operation along with the original input data.
Note: Using the Mapper Snap allows you to decouple your data source format from the target table structure, making your pipelines more flexible and maintainable.
