Remove columns using the Mapper Snap

This example demonstrates how to remove unwanted columns from input data using the Mapper Snap.


Pipeline for removing unwanted columns

Download this pipeline
  1. Configure the JSON Generator Snap
    Use the JSON Generator Snap to generate structured JSON data that includes the unwanted columns.
  2. Add a Mapper Snap to the pipeline
    Use the Mapper Snap to remove the columns not needed in the downstream process.
  3. Configure the Mapper Snap

    Select the Pass through checkbox to retain all fields except those removed explicitly.

    In the Expression field, enter the following expression to remove specific columns:

    $.filter((value, key) => !key.match("column_to_remove1|column_to_remove2"))

    Set the Target path field to $ to retain all other columns as is.


    Mapper Snap Configuration

    Validate the pipeline to confirm the unwanted columns are removed successfully.

  4. Add a MySQL Insert Snap (Optional)
    If writing to a MySQL database, use the MySQL - Insert Snap to load the output data.

The output contains only the required columns, excluding those removed in the Mapper Snap configuration.